Testing a Code-Savvy Open-Source LLM: From Prompt Writing to Script Deployment, Tool Integration, and On-Premises Deployment
This article documents a hands-on coding test of an open-source LLM with strong code-generation abilities, using a disk space analysis and cleanup tool as the running example. It walks through the full path from breaking down requirements and designing structured prompts, to reviewing and hardening the generated script, integrating a model service, and choosing between cloud API and on-premises deployment. Along the way it shows how streaming output, resume-on-disconnect, observability, and layered deployment turn an AI's raw speed into a stable, maintainable engineering workflow, and closes with a security review checklist for AI-generated code.
1. Introduction: It Started with a Disk-Space Emergency
A developer shared an experience worth unpacking. When his local disk filled up, he typed a single sentence into a large language model. The model produced a runnable disk space analysis and cleanup program, and it worked well. He praised the model’s coding ability: it did not spin in long-winded reasoning while delaying real output. His only regret was that when connecting to a certain third-party model service, output was slow and long-lived connections occasionally dropped. Deploying it locally, he felt, would make the experience even better.
The story is representative. Nearly every backend or operations engineer has faced the tense moment of a disk running out of space. It also highlights two core concerns when using LLMs to assist development:
- Whether the model’s output is “to the point”: can it deliver usable code quickly, rather than circling in drawn-out reasoning?
- Whether the inference service is stable: is the speed sufficient for high-frequency coding work, and are the connections reliable?
These impressions are worth expanding on, but this article’s focus goes beyond “does it feel good.” A script produced from a single sentence still needs careful scrutiny for its logic, edge cases, and safety. Below, we approach the problem from five angles: requirements breakdown, prompt design, code review, model integration, and deployment selection. Together they form a complete path from “firefighting” to “long-term maintainability.”
2. Requirements Breakdown: What Should a Disk Space Analysis and Cleanup Tool Solve?
Before designing the tool, we need to define the requirements clearly. A full disk is never a single problem; it is usually caused by a handful of common situations:
- Runaway logs: the application lacks rotation and archival policies, so a single log file keeps growing.
- Accumulating temporary artifacts: build caches, temp files, and crash dumps are never reclaimed.
- Deleted files still held open: a file is removed from the directory, but a process still holds the handle, and the space is not freed until the process exits or restarts.
- Insufficient capacity planning: partitions gradually fill up as the business grows, with no early warning.
With that in mind, disk space analysis typically has to answer three questions:
- Where is the space going: how usage is distributed across partitions and directories, which files are the largest, and how file types are distributed.
- What can be safely cleaned up: temp files, expired logs, old build artifacts, historical dumps, and so on.
- How to prevent recurrence after cleanup: through monitoring alerts and log rotation, catching the problem before it happens.
From an engineering standpoint, a qualified disk analysis and cleanup tool should have the following capabilities:
| Capability | Key requirement |
|---|---|
| Scan scope | Support specified directories and mount points; avoid double-counting across filesystems; skip symlinks to prevent loops |
| Output format | Support both human-readable text and machine-readable formats (such as JSON) for further processing |
| Permission handling | Skip and record directories that cannot be accessed; do not abort the whole scan because of a single error |
| Cleanup policy | Filter by file type, modification time, retention days, and so on; support a quota limit |
| Safety mechanisms | Preview only by default; provide allow/deny lists, dry-run mode, and an operation log |
| Extensibility | Support scheduled runs, reporting results to monitoring/alerting, and batch execution across multiple hosts |
Once these requirements are clear, looking at the model-generated code lets you quickly judge whether it is usable, convenient, and safe enough to rely on.
3. Prompt Design: Turning “One Sentence” into an Executable Specification
The success of “generate a program from one sentence” comes from a good match between the model’s coding ability and the developer’s mental model, not from luck. To reproduce this effect reliably, the key is to give the prompt a clear structure. A practical template looks like this:
[Role] Senior systems engineer[Goal] Write a disk space analysis and cleanup script[Runtime] Linux server; standard library only, no third-party dependencies[Functional requirements]1. Analysis mode: output partition usage, top-level subdirectory usage ranking, Top N largest files, and file-type distribution;2. Cleanup mode: filter by directory, file extension, and last-modified time; delete only logs and temp files past their retention period.[Safety constraints] Preview only by default, no deletion; deletion requires an explicit confirmation flag;do not follow symlinks or cross mount points; skip directories with no access and report them; write an operation log before deleting.[Output format] Provide the complete code first, then list the parameter descriptions and typical usage examples.
This template captures several key points:
- Role setting: establish a professional perspective so the model prioritizes correctness and safety rather than just “getting it to run.”
- Constraints up front: the runtime, dependency limits, and safety boundaries directly shape the generated code.
- Bulleted features: breaking vague requirements into checkable items makes point-by-point verification easy.
- Output format agreement: specifying the output structure reduces the room for the model to drift into explanatory tangents.
On top of this, you can also use a “plan first, then code” strategy:
- In the first round, ask the model only for a design approach: scan strategy, cleanup conditions, and safety boundaries.
- A human confirms there are no omissions or dangerous designs.
- In the second round, have it generate the complete code following the confirmed plan.
This approach sharply reduces the risk of “generate once, accept everything,” and makes it easier for the code to fit your team’s existing conventions and habits.
4. Code Example: A Usable Skeleton for Disk Analysis and Cleanup
For clarity, the following uses Python-like pseudo-code to show a typical implementation skeleton. The same ideas apply to other languages.
4.1 Disk Space Analysis
import osimport heapqfrom collections import Counterimport shutilfrom pathlib import Pathdef human_size(num_bytes: int) -> str:"""Convert a byte count into a human-readable format"""for unit in ("B", "KB", "MB", "GB", "TB"):if abs(num_bytes) < 1024.0:return f"{num_bytes:.1f} {unit}"num_bytes /= 1024.0return f"{num_bytes:.1f} PB"def analyze(root: str, top_n: int = 20):"""Analyze space usage of a directory: top-level subdirectory ranking + Top N largest files + file-type distribution"""root = os.path.abspath(root)mount_point = os.stat(root).st_dev # record the starting device ID to avoid double-counting across mount pointsdir_totals = Counter() # usage per top-level subdirectorytop_files = [] # min-heap of (size, path)type_totals = Counter() # usage per file typeskipped = [] # directories skipped due to permission or read errorsfor dirpath, dirnames, filenames in os.walk(root, onerror=lambda e: skipped.append(e.filename)):# in-place filter: do not follow symlinks, do not cross mount pointsdirnames[:] = [d for d in dirnamesif not os.path.islink(os.path.join(dirpath, d))and os.stat(os.path.join(dirpath, d)).st_dev == mount_point]for name in filenames:path = os.path.join(dirpath, name)try:if os.path.islink(path):continuesize = os.stat(path).st_sizeexcept OSError:skipped.append(path)continue# attribute to a top-level subdirectoryrel = os.path.relpath(path, root)top_dir = rel.split(os.sep)[0] if os.sep in rel else "."dir_totals[top_dir] += size# group by extensionext = os.path.splitext(name)[1].lower() or "(no extension)"type_totals[ext] += size# maintain a min-heap of size top_n; the heap root is the top_n-th largest fileitem = (size, path)if len(top_files) < top_n:heapq.heappush(top_files, item)elif size > top_files[0][0]:heapq.heapreplace(top_files, item)print("== Top-level directory usage ranking ==")for name, size in dir_totals.most_common(10):print(f"{human_size(size):>10} {name}")print("\n== Top N largest files ==")for size, path in sorted(top_files, reverse=True):print(f"{human_size(size):>10} {path}")print("\n== File-type usage ==")for ext, size in type_totals.most_common(10):print(f"{human_size(size):>10} {ext}")if skipped:print(f"\nNote: {len(skipped)} paths were skipped due to permission or read errors")def show_partitions(paths):"""Show capacity and free space for each partition"""for p in paths:total, used, free = shutil.disk_usage(p)print(f"{p}: used {human_size(used)} / total {human_size(total)}, free {human_size(free)}")
Two details here are worth learning from:
- Top-level aggregation: summing by top-level subdirectory before showing the ranking lets you pinpoint the “big consumers” faster than flatly traversing every file.
- Top N via a min-heap: maintain a min-heap of size N whose root is always the current N-th largest file. A new file replaces the root as long as it is larger, giving O(n log N) time complexity and constant memory overhead, so scanning millions of files does not consume much memory.
4.2 Safe Cleanup: Preview by Default, Deletion Requires Explicit Confirmation
import osimport timeimport loggingfrom pathlib import Pathdef cleanup(targets,suffixes,min_age_days: int = 7,max_total_mb: int = 5120,white_dirs=None,black_dirs=(),execute: bool = False,log_file: str = "cleanup.log",):"""Clean up files by extension and retention period; when execute=False, preview only and delete nothing"""white_dirs = {os.path.realpath(d) for d in (white_dirs or [])}black_dirs = {os.path.realpath(d) for d in black_dirs}cutoff = time.time() - min_age_days * 86400candidates = []total_size = 0for target in targets:base = os.path.realpath(target) # resolve the real path to prevent bypassing restrictions via ../ or symlinksif base in black_dirs:print(f"[SKIP] blacklisted directory: {base}")continueif white_dirs and base not in white_dirs:print(f"[SKIP] not in whitelist: {base}")continueif not os.path.isdir(base):continuefor dirpath, dirnames, filenames in os.walk(base):# prune: stop descending before entering blacklisted subdirectoriesdirnames[:] = [d for d in dirnamesif not os.path.islink(os.path.join(dirpath, d))and os.path.realpath(os.path.join(dirpath, d)) not in black_dirs]for name in filenames:path = os.path.join(dirpath, name)if os.path.splitext(name)[1].lower() not in suffixes:continuetry:st = os.stat(path)except OSError:continueif st.st_mtime >= cutoff or st.st_nlink != 1: # only handle regular files, ignore hard linkscontinuecandidates.append((st.st_size, path))total_size += st.st_sizeif total_size >= max_total_mb * 1024 * 1024:breakcandidates.sort(reverse=True)print(f"Matched {len(candidates)} files, total {human_size(total_size)}")for size, path in candidates[:50]:print(f"[{'DELETE' if execute else 'PREVIEW'}] {human_size(size):>10} {path}")if not execute:print("Currently in preview mode; nothing was deleted. Add --execute after confirming.")returnlogging.basicConfig(filename=log_file, level=logging.INFO,format="%(asctime)s %(levelname)s %(message)s")freed = 0for size, path in candidates:try:os.remove(path)freed += sizelogging.info("Deleted %s (%s bytes)", path, size)except OSError as exc:logging.warning("Delete failed %s: %s", path, exc)print(f"Freed {human_size(freed)}")
The safety design here includes:
- No deletion by default: without an explicit execution flag, the script only prints a preview.
- Path normalization: use
realpathto resolve the real path, preventing bypass of the whitelist via relative paths or symlinks. - Allow and deny lists: narrow the permitted cleanup scope to explicit directories, and prune during traversal to avoid wandering into unrelated directories.
- Three-way filtering: a file becomes a candidate only when the extension, last-modified time, and regular-file checks all pass.
- Quota limit: cap the total volume of a single cleanup run to avoid deleting too much in one operation.
- Operation trail: every deletion is written to a log, making audits and traceability easy.
4.3 How It Differs from LLM-Generated Code
A model-generated script is often “usable” but not necessarily “trustworthy.” Common differences and ways to harden them include:
- File-type filtering is too loose: if it decides only by “log directory” without checking the extension, a misconfiguration or a wrongly passed path could delete source code and data files. Add extension and suffix validation.
- No hard-link protection: when a file has multiple hard links, deleting one does not free space and can make expectations diverge from reality. Check the link count when necessary.
- No quota limit: without a single-run cap, cleaning a very large directory can cause irreversible loss.
- Insufficient exception granularity: some model code wraps the whole loop in a single
try/except, so one failed file read aborts the entire scan. Catch exceptions at the per-file level instead.
Only after each of these is reinforced is the script truly ready to run in production.
5. Inference Pace and Performance: Why “Not Verbose” Is an Important Metric
One easily overlooked bright spot in the developer’s experience is that the model did not have the problem of “thinking extremely, extremely long.” From an engineering standpoint, this matters in four ways:
- Lower time to first token: the wait from issuing a command to seeing code is noticeably shorter, so attention is less easily broken.
- Smaller context usage: the longer the reasoning, the more of the context window it consumes, and the more it crowds out useful information.
- More controllable time and cost: when billed by output length, verbose reasoning inflates the cost of a single call.
- Higher predictability: for tasks like code generation with clear rules and well-defined goals, lengthy reasoning does not necessarily deliver a comparable quality improvement.
That said, this does not mean shorter reasoning is always better. Different tasks need different reasoning depths:
| Task type | Characteristics | Recommendation |
|---|---|---|
| Script and tool writing | Clear rules, deterministic output | Favor low latency and high throughput |
| Unit tests and code completion | High frequency, short output | Prioritize response speed and consistency |
| Tricky debugging, architecture design | Requires multi-step reasoning and trade-offs | Optionally use a model with adjustable reasoning depth |
For coding assistants, response speed directly affects the experience: slow output breaks your train of thought, and a dropped long connection forces you to start over. So when evaluating whether a model suits a coding scenario, look at:
- Time to first token: the time from sending a request to receiving the first content.
- Output rate: the content generation speed during sustained generation.
- Long-request stability: whether long tasks, such as generating thousands of lines of code, can complete smoothly.
6. Service Integration Layer: Handling Speed Fluctuations and Dropped Long Connections
Experience is one thing; engineering must face another class of problems. With a third-party model service, output rate fluctuates with server load, long connections occasionally drop, and clustered calls at peak times more often hit rate limits. Below are several practical strategies for the integration layer.
6.1 Streaming Output: Make the Wait Perceptible
With streaming output enabled, content is returned as it is generated. Users can immediately watch the code being “written” line by line, which sharply lowers the perceived wait, and the frontend can render as it receives rather than displaying everything at once after the response ends. Streaming also enables finer-grained timeout judgment: you can treat “no new content for a while” as an anomaly, instead of blindly waiting for the whole request to finish.
6.2 Resume from Checkpoint: No Need to Start Over After an Interruption
Generating a long answer can last tens of seconds or more, and during that time network jitter or a server-side timeout can cause an interruption. You can save the received content to a file together with an offset, then request the remaining part from the checkpoint after a disconnect:
import jsonimport timefrom pathlib import Pathdef generate_with_resume(client, prompt, save_path, max_retries=5):"""Stream generation with resume support: continue from the saved position after an interruption"""save_path = Path(save_path)done_text = ""if save_path.exists():done_text = json.loads(save_path.read_text(encoding="utf-8"))["content"]for attempt in range(1, max_retries + 1):request = prompt if not done_text else f"{prompt}\n\nHere is the content you had not finished last time. Continue from where it stopped without repeating:\n{done_text[-2000:]}"buffer = []try:for chunk in client.stream_chat(request):buffer.append(chunk)done_text += chunksave_path.write_text(json.dumps({"content": done_text}, ensure_ascii=False), encoding="utf-8")return done_textexcept (TimeoutError, ConnectionError) as exc:# if the connection fails right after being established, received content is still empty; a full retry is fineprint(f"Generation interrupted on attempt {attempt}: {exc}; continuing from the saved position")time.sleep(min(2 ** attempt, 30)) # exponential backoff with a cap to avoid a retry stormraise RuntimeError("Generation did not complete after multiple retries")
6.3 General Request-Level Guidelines
- Exponential backoff: on timeouts or rate limits, gradually lengthen the retry interval to avoid dense retries in a short window.
- Distinguish error types: non-retryable problems like invalid parameters or permission errors should be raised immediately; retryable problems like network timeouts or a temporarily unavailable service enter the retry flow.
- Context length control: pass only the code snippets and error messages relevant to the task, reducing transfer volume and model processing time.
- Unified gateway governance: manage timeouts, retries, and concurrency limits centrally in a unified integration layer, so business code does not have to reimplement them.
6.4 Observability: Making Speed Problems Measurable and Locatable
When performance is “sometimes fast, sometimes slow,” subjective impression alone makes it hard to tell whether the problem lies in the network, the server, or the client. Record the following metrics for each request and connect them to logging and monitoring/alerting:
- request start and end times;
- time to first token and total duration;
- output rate;
- number of retries and error types;
- input and output lengths.
With this data, you can answer “which stage is slow, by how much, and is it concentrated on a certain type of request,” and you get a basis for later model selection and capacity planning.
7. Deployment Options: Weighing Cloud API Against On-Premises Deployment
“If I could deploy it locally, it would be unbeatable” captures what many developers feel. On-premises deployment does have clear advantages:
- Data stays inside the network: context such as code snippets, log paths, and hostnames does not need to be sent to an external service, better meeting the requirements of sensitive projects.
- Predictable performance: since you control the inference resources, output speed is not affected by load fluctuations on a shared service.
- Controllable long-term cost: with high-frequency usage, the marginal cost of a self-hosted inference cluster is lower than pay-as-you-go billing.
But on-premises deployment also has real barriers:
- Hardware cost: large models have high demands on GPU memory and compute, requiring matching accelerator cards.
- Operational complexity: deploying, upgrading, scaling, and monitoring inference services all require dedicated engineering capability.
- Model quality: the performance of open-source models on coding tasks iterates quickly across versions, requiring ongoing evaluation and updates.
A more pragmatic idea is therefore to layer by task:
| Task tier | Typical scenarios | Deployment recommendation |
|---|---|---|
| High-frequency, lightweight tasks | Code completion, simple script generation, log categorization | Prefer local or small-model inference |
| Complex, deep tasks | Architecture design, tricky defect localization, cross-module refactoring | Use larger models; call cloud services as needed |
| Sensitive-data tasks | Code processing that involves core business logic or internal data | Must be handled locally |
After layering, you can route different tasks to the appropriate inference backend through a unified model gateway. This gives you a stable, low-latency experience in common scenarios while still leveraging stronger model capabilities for complex tasks.
Whichever approach you choose, pay attention to three things:
- A unified calling interface: decouple business code from specific models so you can switch and compare them at any time.
- Canary releases and evaluation: before a new model or version goes live, run a fixed set of tasks for a quality comparison, then ramp up gradually.
- Cost visibility: track call volume and cost by project and by user to avoid runaway spending.
8. Security and Compliance: A Review Checklist for AI-Generated Code
Model-generated scripts often “look right” but are not necessarily “safe to use.” Add the following checks to your code review process:
- Deletion scope: are the deletion conditions strict enough? Could non-target files be deleted along with them?
- Path handling: are paths from user input or external sources normalized and validated?
- Least privilege: is the script granted only the minimum permissions needed to complete the task?
- Reversibility: are files backed up, archived, or moved to a quarantine directory before deletion?
- Concurrency control: if a scheduled task and a manual run happen at the same time, could they conflict?
- Exception handling: if a single file errors out, does it abort the whole process?
For enterprise users, you can also bring model inputs and outputs into a unified security policy: sensitive code and data stay inside the network, and generated results go through static checks before entering the code repository. Cloud service providers continue to improve these capabilities as well, and you should choose based on your own security requirements.
9. Summary: Turning the AI’s “Speed” into Engineering “Stability”
Generating a disk cleanup program from a single sentence shows the practical value of large models in coding scenarios:
- For the developer, clear, structured prompts plus review and hardening of the generated code turn AI into a genuine productivity tool.
- For the team, streaming output, resume-on-disconnect, observability, and layered deployment turn a service’s “sometimes fast, sometimes slow” fluctuations into a stable, controllable engineering workflow.
- For the individual, mastering the basics of disk space analysis and a security mindset matters more than simply relying on a tool.
The “speed” AI brings is the starting point; the “stability” of engineering is the destination. Only by combining the two can you truly benefit in day-to-day development.
