LLM Programming in Practice: From Fixing Stubborn Bugs to Cost Control

I. When Legacy Bugs Meet Long-Context Models

Almost every long-term, self-developed project harbors a few “legacy bugs”: vague trigger conditions, lengthy reproduction paths, and scattered log information. In a content distribution tool I developed, there was exactly this kind of issue—during concurrent writes of specific data streams, the downstream consumer occasionally experienced data misalignment. The system would self-recover after a restart, but a stable reproduction path remained elusive. Previous attempts with general programming assistants either applied patches to the wrong locations or lost context when reading multiple associated files, ultimately leading nowhere.

The turning point came after an iteration in model capabilities. I fed the complete task directly to a model supporting a million-token context window: containing the project directory tree, relevant module source code, exception logs from the past two weeks, and diff records of previously failed patches. From hitting Enter to the model providing a runnable patch that passed regression testing, the entire process took about 9 minutes. Human intervention was limited to two confirmations—one for the reproduction steps and one for the patch merge.

The true value here wasn’t the “speed,” but the fact that the model maintained a complete reasoning chain across files and modules within a single session. Let’s break down this engineering workflow.

II. Capability Matrix: Which Benchmarks to Watch

Before diving into specific practices, it’s necessary to establish an evaluation framework. The industry’s measure of programming capability has shifted from simple code completion to multi-dimensional benchmarks:

• Long-horizon continuous programming: Evaluates whether the model can maintain goal consistency over tasks spanning hours without introducing regression bugs. Common benchmarks simulate “commit-test-fix” loops.
• Constrained terminal tasks: Evaluates the model’s operational ability in real command-line environments, facing constraints like file systems, permissions, and process states. Score improvements here often indicate the model evolving from “writing code” to “getting work done.”
• Autonomous task execution: Evaluates the model’s ability to break down steps, call tools, and verify results autonomously after being given a high-level goal.

The commonality among these dimensions is: they measure engineering loop capability, not isolated code quality. A model performing well on long-horizon benchmarks usually means it can remember original constraints after multiple rounds of interaction without drifting off-topic as the context grows.

III. The Complete Workflow for Fixing Stubborn Bugs

3.1 Task Packaging: Turning “Vague Problems” into “Structured Inputs”

No matter how powerful the model is, it relies on input quality. My approach is to package the task into a four-part structure:

[Background] Project type, tech stack, runtime environment
[Symptoms] Exceptional behavior, trigger frequency, impact scope
[Evidence] Log snippets, stack traces, relevant source code paths
[Constraints] Interfaces that must not be broken, test cases that must pass

The key is that the evidence section should provide raw materials, not conclusions. For example, instead of writing “suspected concurrency issue,” directly paste the thread states and shared variable snapshots at the time of the exception. The model will formulate and verify hypotheses on its own.

3.2 Root Cause Localization: Letting the Model “Read” the Code

In this practice, the model first pulled the source code of three associated modules and checked them layer by layer along the data flow. It output a reasoning summary, pointing out that the issue was caused by a non-atomic update of a state flag during buffer switching, causing the consumer to read an old offset within a very short time window.

Notably, the model did not rush to provide a patch. Instead, it first restated the reproduction path and requested confirmation. This step is crucial: it separates “understanding” and “modification” into two phases, avoiding direct rewrites based on incorrect assumptions.

3.3 Patch Generation and Verification

The patch itself was only a dozen lines long, with the core change replacing the non-atomic state update with a versioned compare-and-swap operation. The model simultaneously provided a minimal reproduction script and three regression test cases. After running locally, the reproduction script went from “occasionally failing” to “stably passing,” with no regressions in the original test suite.

Throughout this process, the human only did two things: confirm the reproduction steps and merge the patch. This validates a trend: with sufficiently long context, the model can independently complete the closed loop of “read-think-modify-verify”.

IV. Frontend Landing Page: From Description to Runnable Code

After fixing the bug, I casually tested the model’s frontend generation capability. The task description was: “Generate a single-page landing page, including a product intro, three feature cards, an API example code block, and footer copyright info. Use a clean light theme, responsive layout, and do not rely on any external UI frameworks.”

The model output a complete HTML file, with inline styles and a small amount of interaction script. The structure was clear, class names were semantic, and mobile breakpoints were set reasonably. Although visual details required manual fine-tuning, the usability of the skeleton code was already very high, saving the time of building from scratch.

There is a lesson here: frontend generation tasks should constrain the tech stack as much as possible. If you just say “make a landing page,” the model might introduce unnecessary dependencies; specifying “no external frameworks” and “inline styles” results in cleaner, more maintainable output.

V. The Cost Ledger: Managing Annual Budgets

No matter how capable the tool, you have to do the math. Current mainstream API billing is typically divided into three tiers: input, cached input, and output. The price for cached input is often about one-tenth of the standard input price.

Taking an independent developer with an annual budget of roughly $2,500 as an example: assuming a daily consumption of 500,000 input tokens and 50,000 output tokens, with a cache hit rate of 60%, the annual cost will fall well within the budget. There are three core strategies for cost control:

1. Improve cache hit rate: Place stable system prompts, project background, and coding standards at the beginning of the session. This content can be cached across multiple requests.
2. Context pruning: Only feed the model file snippets relevant to the current task, not the entire repository. A million-token window is a ceiling, not a goal.
3. Task consolidation: Complete “localization-modification-verification” within the same session to avoid repeatedly transmitting the same context.

It must be emphasized that the prerequisite for cost control is task packaging quality. The more precise the input, the fewer detours the model takes, which paradoxically lowers Token consumption.

VI. Practical Advice for Developers

Synthesizing the practices above, here are a few reusable principles:

• Treat the model as a collaborator, not a code completion tool: Give it the full background and raw evidence, let it participate in reasoning, rather than just asking it to fill in the blanks.
• Phase-by-phase confirmation: Set confirmation points between “understanding” and “modification” to prevent large-scale rewrites based on wrong assumptions.
• Focus benchmarks on long-horizon capabilities: Isolated code quality has largely become commoditized; what truly sets models apart is their goal-retention capability after multi-turn interactions.
• Cost is designed: Cache strategy, context pruning, and task consolidation—do these three things well, and the budget will naturally be controllable.
• Constrain tech stacks for frontend generation: Clearly define dependency boundaries so the output is clean and usable.

LLM-assisted programming has moved past the stage of “can it write code” and entered the stage of “can it independently complete an engineering loop.” For independent developers and small teams, this means completing tasks that previously required multi-person collaboration at extremely low personnel costs. Whether you can use it well depends on how you package problems, design context, and control costs—these remain the core competencies of an engineer.