Hands-On LLM Coding: Fixing a Stubborn Bug in 9 Minutes on a Limited Budget

From Benchmarks to Real Projects: Why a Real-World Test Matters

High leaderboard scores can’t answer one specific question: can this model fix the bug sitting in front of me? Standardized benchmarks reflect a model’s statistical performance on typical tasks, while real-world engineering is full of “dirty work”: years-old code, unmaintained dependencies, and reproduction conditions nobody can quite pin down. Rather than chasing leaderboards, it makes more sense to run a real test on a genuine legacy issue and record time spent, token consumption, and change quality throughout — replacing emotional takes with engineering metrics.

For this test I picked a bug that had been sitting in my self-built content distribution tool for three months. Multiple models and prompt strategies had failed to fix it — a classic stubborn case. This time I organized the test around a complete engineering pipeline: from pressing Enter to reproduction, modification, and verification, the whole process took about nine minutes, and I didn’t hand-write a single line of code. The rest of this article breaks down the benchmark interpretation, the fix workflow, the frontend practice, and the cost bill.

Understanding LLM Coding Ability: Architecture, Context, and Three Key Benchmarks

Mixture-of-Experts: Decoupling Capacity from Cost

Today’s flagship models generally use a Mixture-of-Experts (MoE) architecture: total parameters can reach the trillion scale, but each token activates only a small slice — typically a few percent of the total. This brings two direct benefits. First, the model’s knowledge breadth and capacity are determined by total parameters, so it “has room” for more programming languages, frameworks, and engineering conventions. Second, the cost of a single inference is driven mainly by activated parameters, so you don’t pay for everything the model knows. The practical takeaway: when choosing a model, don’t obsess over the total parameter count. Focus on the activation ratio and the actual inference price, and validate with samples of your own real tasks.

The Engineering Meaning of Million-Token Context Windows

The industry has entered the era of million-token context windows. For coding, this means the key modules of a small-to-medium repository, long logs, and multiple configuration files can all fit into a single session, letting the model reason globally across files instead of relying on manual chunking and repeated feeding. Two caveats: the longer the context, the higher the risk of diluted attention, so key information should be placed at the beginning or end and clearly labeled; and very long inputs cost real money, which makes caching mechanisms a critical lever for cost control.

Three Benchmark Dimensions That Map to Real Work

Aggregate leaderboards are good for the big picture, but independent developers should pay closer attention to three dimensions:

• Long-horizon continuous coding benchmarks: these simulate multi-step bug fixes and feature development in a real repository, testing cross-file understanding and long-task planning. The latest generation of models has reached the first tier here, with the gap between the top few narrowed to single-digit percentage points.
• Agentic autonomous task benchmarks: the model gets tools to read and write files, execute commands, and search code, and is measured on its success rate with minimal human intervention. This directly determines whether “fix a bug with one sentence” is actually feasible.
• Constrained tasks in a real terminal: the model must set up environments, troubleshoot builds, and debug processes in a real shell, testing hands-on command-line skill. This is the foundation of agentic workflows — when this ability is weak, the model keeps getting “stuck on the environment.”

Rule of thumb for reading leaderboards: absolute scores matter, but trends matter more, and you should weight the dimensions that match how you actually work. If your daily work revolves around terminals and scripts, terminal-focused benchmarks deserve more weight.

What Open Weights Mean for Individual Developers

If a vendor commits to releasing full open weights at a clearly stated point in time, it means the model can eventually be deployed privately, fine-tuned, and kept on-premises so data never leaves your domain. For individual developers, calling the API remains the best value in the short term, but “having weights as an option” is itself bargaining power and an escape hatch — worth factoring into model selection.

Test 1: The Complete 9-Minute Workflow for Fixing a Legacy Bug

Before the Fix: Turning a Vague Problem into an Executable Task

Of the nine minutes, seven came from preparation. Before handing the problem to the model, complete three things: pin down the reproduction path — settle on a single command that reliably triggers the issue, with the full error stack attached; narrow the suspect range — list the relevant module paths, recent change history, and the hypotheses already ruled out; and write clear acceptance criteria — specify which tests must pass and which existing behaviors must be preserved.

A Structured Prompt Template

Open the session with a structure like this (illustrative):

[Background] Tech stack and module responsibilities
[Symptom] Running <reproduction command> produces <key error>; full stack trace below
[Expected] The correct behavior is ...
[Already tried] Upgraded dependencies, rolled back commits, commented out code section by section — none of it worked
[Constraints] No new dependencies; changes kept minimal; API stays backward compatible
[Acceptance] Run the following commands in order; the task is done only when all pass

The “Already tried” section is especially important: it stops the model from wasting time on paths you’ve already eliminated. The “Constraints” section curbs the model’s common “drive-by refactoring” instinct, keeping a small fix from turning into a big rewrite.

Autonomous Agent Execution: Reproduce, Locate, Fix, Verify

In agent mode, the model completed the standard loop on its own: it first ran the command to confirm the problem and read the stack trace; then searched the relevant code based on the trace, formed hypotheses about the suspect functions, and verified them with logs or assertions; then produced a minimal change — typically delivered as a diff patch rather than a full-file rewrite; and finally ran the acceptance commands and regression tests, looping back to the locating step when something failed. The human’s role shrank to two things: approving each change and confirming the final acceptance result. That’s what “not writing a single line of code” really means — the code-writing step got compressed, but the responsibility for judgment and acceptance never disappeared.

Retrospective: Why the Model Fixed in 9 Minutes What Humans Sat On for 3 Months

This bug wasn’t unsolvable — its “psychological cost” was just too high. Every time I picked it up, I had to reload all the context, and after two failed attempts I would shelve it again. The model has no sunk costs and no fear of difficulty; it mechanically executes bisection and hypothesis testing, which is exactly the weakest link in human workflows. The reusable lesson: describing the problem clearly is worth more than any prompt-engineering trick.

Test 2: Efficient Generation and Validation of a Frontend Landing Page

Generation tasks call for a slightly different workflow. I recommend three steps: structure first, single-file prototype, split after acceptance.

1. Structure first: write out the sections (hero, features, testimonials, FAQ, footer), the key copy points, and the visual style keywords as a list. This keeps the model from improvising and forcing rework.
2. Single-file prototype: ask for a single HTML file with inline styles and scripts, then open it directly in a browser to validate responsive breakpoints and interaction details.
3. Split after acceptance: once the visuals and interactions are confirmed, have the model split the code according to your team’s component conventions and wire it into the existing build pipeline.

Three common pitfalls: for design fidelity, agree on spacing and typography scales up front; for images, use placeholders to lock down dimensions and avoid layout drift; and review the generated code for semantic tags and accessibility — models tend to miss these details, but they’re quick to fix.

Counting the Tokens: A Cost Model and Savings Strategies for Solo Developers

The Three Components of Pricing

Whether you call a model API directly or use hosted model services from a cloud platform such as Baidu AI Cloud, pricing is generally built from three parts: the input token price, the output token price (usually several times the input price), and the cache-hit price (often as low as one-tenth of the input price). Coding sessions are “input-heavy, output-light”: reading code and logs consumes a large number of input tokens, while the code actually generated is relatively small.

A Cost Estimate for a Bug-Fix Session (Illustrative)

Assume illustrative prices of ¥10 per million input tokens, ¥50 per million output tokens, and ¥1 per million cache-hit tokens. A typical fix session accumulates about 600,000 input tokens (500,000 of them cache hits) and about 80,000 output tokens. Input cost: 100,000 × 10/1,000,000 + 500,000 × 1/1,000,000 = ¥1.5. Output cost: 80,000 × 50/1,000,000 = ¥4. Total: about ¥5.5. In other words, an efficient agent session typically costs on the order of a few yuan — far below the labor cost of doing the same task by hand.

Annual Budget Planning and Four Money-Saving Strategies

Based on my usage across self-built projects over the past three months — where I’ve generated and modified code on the order of hundreds of thousands of lines — a pace of 3–5 effective sessions per day at a few yuan each works out to roughly ¥10,000 a year, within reach for an independent developer. Four practical strategies:

1. Leverage caching: put stable system prompts and repository structure notes at the front of the session; cache hits significantly dilute input costs.
2. Ask for diffs, not full files: explicitly request minimal patches — this directly compresses output tokens.
3. Tier your models: route exploratory questions and code search to cheaper models, and reserve the flagship model for critical changes and complex debugging.
4. Batch your tasks: collect scattered small requests into batches to reduce the overhead of repeatedly feeding context.

Methodology: Folding an AI Coding Assistant into Your Engineering Process

A few reusable principles to close:

• Divide the roles: treat the model as a tireless locator and junior engineer; the human owns architectural judgment, security review, and final acceptance.
• Tests are the moat: the more complete your regression tests, the wider the trust boundary within which agents can work autonomously.
• Review is non-negotiable: manually check generated code for leaked secrets, injection risks, dependency licenses, and performance issues.
• Set clear boundaries: humans lead on core algorithms and strongly deterministic logic; boilerplate code, test completion, bug locating, and documentation generation are good candidates for the model.

Benchmarks decide which model you pick; the engineering pipeline decides whether you can use it well. Once the “reproduce–locate–fix–verify” loop is executed efficiently by the model, a developer’s real leverage comes down to two things: define the problem clearly, and set the acceptance criteria.