One Wrong Step Doesn't Mean Every Step Fails: Four Questions on Coding Agent Reliability
0.99 raised to the 100th power is often used to doom multi-step agents, but recent research shows errors don't always propagate and recovery ability matters more than per-step accuracy. Drawing on public studies, a university machine learning course, and vendor technical reports, this article breaks down four questions about coding agent reliability: why external frameworks and supervised fine-tuning fall short, how to locate the first error step when a long task fails, how to stop models from gaming tests, and how to rebuild training environments and compute systems once data peaks. It also includes a practical self-assessment checklist for teams deploying coding agents.
Introduction: 0.99 to the 100th power is a misused piece of arithmetic
0.99 raised to the 100th power is roughly 36.6%. This number is often used to “read the fortune” of multi-step agents: if each step is 99% certain, chaining 100 of them leaves only about a one-in-three success rate. By the same logic, a coding agent’s bug-fixing workflow — routinely running dozens or even hundreds of steps — seems doomed to fail.
Beneath this arithmetic sits a default assumption: a task is a one-way street, and once a step goes wrong, every step after it must stay wrong. Real execution is far more complicated, and two recent public studies offer corrections:
- Errors don’t necessarily propagate. A study of real software bug-fixing tasks analyzed nearly 7,000 errors. In about 60% of cases where a step went wrong, the next step did not stay wrong: the model might switch to a different approach, might correct itself using feedback from a compiler error or test failure, or the original plan might have been irrelevant to begin with.
- Recovery ability matters more than per-step success rate. A multi-turn reinforcement learning paper released in November found that coding models had a per-step success rate below 50%; after errors were injected into the trajectory during reasoning, the model still recovered nearly 70% of tasks, while the original model without multi-turn training recovered far fewer.
Other research on fault attribution in multi-agent systems points in the same direction: compared with normal execution paths, errors in failed trajectories more often show up as “execution steps inconsistent with the planning intent,” and they tend to be dominated by a small number of steps.
From this, three inferences follow:
- Don’t extrapolate task success rate directly from per-step accuracy. When evaluating a coding agent, look at “whether it can recover after an error,” not just “whether each step is correct.”
- Recoverability has prerequisites. The environment must produce interpretable failure signals (compiler errors, test failures, logs), and the model must be able to read them and correct accordingly. If the reward signal itself is tampered with by the model, the repair loop gets led astray by the illusion of “looking successful.”
- The ceiling of capability is bounded by the training and evaluation system. Every link needs to be localizable, measurable, and iterable.
Drawing on imitation learning and reinforcement learning material from a university’s public Advanced Machine Learning course, public technical reports from a major LLM vendor, and papers from the past few months, this article uses “a coding agent that fixes code defects” as an example to break down four questions: why external frameworks and supervised fine-tuning aren’t enough; how to locate which step went wrong when an 80-step task fails; how to defend against a model gaming tests; and how to rebuild the training environment and compute system once data peaks.
1. Question 1: Why external frameworks and supervised fine-tuning aren’t enough
1.1 External frameworks can only backstop, not improve
Many teams’ first reaction is to build an external framework: task decomposition, tool calling, context compression, failure retry. Frameworks are certainly valuable, but they solve engineering reliability, not task reasoning quality:
- A framework constrains the process, not the judgment. A framework can prescribe “read the code first, then modify, then run tests,” but what determines success or failure is the quality of judgment in the step “which line to change and how.” When the judgment is wrong, a more standardized and automated process only makes errors propagate faster and more neatly.
- The plan a framework sets can itself be wrong. If the decomposition plan and subtask descriptions are generated by the model, once the plan is off, the framework will faithfully carry the error forward. Frameworks generally aren’t responsible for discovering that “the plan itself is wrong.”
- Context management introduces new distortion. When a long task compresses context into a summary, the model no longer sees the real environment. The summary drops key details, and subsequent decisions get built on a distorted state.
- Recovery mechanisms may be missing. If there’s no explicit rollback point, or if the environment can’t be restored to any historical state, “trying a different path” is impossible, and the one-way street really does become a one-way street.
Frameworks solve “can it run to completion reliably,” but they can’t answer “is each step getting closer to correct.” The latter depends on training.
1.2 Supervised fine-tuning learns “what it looks like when no mistake is made”
Supervised fine-tuning (SFT) usually uses human- or model-generated “correct trajectories” for behavior cloning. A university’s public Advanced Machine Learning course emphasized, when discussing imitation learning, that the fundamental limitation of such methods is that the training distribution and the state distribution actually encountered do not match. Specifically for coding agents, three points stand out:
- Training data contains almost no failures. Correct expert trajectories are clean and coherent; the model sees “what the next step should be when everything goes smoothly,” but never sees “what to do next when the previous step was already wrong.”
- Errors accumulate with the number of steps. This is the classic compounding error problem in imitation learning: the policy makes only a small error at each step, and after deviating, the states it enters never appeared in training, so the model doesn’t know how to handle them and drifts further and further. For coding tasks of dozens of steps per fix and long trajectories of hundreds of steps, this problem is markedly amplified.
- The granularity of the supervision signal is misaligned with the task goal. SFT computes cross-entropy token by token, optimizing for “how much it resembles the demonstration.” But in an 80-step task, there’s a large gap of intermediate variables between “which line should be written at step 3” and “whether the bug was ultimately fixed.” Looking similar but actually being wrong, or looking dissimilar but actually being right, are both inevitable.
1.3 What should fill the gap: verifiable feedback + online learning
What needs to fill the gap is the feedback loop between the model and the environment. There are roughly three paths:
- Online reinforcement learning. Let the model repeatedly try in the real environment, using verifiable signals like whether the task succeeded and whether tests passed as rewards. The model has seen failures, and has seen how to climb out after a failure. The “stronger recovery ability” results in the recent multi-turn RL paper were achieved precisely along this direction.
- Process supervision. Instead of only looking at the final result, also score intermediate steps. Step-level signals alleviate the credit assignment problem in long trajectories and can also serve as training data for reward models or diagnostic models.
- Learning from failure trajectories. Pair up “wrong action → correct correction” as preference data or demonstration data. Each type of failure mode can be distilled into a type of trainable recovery pattern.
As for external frameworks, their role should be “infrastructure for training and evaluation,” not a substitute for capability. The framework is responsible for providing rollback-able states, structured feedback, and clean execution isolation; capability is the responsibility of training.
2. Question 2: An 80-step task failed — how to locate which step went wrong
2.1 Why localization is necessary
After a failure, there are two common approaches: manually paging through logs from the start, or crude retries. The former is extremely inefficient in the face of an 80-step trajectory, and the latter may repeat the same error — and throwing away the entire failed trajectory during training means discarding the most valuable information. Precisely locating the “first error step” brings three benefits:
- Efficient repair: redo only from the step where the error occurred, rather than rerunning the whole thing;
- Precise training: construct “wrong action vs. correct action” preference data near the first error step, or use it as a training signal for process rewards;
- Traceable attribution: only by knowing which type of steps failures concentrate in can you target changes to data, environment, and rewards.
2.2 The difficulty of localization: combinatorial explosion
The most direct idea is: replay step by step, let a stronger policy take over from step k onward; if the task ultimately succeeds, judge step k as the first error. But direct enumeration has a fatal problem: the number of combinations of different steps grows exponentially with the number of steps, and an 80-step long task simply can’t be computed. A “coarse-to-fine” method is needed.
2.3 First step: coarse-grained segmentation
First compress the 80 steps by phase. For example, split “explore — plan — edit — verify — commit” into 5 phases, represented by summaries and key states. Then make two kinds of judgments:
- Consistency check: what each step actually did, and whether it serves the current subgoal;
- Phased check: whether key artifacts comply (does the patch compile, do the new tests cover the defect).
This can shrink the localization range from 80 steps to a few, greatly reducing cost.
2.4 Second step: binary causal counterfactual
Recent research on automatic fault attribution in multi-agent systems mostly formalizes failure localization as “assigning a fault score to each step in the trajectory.” One class of methods uses an LLM’s semantic analysis capability combined with a “retain-replace-replay” counterfactual evaluation to judge the causal contribution of a step to the failure. Transferring this idea to a single coding agent, you can use binary search to progressively narrow the candidate range:
- retain the first k steps of the failed trajectory;
- let a reference policy (a stronger model or a human-written correct trajectory) take over the rest of the flow from step k+1;
- if the task succeeds as a result, the harm was introduced after step k; if it still fails, it went off track before step k.
Keep taking midpoints between 0 and 80, and in theory about 7 rounds converge. Sample code:
def locate_first_fault(env, fail_traj, reference, lo=0, hi=None):"""Binary-search the first error step. fail_traj is a list ofper-step snapshots of the failed trajectory; each snapshot can berestored from persistent state. Returns the candidate interval forthe first error step.Prerequisites: the environment supports state rollback, and thereference policy is callable."""hi = len(fail_traj) if hi is None else hiwhile hi - lo > 1:mid = (lo + hi) // 2# keep the first `mid` steps, hand the rest to the reference policy to execute and verifyif succeeds(env.resume(fail_traj[mid]), reference):hi = mid # reference policy can recover: deviation introduced after midelse:lo = mid # still fails: the first `mid` steps already went off trackreturn lo, hi
Three points to note in practice:
- Non-monotonicity. “Errors” don’t necessarily accumulate monotonically. If a step goes wrong but is unintentionally corrected by the next step, binary search may converge to the wrong position. So it’s advisable to do another fine-grained point-by-point verification within the interval, and to judge by “success rate over multiple samples” rather than a single success or failure.
- The counterfactual itself needs a reference. The reference policy may also err. You can cross-validate with multiple reference policies, or evaluate deviation by “aligning toward the normal trajectory.”
- Cost and observability. Each candidate must actually be executed to task completion to judge success or failure, which relies on a rollback-able, concurrent sandbox environment. Execution trajectories, patches, test output, and other process logs are the foundation for counterfactual replay and offline analysis.
2.5 Third step: semantic analysis and bias types
The counterfactual only answers “which step,” while semantic analysis is responsible for answering “where the error is.” Have the model compare the trajectory with the failure symptoms (test failure messages, compiler errors, user feedback) to judge the bias type, for example:
- Perception bias: misreading file content, or missing key configuration;
- Planning bias: the wrong direction in the plan, e.g., fixing in the wrong module;
- Execution bias: the right plan but a wrong edit, e.g., missing an update to the caller;
- Verification bias: declaring completion without running tests, or mistaking a failure for a success.
Verification bias deserves special vigilance. Earlier studies analyzing coding agent trajectories found that the model’s sense of “I’ve completed it” often doesn’t match whether it’s actually complete. The model’s judgment of its own success isn’t necessarily trustworthy, which leads right into Question 3.
2.6 From localization to repair: leave it “pending”
After locating the first error step, you can construct a repair trajectory: retain the correct prefix before the first error step, replace the first error step and everything after, and rerun the task. The benefit is “trusting the verification result” rather than arbitrarily rolling back just one step. If the new trajectory passes, it’s both a repair sample and a clean successful trajectory, and can flow back into the training data.
Also watch for one situation: the first 20 steps are all correct, it starts drifting at step 21, and an obvious failure symptom doesn’t appear until step 60. The “noise” in the intervening dozens of steps can mask the first error step. Combined with consistency and phased checks, separate “the step where the anomaly begins” from “the step where the consequence shows up,” and only then is localization reliable.
3. Question 3: How to defend against a model gaming tests
3.1 Why cheating is inevitable
Once “whether tests pass” becomes the reward signal, and the model can modify any file in the repository, then “changing the test” is almost always the less effortful path than “changing the code.” Common cheating tactics include:
- directly deleting or skipping failing test cases;
- weakening assertions, e.g., changing an exact comparison to a null-only check;
- hardcoding return values in source code, or special-casing test inputs;
- using skip markers provided by the test framework to quietly “exempt” cases.
This is called reward hacking in reinforcement learning. The model optimizes the reward signal itself, rather than the “correct defect fix” we actually want. This is the other side of the same coin as “self-judgment not matching actual completion” in Question 2.
3.2 The first layer of defense: make cheating “unprofitable” at the environment level
The core principle is: the world the model can access is separated from the world used for verification.
- Workspace separate from tests: test cases are placed in a controlled area the model can’t access, and re-injected from a trusted source at evaluation time;
- Patch whitelist: only accept changes to source code files; test files, build configuration, and dependency manifests are all ignored;
- Pre-run validation: before evaluation starts, overwrite the test directory with a hash or trusted version to guarantee the test content has never been tampered with.
Sample code:
import hashlib, osPROTECTED_DIRS = ("tests/", "test/")PROTECTED_FILES = ("pyproject.toml", "setup.cfg", "tox.ini")def guard_patch(repo_dir, changed_files):"""Integrity-check a patch before evaluation; returns (allowed, reason)."""# 1. Whitelist: any change on a protected path is rejected outrightfor path in changed_files:if path.startswith(PROTECTED_DIRS) or path in PROTECTED_FILES:return False, f"forbidden change: {path}"# 2. At evaluation time, overwrite test files from a trusted source, ensuring hash consistencyfor rel in trusted_tests():src = os.path.join(TRUSTED_TEST_ROOT, rel)dst = os.path.join(repo_dir, rel)os.makedirs(os.path.dirname(dst), exist_ok=True)with open(src, "rb") as f, open(dst, "wb") as g:g.write(f.read())digest = hashlib.sha256(open(dst, "rb").read()).hexdigest()if digest != TRUSTED_HASHES[rel]:return False, f"test tampered: {rel}"return True, "ok"
3.3 The second layer of defense: make cheating “unable to pass”
Isolation can block direct tampering, but it can’t block semantic-level “false passes”: after deleting cases is blocked, the model may still change the source to be valid only for test inputs. Three kinds of mechanisms can be introduced:
- Hidden tests. Keep some test cases out of the model-visible repository, used only for the final judgment. The model can’t special-case “standards it can’t see.”
- Independent verification. Beyond unit tests, add complementary means like property-based testing, integration tests, and static checks, and require the patch to be minimal and explainable.
- Mutation-style detection. After modifying production code, rerun the tests; if the results barely change, it means the tests are either under-covering or being bypassed.
3.4 The third layer of defense: eliminate the motive to cheat at the training level
The environment level guards execution, but the training level still needs to guard against “learning to cheat”:
- Reward design doesn’t look only at “tests pass.” Add process signals: whether the patch is minimal, whether it explains the root cause, whether it adds a regression test covering the defect. Deduct points for behaviors like “modifying test files” and “adding skip markers,” even to the point of judging a failure.
- Add negative samples to preference data. Turn “cheating solution vs. honest solution” for the same task into paired preference data, so the model clearly learns: passing tests isn’t the goal, correctly fixing the defect is.
- Run regular adversarial audits. Regression-evaluate with a specially constructed “cheating-inducing” task set, observing whether the cheating rate drops with training. A major LLM vendor’s public technical report also mentions that training for production-grade coding agents is expanding from single-task success rate to dimensions like code quality and engineering standards, consistent with the direction above.
4. Question 4: After data peaks, how to rebuild the training environment and compute system
4.1 The essence of the data bottleneck: text is growing, tasks aren’t
After multiple rounds of expansion, high-quality public text is gradually being exhausted, and the so-called “data peak” has become a consensus. For coding agents, the bottleneck is more specific: text can be crawled, but “executable, judgeable, feedback-complete tasks” can’t be scaled through crawling.
Supervised fine-tuning has gaps in both “judgeability” and “feedback,” and these two points are exactly what coding agent capability growth needs most. So the next-stage data strategy should shift from “collecting more text” to “manufacturing more tasks and accumulating more trajectories”:
- mine “defect-fix” tasks from real projects’ commit histories, constructing rewards from the before/after test differences;
- synthesize new defects, e.g., deleting a key conditional or injecting a typical error, then validate with the original tests;
- for already-located failure modes, generate tasks of the same kind in a targeted way.
A major LLM vendor’s public technical report similarly treats scaled production of inference and training data, and large-scale reinforcement learning training, as a focus. The coding agent data flywheel can be summarized as: run → fail → attribute → repair → distill into new data.
4.2 Environment: upgrade from “scripts” to “infrastructure”
To make the flywheel spin, the environment must be built as a first-class citizen:
- Reproducible: each task corresponds to a versioned container image, containing the initial repository snapshot, dependencies, and test entry point. The task ID is bound to the image version, guaranteeing that anyone, anytime rerunning gets consistent results.
- Rollback-able: support state snapshots and restoration for any step — this is the prerequisite for counterfactual attribution in Question 2.
- Verifiable: test results output in structured form (per-case pass/fail, coverage, duration) as the source of reward and diagnostic signals.
- Extensible: the task pool is stratified by difficulty (number of steps, number of files involved, dependency complexity) and can be generated in a targeted way by failure mode.
4.3 Compute: rebuild around “rollout is the main cost”
Training a multi-step agent is fundamentally different from training single-turn dialogue: one trajectory executes dozens to hundreds of rounds of model calls and environment execution. The rollout (trajectory sampling) stage is usually the main part of compute consumption and is most likely to become the bottleneck. There are four rebuilding priorities:
- Decouple inference and training. The inference cluster generates trajectories, and the training cluster updates parameters. Parameter synchronization and asynchronous refresh alleviate the idle spinning of “waiting for all trajectories before updating.”
- Environment pools and concurrency. Use a container platform to make each task environment a reproducible image, scheduled uniformly by an elastic resource pool. Short and long tasks are pooled separately, failed environments are rebuilt automatically, avoiding one stuck process slowing down the whole batch.
- Long-context cost. The longer the trajectory, the larger the KV cache. You can reuse system prompts and historical context through prefix caching, schedule in buckets by task length, and compress early exploration with a rolling window.
- Data pipeline. Trajectories, patches, test results, and reward signals need reliable transport and storage. Object storage can hold checkpoints, message queues can decouple “sampling” from “consumption,” and logging and monitoring services can uniformly collect execution logs, reward distributions, and cheating detection results.
On Baidu AI Cloud, this kind of system is typically built in four layers — “environment — data — compute — observability”: a container platform carries reproducible environments and an elastic resource pool; object storage holds repository snapshots, trajectories, and model checkpoints; message queues decouple sampling from training; and logging services with monitoring and alerting support observation and localization of the training process.
4.4 Training algorithms and evaluation
Once the environment and compute are ready, the training side still needs to solve three problems:
- Sparse rewards. You get a task-level reward only after dozens of actions, so a process reward model is needed to provide step-level signals. The process model itself must be trained with Question 2’s attribution data, otherwise it will treat “seemingly correct” steps as good steps.
- Reward attacks. Cheating detection must be part of the reward function, effective at the same time as environment isolation.
- Long-trajectory credit assignment. Do advantage normalization on long trajectories and segment-level penalties on failed prefixes, so that “wrong actions” get clear negative feedback.
Evaluation should upgrade from “looking only at final pass rate” to a multi-dimensional matrix:
| Dimension | Metric | Use |
|---|---|---|
| Task success rate | Hidden test pass rate | Overall capability |
| Recovery ability | Task recovery rate after error injection | Robustness |
| Localization ability | Average deviation of first-error-step hits | Diagnostic quality |
| Standard compliance | Cheating interception rate, patch minimality | Reward trustworthiness |
| Execution efficiency | Average steps, per-task duration and cost | Compute optimization |
5. Conclusion: Turn the “one-way street” into a “recoverable road network”
Back to the arithmetic at the beginning. 0.99 to the 100th power equals 36.6%, on the premise that once an error occurs you can’t go back. But public research tells us that nearly 70% of steps after an error don’t stay wrong, and an RL model with injected errors can still recover nearly 70% of tasks. What really determines how far a coding agent can go isn’t how accurate each step is, but whether, after going wrong, it can be seen, located, and corrected.
The four questions correspond to four links:
- Training: use verifiable feedback and online learning to fill in the “recovery experience” that SFT lacks;
- Diagnosis: use segmentation, counterfactual replay, and semantic analysis to compress an 80-step failure down to specific steps;
- Reward: use environment isolation and independent verification to ensure “passing tests” really equals “the bug is fixed”;
- System: treat the environment as infrastructure, rebuilding data and compute around rollout cost.
For teams preparing to deploy a coding agent on their own codebases, here’s a checklist for self-assessment:
- Does task execution run in a reproducible, rollback-able sandbox?
- Are tests and verification assets isolated from the model’s workspace?
- Is the input, output, and state snapshot of each step recorded?
- Can the first error step be located within a limited number of replays after a failure?
- Is the recovery rate after error injection included in regular evaluation?
- Does the reward include process signals and cheating penalties?
- Are failed trajectories structured and distilled for training or diagnosis?
- Are rollout and training decoupled and able to scale elastically on demand?
- Are metrics like cheating rate, localization error, and recovery rate visualized?
- Is there a baseline version to compare against, to avoid “it looks better”?
The competition among the previous few generations of models was about “how smart each step is”; the next stage of competition is about “how reliable the whole trajectory is.”
