How Coding Agents Move from Single-Step Accuracy to Long-Horizon Reliability

    Long-horizon coding tasks are not simply a chain of independent steps whose success probabilities must be multiplied. Reliable agents need to detect errors, contain their impact, recover through feedback, and deliver verifiable fixes. This article examines task definitions, supervision, reinforcement learning, credit assignment, evaluation security, training environments, systems architecture, and metrics for measuring real-world reliability.

    1. Long-Horizon Tasks Are Not Irreversible Straight Lines

    Suppose a system must complete a task in 100 consecutive steps, with a 99% probability of being correct at each step:

    [
    0.99^{100} \approx 36.6\%
    ]

    The calculation is correct, but it describes one specific event: all 100 steps are correct under the stated probability assumptions. Treating this number directly as the success rate of a coding agent can be misleading.

    In real software repair, the agent can often roll back, verify its work, and replan. It may initially identify the wrong file and later correct course after inspecting call relationships. It may submit an incorrect patch, then use a failed test to guide it toward the right implementation. As long as the final delivery satisfies the requirements, intermediate mistakes do not necessarily mean task failure.

    Even without assuming that the steps are independent, the probability of being correct throughout the entire sequence should be expressed as a product of conditional probabilities:

    [
    P(\text{all steps correct}) =
    \prod_t P(\text{step } t \text{ is correct} \mid \text{all previous steps were correct})
    ]

    This is still not the same quantity as the probability of eventually completing the repair successfully. The latter also includes recovery after wrong turns, convergence through different paths, and the use of validation feedback.

    Some error statistics and recovery-improvement figures cited in the original material lack verifiable study names, sample definitions, and experimental conditions. They should therefore not be treated as universal conclusions. The more durable technical insight is this:

    Evaluating a long-horizon agent requires measuring not only whether it makes mistakes, but also whether it can detect mistakes, stop them from spreading, and recover the task.

    For a bug-fixing task, a more realistic process is:

    Understand the problem → form hypotheses → inspect the code → modify the implementation → validate the result → continue correcting based on feedback.

    This is a feedback loop, not a one-shot answer-generation pipeline.

    2. Define What “Fixed” Means Before Training

    Before discussing training, define the conditions for task success. Otherwise, a model may learn to pass a check without solving the underlying problem.

    A bug-fixing task suitable for training and evaluation should include at least the following elements:

    • Clear requirement boundaries: Describe the actual behavior, expected behavior, and the scope of permitted changes.
    • A fixed initial state: Include the code snapshot, dependencies, configuration, and any required data.
    • A reproducible failure trigger: Confirm that the defect exists and is not merely an intermittent environment failure.
    • Credible acceptance criteria: Cover the target defect while also checking important regression behavior.
    • Controlled permissions: Distinguish business code the agent may modify from evaluation assets it must not modify.
    • Reasonable budget constraints: Limit execution time, interaction count, and resource consumption.

    Two extremes should be avoided.

    The first is checking only whether one test passes. This encourages overfitting to a narrow check. The second is requiring the patch to match a reference implementation exactly. That can reject logically correct fixes that use a different implementation.

    Acceptance should therefore focus on behavior and constraints rather than the appearance of the patch. Passing an evaluation provides evidence within the scope of the tests and checks that were run; it does not prove that the software contains no other defects.

    3. Why External Frameworks and Supervised Fine-Tuning Are Still Not Enough

    3.1 External frameworks provide execution structure, not automatic judgment

    An external framework can manage tool calls, maintain task state, restrict file permissions, establish checkpoints, and trigger retries after failures. These capabilities are important, but they primarily answer how to organize execution, not what should be done next.

    For example, when the same test fails, the agent must distinguish among several possibilities:

    • The implementation logic is actually wrong.
    • The test environment is incomplete.
    • The requirements were misunderstood.
    • The latest patch introduced a regression.
    • The execution result was incomplete because of resource limits.

    A fixed workflow can require the agent to retry after failure, but it may not determine whether another retry is worthwhile. If the problem comes from a false assumption, repeating the same operation only wastes the budget.

    A framework is therefore best understood as safety infrastructure and execution scaffolding. It cannot replace the model’s internal abilities to diagnose problems, plan actions, and assess evidence.

    3.2 Supervised fine-tuning learns demonstrations but may lack post-failure experience

    Supervised fine-tuning teaches a model to generate suitable actions in a given context. If the training data consists mostly of cleaned-up successful trajectories, the model may learn an idealized workflow: locate the issue quickly, make the correct change, and pass on the first attempt.

    Real interactions contain many deviations:

    • Search returns a similarly named but unrelated module.
    • A change exposes another boundary condition.
    • A tool returns truncated logs.
    • The failure message is several call layers away from the root cause.

    After an incorrect action, the model may encounter a state that differs substantially from its training demonstrations. As deviations accumulate, strong performance on standard steps does not guarantee that the model can return to a productive path.

    Better supervised data should include failure diagnosis, rejected hypotheses, patch rollback, and additional checks when the evidence is insufficient—not merely more examples of one-shot success.

    3.3 Multi-turn reinforcement learning focuses on long-term consequences

    Multi-turn reinforcement learning can teach a model whether the current action improves the probability of completing the task later.

    For example, “run a targeted check first” may not change the code immediately, but it can significantly narrow the fault domain. “Modify the most suspicious function immediately” may appear faster, yet rest on a wrong interpretation of the requirements.

    An action that looks better in the short term does not necessarily produce a higher final success rate. Conversely, stronger recovery ability alone does not prove that a model is better overall. Recovery may require more execution cost, create additional risk, or reduce normal-task performance. These factors must be evaluated together.

    4. How to Identify the Real Problem When Failure Occurs After 80 Steps

    One of the central challenges in long-horizon reinforcement learning is credit assignment: the final outcome is known, but which earlier actions deserve credit or blame?

    4.1 Do not assign the final failure equally to every step

    Consider an 80-interaction repair trajectory:

    • The first 20 steps correctly reproduce and localize the issue.
    • At step 21, the agent misunderstands an interface constraint.
    • Later actions repeatedly modify the code around that incorrect assumption.
    • Final acceptance fails.

    If every action is labeled negative, the model may suppress the useful reproduction and localization behavior as well. A better objective is to identify the pivotal deviation and the missed opportunities to correct it afterward.

    4.2 Organize trajectories around decision points

    Raw logs may be long, but not every file read is equally important. A trajectory can be summarized around a set of decision points:

    1. What is the current hypothesis?
    2. What evidence supports it?
    3. What is the next action intended to verify?
    4. Does the observation support the hypothesis?
    5. Should the agent continue, roll back, or replan?

    The system should retain file diffs, tool inputs and outputs, test results, environment versions, and concise summaries of key decisions. These summaries should record verifiable judgments and evidence; they do not need to depend on undisclosed internal reasoning.

    4.3 Use checkpoints and branch replay to support attribution

    Saving an environment snapshot before an important modification makes it possible to try alternative actions from the same state. For example, before an incorrect patch is generated, restore the state and try:

    • Adding a boundary-condition check.
    • Inspecting callers of the interface.
    • Directly modifying the implementation.

    Comparing the outcomes under the same subsequent budget can help estimate which action was more valuable. However, the comparison is still affected by sampling randomness, later policy decisions, and environmental noise. A single replay is not enough to establish causality.

    In practice, replay should focus on high-value decision points rather than exhaustively covering every step, or training costs will grow rapidly.

    4.4 Treat process rewards as supporting evidence

    Process rewards can encourage useful behavior such as reproducing the defect, performing effective diagnosis, and respecting permissions. They should not simply reward “running a test” or “producing a patch.”

    Otherwise, the model may repeatedly run irrelevant checks or create large numbers of useless modifications to collect process points.

    A safer design uses credible final acceptance as the primary objective and process signals to improve learning efficiency. Those signals should be continuously checked for correlation with genuine task success.

    5. Preventing Agents from Modifying Tests to Game the Score

    When an agent is rewarded for making tests pass and is also allowed to modify the tests, it may replace the actual task with an easier one.

    Common reward-hacking strategies include deleting assertions, skipping failing cases, narrowing test discovery, catching exceptions and returning a fixed result, or writing special branches that handle only visible inputs.

    This does not require deliberate deception. If the training mechanism consistently rewards these shortcuts, the learned policy may gradually favor them.

    5.1 Separate validation assets from the workspace

    Final acceptance should run in an independent evaluation environment. Test assets, evaluation configuration, and result aggregation logic should not be freely modifiable by the agent.

    Possible controls include:

    • Set evaluation files to read-only.
    • Check the integrity of protected assets before and after the task.
    • Reapply the patch from a clean snapshot.
    • Rebuild and run acceptance checks in an independent environment.
    • Mark submissions with unauthorized changes as invalid.

    Rather than relying on an instruction such as “do not cheat,” make these operations ineffective through permissions and execution design.

    5.2 Distinguish development tests from acceptance tests

    During development, the agent may be allowed to add tests to reproduce a defect or document expected behavior. However, newly added tests should not automatically become the sole basis for acceptance.

    If modifying existing tests is genuinely part of the task, define that task separately, specify which test changes are required, and retain independent behavioral validation.

    Hidden tests are not a complete solution either. If their coverage is weak, an agent may still submit a patch that works only for a narrow set of cases. Acceptance should therefore combine regression checks, boundary conditions, patch review, and property-based checks where appropriate.

    5.3 Make the evaluator produce auditable evidence

    Evaluation results should contain more than “pass” or “fail.” They should also record:

    • Which checks were actually executed.
    • Whether any checks were skipped.
    • Whether execution completed fully.
    • Whether a timeout or resource limit occurred.
    • Whether protected files changed.
    • Which code and environment version produced the result.

    This evidence helps prevent gaming and distinguish model failures from evaluation infrastructure failures.

    6. Building Training Environments When Data Growth Slows

    A data plateau does not mean learning must stop. More often, the marginal value of adding homogeneous samples has declined. At that point, environment quality and task distribution matter more than simply increasing text volume.

    6.1 Build a reproducible task factory

    A training task should start from a fixed snapshot and reliably reproduce the initial failure. For tasks involving external resources, random seeds, or concurrent behavior, the relevant conditions must be controlled explicitly.

    The following samples should be removed or isolated:

    • Defects that cannot be reproduced consistently.
    • Environments that are already broken at initialization.
    • Acceptance criteria that conflict with the requirements.
    • Tests that pass without distinguishing a real fix from a no-op.

    Otherwise, the model learns from noisy rewards and may mistake an infrastructure failure for a code defect.

    6.2 Use controlled perturbations to train recovery

    Limited, interpretable perturbations can be injected into valid tasks. Examples include providing an irrelevant search result, leaving an incorrect candidate patch in place, or making one tool execution return a recoverable error.

    The goal is to teach the agent to inspect the state and rebuild its evidence chain, rather than mechanically retrying.

    Perturbations must remain controlled. They should resemble realistic usage while keeping the task solvable. Evaluation should also report recovery from naturally occurring failures separately from recovery from artificial perturbations. Otherwise, gains on a particular perturbation may be overstated as general reliability.

    6.3 Control data leakage

    Training and evaluation sets should not be split randomly at the individual-task level alone. Similar defects, near-identical patches, and repeated descriptions from the same codebase can create hidden leakage.

    A more reliable split considers code provenance, time, defect type, and similarity. Evaluation tasks should be isolated accordingly. This makes the measurement closer to transfer and diagnostic ability rather than memorization of existing patches.

    7. Why the Compute and Execution System Also Needs to Change

    Multi-turn agent training is not just text generation. A single task may involve file access, environment startup, code execution, test waiting, and result validation. Model computation and environment execution have different resource patterns and should not all be placed in one synchronous pipeline.

    7.1 Decouple generation, execution, and learning

    The system can be divided into four types of components:

    • Generation: Proposes the next action.
    • Execution: Performs controlled operations and runs tests.
    • Evaluation: Conducts trustworthy acceptance checks.
    • Learning: Consumes qualified trajectories and updates the policy.

    These components can coordinate through task state and messaging. This reduces idle capacity while the model waits for execution results.

    Asynchronous execution also introduces costs. A queued trajectory may have been generated by an older policy. The system therefore needs to record policy versions, limit staleness, and use data filtering or correction methods suited to the selected algorithm.

    7.2 Optimize environment cost, not only inference speed

    Dependency caches, environment snapshots, and staged validation can substantially reduce waiting time.

    For example, the system can first run lightweight checks directly related to the change, followed by the necessary full regression suite. Fast checks provide intermediate feedback but cannot replace final acceptance.

    Caches must be bound to the code, dependencies, configuration, and execution conditions. Otherwise, a passing result from an older state may be incorrectly reused for a new patch.

    7.3 Allocate budgets according to task state

    Simple tasks do not need unlimited exploration, while complex tasks should not be terminated mechanically.

    The budget can be adjusted using signals such as failure type, remaining hypotheses, and recent information gain:

    • When the same failed action is repeated, trigger diagnosis or terminate.
    • When there is clear evidence of a fix, prioritize final acceptance.
    • When a high-value hypothesis remains unverified, allow limited additional exploration.
    • When the environment is unavailable, stop meaningless code modifications.

    The budget policy itself must also be evaluated. It should not rely solely on the model’s subjective judgment about whether it is close to success.

    8. Metrics for Measuring Real Reliability

    Final success rate is important, but it is not sufficient to explain system behavior. Track the following metrics together:

    Metric Primary question
    Independent acceptance rate Does the final delivery satisfy the task requirements?
    Conditional recovery rate Can the agent complete the task after entering a specified error state?
    Regression rate Did the fix break existing behavior?
    Cost and latency distribution How many interactions, compute resources, and waiting time does success require?
    Invalid repetition rate Does the agent enter loops without gaining information?
    Unauthorized or gaming behavior Does it attempt to bypass trustworthy acceptance?
    Cross-task generalization Can it handle unseen code and defect types?

    Recovery rates must specify the source of the error, where it was injected, the remaining budget, and the task distribution. Without this information, comparisons between systems are difficult to interpret fairly.

    Cost should not be reported only as an average. Long-tail tasks often reveal system bottlenecks more clearly than typical cases.

    Conclusion

    The reliability of a long-horizon coding agent is not obtained by simply multiplying per-step accuracy, nor does it emerge automatically from adding more tools.

    It requires three coordinated loops:

    • A task loop built around evidence and validation.
    • A learning loop built around failure attribution and policy updates.
    • A trust loop built around permission isolation and independent acceptance.

    Supervised fine-tuning provides basic behavioral patterns. External frameworks provide execution structure. Multi-turn reinforcement learning helps optimize long-term decisions. Training environments and compute systems determine whether these capabilities can improve consistently.

    The real goal is not to ensure that the agent never takes a wrong step. It is to make errors visible early, limit their impact, and help the agent return to a verifiable path within a clear budget.