Long-Horizon Coding Agents: From Error Recovery to Reinforcement Learning Training

    This article explains why per-step accuracy alone does not predict task success for long-horizon coding agents. It covers how to model bug fixing as a closed-loop decision problem, why workflow orchestration and supervised fine-tuning fall short, how trajectory logging, counterfactual branching, and multi-turn reinforcement learning support failure attribution, how to prevent reward hacking, and how to design training environments, compute systems, and evaluation metrics around error recovery.

    1. Why Per-Step Accuracy Does Not Explain Task Success

    Suppose a task contains 100 steps, and each step is correct with 99% probability. The probability that all steps are correct is:

    0.99^100 ≈ 36.6%.

    The calculation itself is correct. However, using it directly as the task success rate of a coding agent assumes several things: the number of steps is fixed, the probability of each step being correct is unaffected by history, and any single error leads to final failure.

    Real software repair often violates these assumptions.

    A developer may first locate the wrong file, then find the real entry point through call relationships. A developer may also write an incorrect patch, run the tests, revert the change, and re-analyze the boundary conditions. As long as an error does not cause irreversible damage, later actions can bring the task back on track.

    Three metrics must therefore be distinguished:

    • Single-step correctness: Whether an action is reasonable in the current state.
    • Error recovery ability: After entering an erroneous state, whether the agent can identify the problem and regain effective progress.
    • End-to-end success rate: Whether the real goal is completed within the specified time, tool permissions, and compute budget.

    An agent may explore inefficiently yet still complete tasks through verification and rollback. Another agent may look reasonable for dozens of steps but submit an unverified patch at the end. The task value of the two cannot be judged from single-step performance alone.

    A simplified model illustrates the importance of recovery. Suppose the success probability of a single attempt is p, and failed attempts can be effectively retried, with attempts independent of each other. Then the probability of success within at most k attempts is:

    1 - (1 - p)^k.

    This is not a prediction formula for the actual success rate of an agent. In practice, retries are usually correlated, and the environment may not be fully restorable. The model only shows that when a system allows error correction, task performance no longer depends on the probability of getting every step right on the first try.

    The source material mentions several error statistics and training experiments, but it lacks verifiable paper information, experimental conditions, and metric definitions. The proportions should not be treated as general conclusions. A more reliable subject of discussion is the technical problem behind these descriptions: how to train agents to keep making effective progress after a failure.

    2. Modeling Bug Fixing as a Closed-Loop Decision Problem

    Automated bug fixing is not “reading a problem and generating a block of code.” It is decision-making in a changing workspace.

    A typical process includes:

    1. Reading the problem description and runtime constraints.
    2. Retrieving relevant code and forming initial hypotheses.
    3. Constructing reproduction conditions.
    4. Modifying the implementation.
    5. Running targeted tests.
    6. Continuing to localize the problem based on failure information.
    7. Expanding the verification scope.
    8. Reviewing the patch and submitting it.

    In this process, the model often sees only file fragments, logs, and test results, not the complete system state. It is therefore better understood as a partially observable sequential decision problem.

    Training and evaluation need to define at least four elements.

    State and observations. The workspace contents, dependency state, existing modifications, test output, and remaining budget jointly affect the next action. Observations are only part of the state. Not seeing an anomaly does not mean the system has no anomaly.

    Actions. These include reading, retrieving, editing, running verification, reverting changes