Long-Horizon Agent RL: A Low-Cost Path to High Performance
Agent systems are shifting from generating correct answers to reliably completing complex tasks over extended interactions. This article examines a recently disclosed approach that applies full-parameter reinforcement learning to a large general-purpose model, turning it into an agent for terminal and coding scenarios. The reported system achieves 46.2% Pass@1, or single-attempt success, on terminal-task evaluations while reducing inference cost per task by more than 30x compared with a high-intensity inference mode. It was trained on only 64 high-end GPUs in under five days. We discuss why long-horizon agents need RL, how full-parameter training affects task decomposition, tool use, and error recovery, why performance and cost can improve together, how to interpret benchmark results, and which engineering and enterprise practices matter.
In recent years, competition among agent systems has been shifting from whether a model can generate the correct answer to whether it can continuously complete complex tasks. In scenarios such as terminal operations, code modification, test execution, and troubleshooting, a model often has to perform dozens or even hundreds of consecutive actions. During this process, it must read files, interpret errors, adjust its strategy, and verify results. The central challenge is not single-turn text generation, but the combined ability to make long-horizon decisions, call tools, and use feedback from results.
A recent publicly disclosed study demonstrates a notable technical path. Starting from a large general-purpose model, the work applies full-parameter reinforcement learning to further optimize the model into an agent for terminal and coding scenarios. The reported results show a 46.2% single-attempt pass rate, or Pass@1, on terminal-task evaluations, placing the system at a level comparable to current high-performance systems. At the same time, inference cost per task was reduced by more than 30x compared with a high-intensity inference mode. More notably, training used only 64 high-end GPUs and was completed in under five days, and the research team disclosed a relatively complete training methodology.
1. Why Long-Horizon Agents Need Reinforcement Learning
Traditional supervised fine-tuning is well suited to teaching a model “how it should answer,” but a terminal agent operates in a dynamic environment. After the model takes an action, the environment returns a new state, and subsequent decisions may change as a result.
For example, a software repair task typically includes the following steps:
- Read the project structure and requirements description.
- Locate potentially relevant source code.
- Modify one or more files.
- Run builds, tests, or static checks.
- Analyze failure logs.
- Continue modifying based on feedback.
- Repeat validation until the task is complete.
In this process, it is not easy to judge whether an individual action is correct. The model may first make a change that looks reasonable but ultimately causes tests to fail, or it may gradually approach the correct answer through multiple attempts. Therefore, the training objective should not focus only on whether each step matches a demonstration. It must consider whether the entire task trajectory ultimately succeeds.
The advantage of reinforcement learning is that it can use environment outcomes as feedback. Task completion, passing tests, and patch validity can all be converted into reward signals that push the model to learn more effective action policies. For long-horizon tasks, this kind of outcome-oriented training is often more adaptive than simply imitating a fixed sequence of operations.
2. How Full-Parameter Reinforcement Learning Changes Capabilities
The key to this study is not merely increasing the number of inference attempts, but updating the model parameters as a whole. Full-parameter training means that the model’s foundational capabilities, tool-use habits, error-recovery strategies, and task-planning approaches can all be adjusted together during reinforcement learning.
Compared with methods that train only a small number of additional parameters, full-parameter reinforcement learning may provide three kinds of benefits.
2.1 Stronger Task Decomposition
Complex tasks usually require forming a plan before executing it step by step. After training, a model may be more inclined to inspect the environment, identify constraints, and plan its validation approach before immediately modifying files. This behavior can reduce ineffective actions and lower the accumulation of errors along long trajectories.
2.2 More Stable Tool Invocation
A terminal agent needs to frequently invoke tools for viewing files, searching, editing, building, and testing. The parameters, order, and frequency of tool calls all affect task outcomes. Reinforcement learning enables the model to adjust its invocation strategy based on actual feedback—for example, confirming context before making changes and running focused tests promptly after broad modifications.
2.3 Better Error Recovery
Long-horizon tasks cannot guarantee that every step is correct. An important capability of a high-quality agent is to detect errors and promptly roll back or change course, rather than continuing in the wrong direction. By comparing failed and successful trajectories, a model can gradually learn to recognize abnormal states and adopt more reasonable repair strategies.
3. Why Performance Gains and Cost Reduction Can Coexist
Normally, a stronger agent requires a longer reasoning chain, a higher sampling count, or a larger model scale, all of which directly increase per-task cost. However, the reported results indicate that performance and cost are not necessarily in strict opposition. The key is to reduce ineffective computation.
Cost reduction may mainly come from the following areas.
3.1 Fewer Ineffective Steps
If the model can understand the task objective earlier and generate a modification plan closer to correct on the first attempt, it can reduce repeated searches, repeated execution, and meaningless tool calls. For terminal tasks, avoiding even one complete build or test can often mean significant savings in time and compute.
3.2 Controlled Reasoning Depth
Not every task requires the highest-intensity reasoning. Simple tasks can use shorter decision chains, while complex tasks can gradually receive a larger budget. By training the model to allocate reasoning resources according to task difficulty, it is possible to avoid the waste of applying the highest configuration to every request.
3.3 More Efficient Local Deployment
When a model is deployed by the research team itself, inference can be optimized for batch processing, memory utilization, request scheduling, and caching strategies. Public prices are often only a uniform billing reference, while actual operating cost also depends on hardware utilization, concurrency level, task length, and the number of failed retries. Therefore, when evaluating agent cost, it is not enough to look only at the model unit price. It is also necessary to count the total number of calls required to complete one effective task.
4. How Evaluation Results Should Be Interpreted
The study validated the approach on multiple terminal and software-engineering benchmarks and reported results averaged over multiple trials. This design helps reduce the impact of single-run randomness, but readers should still pay attention to several important issues.
First, Pass@1 indicates the proportion of tasks completed successfully in one attempt. It reflects the system’s direct success capability under a limited inference budget. A higher value generally means that the agent relies less on repeated sampling and human selection.
Second, benchmark tests are not equivalent to a real production environment. Real projects may include more complex dependencies, longer build times, stricter permission restrictions, and incomplete requirement descriptions. Therefore, laboratory results can show that a model has strong potential, but they cannot directly replace internal enterprise validation.
Third, cost and performance must be analyzed on the same chart. Pursuing only the highest pass rate may cause inference overhead to grow rapidly; pursuing only the lowest price may increase failed retries. A more useful metric is the average cost of completing one effective task, including first-attempt failures, retries, tool calls, and validation.
5. Engineering Priorities for Training Long-Horizon Agents
Developers seeking to reproduce a similar approach should focus on the following areas.
5.1 Build a Reliable Interaction Environment
The training environment must accurately record file changes, command execution results, test output, and resource consumption. Any state-synchronization error can distort reward signals and, in turn, teach the model to engage in opportunistic behavior.
5.2 Design Layered Rewards
Using only final success or failure as the reward may make the training signal too sparse. Without changing the final objective, process-level evaluations can be added—for example, whether necessary checks were completed, whether obvious errors were introduced, whether local tests passed, and whether ineffective loops occurred.
An illustrative reward structure can be expressed as:
Total reward =Task completion reward+ Test pass reward- Penalty for ineffective actions- Timeout penalty- Resource consumption penalty
Note that process rewards must not dominate the objective. If “running tests” or “modifying code” is rewarded excessively, the model may learn to farm steps without actually completing the task.
5.3 Control Trajectory Length
Long trajectories can provide more learning information, but they also increase memory usage, training instability, and error accumulation. Engineering methods such as trajectory truncation, key-state retention, and resampling of failed samples can help focus training on segments with learning value.
5.4 Prevent Environment Gaming
An agent may achieve superficial success by deleting tests, modifying configurations, or skipping validation. Therefore, the evaluation environment should isolate test code and critical configurations, and it should perform independent verification after the task ends. Only when “the result is correct and the process complies with constraints” should the task be considered truly successful.
6. Implications for Enterprise Adoption
The most important value of this study is not simply the score achieved by a particular system. Rather, it shows that high-performance agents do not necessarily have to depend on extremely high inference costs. Through targeted training, sensible rewards, and system-level optimization, medium-scale models can also become highly competitive on specific tasks.
For enterprises, adoption can begin with low-risk, verifiable scenarios, such as code test generation, log analysis, dependency upgrades, simple defect fixes, and automated operations assistance. During deployment, organizations should also build mechanisms for task replay, permission control, operation auditing, cost monitoring, and human takeover.
Ultimately, whether an agent is worth using should not be judged only by model scale or public leaderboard ranking. It should be evaluated across four dimensions: task success rate, average completion time, per-task cost, and recoverability after failure. Only by balancing these four metrics can an agent deliver real engineering value.
Overall, the development of long-horizon agents is moving from “scaling model compute” toward coordinated optimization of training, environment, policy, and cost. The low-cost, high-performance path presented by public research provides an important reference for building general agents for terminal operations and software engineering.
