The New Trend in LLM Iteration: Post-Training RL and Long-Horizon Agent Practices

Recent flagship large language models (LLMs) share three prominent characteristics: parameter scales entering the trillion-level, context windows reaching the million-token mark, and capability enhancements heavily focused on code generation, complex tool calling, and long-horizon task execution. While some vendors claim their latest models rival top closed-source models in code, Agent, and knowledge benchmarks—and offer high-speed versions with a 2x to 3x throughput improvement—developers often find that running a single long-horizon Agent task can exhaust their API quota in just ten minutes.

Accelerating, becoming more capable, and burning money: these three phenomena occur simultaneously for a reason. This article breaks down the technical logic behind this model upgrade wave from five perspectives: training methods, infrastructure, evaluation systems, cost structures, and deployment models.

1. From Parameter Scaling to Post-Training: The Shift in Focus

Judging by parameters and benchmark scores alone, this might seem like a routine iteration. However, the real transformation lies in the training methodology. Compared to previous generations, the most critical change is a massive increase in post-training compute and the expansion of reinforcement learning (RL) from a few general capabilities to hundreds of downstream domains. This means capability improvements no longer primarily come from “feeding more pre-training data,” but rather from “repeated trial and error in real task environments, shaped by reward signals.”

Why has post-training become the new battleground? There are three main reasons:

1. Diminishing returns on pre-training: High-quality public corpora are approaching their extraction limits, and the cost-effectiveness of simply scaling data and parameters continues to decline.
2. Highly scenario-specific downstream needs: Capabilities like writing code, conducting financial research, or operating 3D creation tools are difficult to acquire solely through next-token prediction. They must be honed via RL in verifiable environments.
3. Agents as the main battlefield: The value of models is shifting from “answering questions” to “getting work done.” Sustained task execution is exactly what RL excels at shaping—as long as the environment can provide clear success or failure signals.

As for the high-speed versions, the 2x to 3x throughput improvements typically stem from system-level optimizations on the inference side. Common industry techniques include speculative decoding, KV cache reuse, quantization, and batch scheduling. The essence is “equal intelligence, higher throughput,” rather than sacrificing capability for speed.

2. Infrastructure Overhaul for Large-Scale RL

Scaling RL to hundreds of downstream domains is not as simple as tuning hyperparameters. It imposes systematic requirements on the training infrastructure. Leading teams generally reconstruct the following layers:

• RL Training Framework: Every RL training step requires numerous rollouts (the model actually executing tasks in an environment). Inference and training alternate, and the bottleneck is rarely the GPU compute itself, but rather the framework’s scheduling and data transfer efficiency.
• Long-Context Inference: Million-token contexts mean a steep increase in the memory footprint and management complexity of the KV cache. This requires paired inference optimizations like paged caching and prefix reuse.
• Sandbox Infrastructure: Code execution, terminal operations, and tool calls must maintain isolation and low latency under high concurrency.
• Scheduling System: Tens of thousands of concurrent environment instances, heterogeneous compute, and varying task durations require a highly fault-tolerant scheduling layer.
• Environment Quality Assurance Platform: One of the biggest risks in RL is “reward hacking”—where the model exploits environment loopholes for high scores without solving the real problem. Continuous quality checks on environments and reward signals are critical to keeping training on track.

This system can be summarized with an illustrative training loop:

# Illustrative: Multi-domain joint RL training main loop (not a real interface)
for step in range(train_steps):
    domains = sample_domains(weights=domain_weights)  # Sample hundreds of downstream domains by weight
    for d in domains:
        env = env_pool.acquire(d)                     # Acquire a sandbox instance from the pool
        traj = policy.rollout(env, max_steps=H)       # Model executes multi-step actions in the environment
        reward = verifier.score(traj)                 # Result verification + rule-based reward
        env_pool.release(env, health_check=True)      # Reclaim environment after health check
    buffer.extend(collect())
    policy.update(buffer)                             # Policy update

This pseudocode illustrates that modern LLM RL is already a complete industrial pipeline of “environment production — sandbox execution — reward verification — policy update.” The upper limit of a model’s capability increasingly depends on the quality of this pipeline, not just the model architecture itself.

3. Long-Horizon Agents: Evaluation and Strengths

Long-horizon tasks are the focal point of this upgrade wave. “Long-horizon” means the task cannot be completed in a single step; it requires the model to maintain goal alignment across dozens or even hundreds of steps: multi-step planning, tool calling, modifying plans based on intermediate results, and ultimately delivering an end-to-end result.

To evaluate these capabilities, the industry currently relies on two main types of benchmarks:

• Real Terminal Environment Tests: The model is placed in a resource-constrained real terminal environment with specific verifications for each task, testing the coding Agent’s ability to complete complex engineering tasks. Public benchmarks show that the latest generation of models has entered the top tier in these tests, narrowing the gap with the strongest closed-source models to less than 1 percentage point.
• Long-Horizon Code Tests: Covering multiple engineering domains, these focus on evaluating multi-step execution, tool calling, and end-to-end problem-solving. One vendor disclosed that its internal long-horizon code benchmark score increased from about 56% in the previous generation to over 61%, a 5-point improvement.

When looking rationally at these numbers, three things must be noted:

1. Benchmarks are just sampling: The real task distribution is far more complex than benchmarks. Similar scores do not equal a consistent user experience.
2. Focus on environment coverage: The closer the test environment is to real-world engineering (real terminals, real codebases, real resource limits), the higher its reference value.
3. Beware of overfitting: If the evaluation environment shares the same source as the training environment, scores may be inflated. This is why an environment quality assurance platform is so important.

4. Token Economics: Why Agent Tasks Burn Quotas in Minutes

New-generation model APIs generally adopt tiered pricing. The mainstream industry structure can be summarized into three tiers: cached input is the cheapest (usually about a tenth of the uncached input), uncached input is next, and output is the most expensive (often several times the input price). For one new model, the prices per million tokens for cached input, uncached input, and output are separated by roughly tenfold and fivefold gaps, respectively.

This explains the common experience developers have when running long-horizon Agents for the first time: the quota is gone in ten minutes. The technical reasons are fourfold:

1. Context Snowballing: The Agent carries its full history at every step. The input length at step N is roughly the sum of all previous N-1 interactions, meaning input tokens accumulate quadratically with the number of steps.
2. Tool Output Inflation: A single file read or log query might return tens of thousands of tokens, and these contents are repeatedly billed at the input price.
3. Retries and Reflections: Failures, error corrections, and replanning are routine in long-horizon tasks. Every retry is a completely new billing cycle.
4. Multi-Agent Parallelism: Splitting tasks among multiple sub-Agents to speed up execution multiplies token consumption accordingly.

The corresponding engineering countermeasures are equally clear:

• Leverage Prefix Caching: Keep stable system prompts and task descriptions fixed at the beginning of the context. Once cached, input costs can drop by an order of magnitude.
• Compress Tool Outputs: Summarize, truncate, or structurally extract tool return results, writing only essential information into the context.
• Task Decomposition and Checkpoints: Slice long tasks into phased sub-tasks. Persist intermediate results to avoid repeatedly replaying the full history.
• Budget Circuit Breakers: Set a token or monetary limit for each run; terminate if the limit is exceeded.

# Illustrative: Agent main loop with budget control and context compression (not a real interface)
class BudgetedAgent:
    def __init__(self, client, tools, token_budget):
        self.client, self.tools = client, tools
        self.remaining = token_budget
    def run(self, task):
        history = [system_prompt(task)]        # Fixed prefix to facilitate cache hits
        while True:
            if self.remaining <= 0:
                raise BudgetExceeded           # Budget circuit breaker
            resp = self.client.chat(history, tools=self.tools)
            self.remaining -= resp.usage.total_tokens
            if resp.tool_call:
                out = self.tools.execute(resp.tool_call)
                history.append(compress(out))  # Compress tool output before adding to context
            else:
                return resp.text               # Task complete, deliver result

Before pushing an Agent to production, it is recommended to perform cost modeling first: run through the full workflow with a typical task, record the input/output token distribution at each step, and then convert it to a per-task cost based on the tiered pricing. Empirically, the per-task cost of a long-horizon Agent is often dozens of times that of a one-shot Q&A for the same content, so budget design must account for this premise.

5. Harness as a Service: Agent Runtimes as Cloud Infrastructure

Another notable signal in this upgrade wave is that leading vendors are beginning to offer the Harness (runtime environment) supporting long-running Agents as a cloud-hosted service, enabling 7x24 long-horizon tasks and multi-agent collaboration. Enterprises can host long-running agents in the cloud without building their own sandboxes, scheduling, and monitoring systems.

The underlying logic is that as Agents move from “demos” to “production,” their runtime complexity surpasses that of the model itself. Process isolation, credential management, failure recovery, audit logging, and concurrent scheduling are each independent engineering challenges. Consolidating these into cloud-hosted capabilities essentially upgrades a “model API” into a “work API.”

Cloud providers are moving in the same direction. Services represented by Baidu AI Cloud already cover LLM APIs, Agent orchestration, hosted runtime environments, and general capabilities like monitoring, alerting, and logging. Developers can combine these on-demand to build long-horizon Agent applications without building the entire Harness from scratch. When selecting a provider, it is recommended to focus on evaluating four aspects:

1. Isolation and Security: Is the code execution environment strongly isolated? Are credentials restricted to least privilege?
2. Observability: Can the token consumption, tool calls, and failure reasons for each step be traced?
3. Elasticity and Recovery: Can long tasks resume from checkpoints after an interruption?
4. Cost Transparency: Are the token metering standards and caching strategies clear and predictable?

6. Four Practical Recommendations for Developers

1. Evaluate First: Don’t just look at public leaderboards. Build a private evaluation set with 20 real tasks from your business to test the multi-step completion rate and unit cost of candidate models.
2. Monitor Weight Release Schedules: Some vendors promise to release full model weights shortly after launch. This opens a window for private deployment and deep customization, making it worthwhile to plan compute and deployment strategies in advance.
3. Treat Cost as a First-Class Citizen: Introduce budget controls, caching design, and output compression during the Agent architecture design phase, rather than being forced to refactor by an unexpectedly high bill after going live.
4. Prioritize Verifiable Tasks: Capabilities shaped by RL yield the highest returns in scenarios with verifiable outcomes (code, data pipelines, form processing). Implementation should start from these scenarios.

Conclusion

The real highlight of this model upgrade wave is not parameters jumping another order of magnitude, but the simultaneous occurrence of two things. On the training side, post-training RL has been systematically expanded to hundreds of downstream domains, and models are being trained as “long-term employees” for the first time. On the application side, Agent runtimes are becoming cloud infrastructure, bringing the engineering thresholds and cost structures of long-horizon tasks to the forefront. For developers, whoever can faster establish an engineering methodology that unifies “evaluation, cost, and runtime” will gain the upper hand in this wave of Agent-ification.