RL Takes Center Stage: Post-Training and Long-Horizon Agents as the Next Frontier for LLM Scaling

1. The Main Upgrade Path: Compute Shifts from Pre-Training to Post-Training

Over the past two years, competition in large language models focused heavily on pre-training scale. But across recent flagship model releases, the center of gravity for compute investment is clearly moving toward post-training, especially reinforcement learning (RL). The latest generation of models shows three common characteristics:

1. Parameter counts are moving into the trillions, with context windows reaching the million-token level;
2. Post-training compute is increasing, and RL is expanding from general alignment tasks into hundreds of downstream domains;
3. The emphasis is shifting from “answering well” to “getting things done” — with code, long-horizon task execution, and tool use becoming priority areas for reinforcement.

The logic is straightforward: pre-training builds the knowledge and language foundation, but its marginal returns are diminishing. Post-training RL directly optimizes the action sequences needed to complete tasks in real environments. For domains where outcomes can be verified, such as programming, financial research, and 3D content creation, the gains are immediate. A model’s capability increasingly depends on the quantity and quality of RL environments, not just parameter stacking.

2. Infrastructure Overhaul for Large-Scale RL

Expanding RL into hundreds of downstream domains is not as simple as writing a few more reward functions. It requires a systematic reworking of the entire training stack. The following areas typically need to be addressed:

• RL frameworks: They must support large-scale parallel rollouts and alternating training-inference scheduling. Single-machine, single-GPU setups are no longer viable;
• Long-context inference: Rollout phases can easily generate trajectories with hundreds of thousands of tokens, placing extreme demands on KV cache management and memory scheduling;
• Training efficiency: Heterogeneous hardware utilization, gradient communication, and checkpointing strategy directly determine iteration speed;
• Sandbox infrastructure: Code tasks must actually execute in isolated environments to produce trustworthy reward signals;
• Scheduling systems: Starting, stopping, queuing, and reclaiming thousands of environment instances require dedicated design;
• Environment quality auditing: If an environment has flaws that allow the reward to be “gamed,” reward hacking becomes a risk. Quality sampling is essential.

A simplified RL training loop looks like this:

# Main loop for large-scale RL training (illustrative)
for batch in domain_sampler.sample():          # Sample tasks from hundreds of downstream domains
    trajectories = policy.rollout(batch.env)   # Execute trajectories in the sandbox
    rewards      = batch.env.verify(trajectories)  # Score with the verifier
    rl_trainer.update(policy, trajectories, rewards)
    env_qc.audit(batch.env)                    # Audit environment quality to prevent reward hacking

The purpose of this overhaul is to determine how broadly RL can be deployed. The more diverse the environments and the more reliable the verification, the more robust the model’s generalization will be on real tasks.

3. Million-Token Context: From “Fitting Everything In” to “Using It Well”

The value of a million-token context window shows up in three kinds of real-world scenarios:

1. Repository-level code understanding: Reading an entire mid-sized codebase in one pass to support cross-file refactoring, dependency mapping, and global impact analysis;
2. Long-document research: Cross-referencing and synthesizing hundreds of pages of financial reports, prospectuses, and legal documents;
3. Long-horizon agent memory: After hours of task execution, early observations and decision rationale remain inside the window.

Engineering practice requires attention to three points. First, long context does not mean perfect memory. The “lost in the middle” phenomenon still exists, so critical constraints should be placed at the beginning or end of the prompt. Second, attention computation and KV cache memory overhead grow with length, making long inputs noticeably slower and more expensive than short ones. Third, when cache is hit, repeated prefixes — system prompts, repository summaries, and other stable content — can significantly reduce actual cost. This is exactly why the pricing structure in the next section matters.

4. How to Evaluate Coding Agents: Terminal Benchmarks and Long-Horizon Evaluation

There are currently two main paradigms for evaluating coding agents:

Terminal environment benchmarks: Agents complete complex tasks in a real, resource-constrained terminal environment. Each task comes with dedicated test cases, and a score is only awarded when the task runs end-to-end and passes validation. The goal is not “writing a piece of code” but “getting things done on a real machine.” Recent public results show top models scoring in the 60% range on such benchmarks, with gaps between them narrowing to one or two percentage points.

Long-horizon code evaluation: These benchmarks span multiple engineering domains and focus on multi-step execution, tool use, and end-to-end problem solving. Newer-generation models typically show improvements of several percentage points over their predecessors.

When interpreting these scores, four principles are worth keeping in mind:

• A one or two percentage point gap is largely within evaluation noise and should not determine model selection;
• The closer the benchmark task distribution is to your real business distribution, the higher its reference value;
• Agent output is stochastic. Variance across multiple runs of the same task is significant, so run each task at least three to five times and use the mean;
• Watch out for data contamination: benchmark problems may have already entered training corpora. Private evaluation sets are irreplaceable.

A minimal private evaluation skeleton:

# Minimal skeleton for a private evaluation set (illustrative)
tasks = load_private_tasks("my_repo_tasks/")   # Each task includes environment and verifier
for model in candidates:
    scores = [run_agent(model, t).verify() for t in tasks for _ in range(3)]
    report(model, mean(scores), stdev(scores))  # Report mean and variance together

5. High-Speed Variants and Three-Tier Pricing: The Economics of Inference

The latest flagship models generally adopt a dual-version strategy: the standard version and a high-speed version share the same underlying model capability, while the latter improves output speed by roughly two to three times through inference optimization at around twice the price. Common optimization techniques include speculative decoding, quantization, operator fusion, continuous batching, and KV cache reuse.

On pricing, mainstream APIs generally use a three-tier structure: cache-hit input, cache-miss input, and output. Their typical relative magnitudes are roughly 1 : 10 : 50 — cache-hit input costs about one-tenth of cache-miss input per token, and output costs about five times cache-miss input.

This structure matters enormously for agent workloads. Agents are inherently multi-turn loops: each step carries all previous history and tool outputs, so cumulative input tokens far exceed output tokens. Single-task cost can be estimated as follows:

def estimate_agent_cost(turns, avg_ctx_tokens, out_per_turn,
                        cache_hit=0.7, p=1.0):
    """p: cache-hit input price (per million tokens). Other tiers estimated as 10p / 50p."""
    M = 1_000_000
    total_in = turns * avg_ctx_tokens
    return (total_in * cache_hit / M * p
          + total_in * (1 - cache_hit) / M * 10 * p
          + turns * out_per_turn / M * 50 * p)
# Example: 20 turns, 80k average context tokens per turn, 2500 output tokens per turn, 70% cache-hit rate
print(estimate_agent_cost(20, 80_000, 2_500))        # ≈ 8.4 (with the example unit price)
print(estimate_agent_cost(20, 80_000, 2_500, 0.0))   # No cache at all ≈ 18.5

For the same task, dropping the cache-hit rate from 70% to zero more than doubles the cost. This also explains why trial quotas sometimes seem to burn through in minutes: a long-horizon agent task can easily consume millions of tokens, and without caching and budget controls, usage drains far faster than intuition suggests.

6. Hosted Agents: The Harness as Infrastructure

Exposing an API alone is no longer enough for long-horizon agents. A new industry trend is to provide the harness — the execution scaffolding that keeps an agent running for extended periods — as cloud infrastructure. This includes sandbox runtimes, tool environments, task scheduling, state persistence, logging and monitoring, multi-agent orchestration, and typically support for 24/7 uninterrupted tasks.

For enterprise developers, hosted solutions offer several benefits:

• No operational burden of building and maintaining sandboxes and scheduling systems;
• Environment isolation and operational auditing naturally support compliance requirements;
• Checkpoint recovery and automatic retry for long-running tasks are handled by the platform.

Self-hosted approaches offer more control and potentially lower cost. If self-hosting, teams can assemble the stack from general cloud components — for example, container services for sandboxes, object storage for intermediate artifacts, message queues for task decoupling, and logging plus monitoring for observability. Major cloud providers offer these foundational capabilities.

7. Why Tokens Burn Fast: Consumption Anatomy and Control Strategies

Token consumption in agent tasks comes from three compounding effects:

1. Turn amplification: Every step rereads the full history. For an n-turn task, total input is roughly “average context × number of turns”;
2. Tool output amplification: A single file read or command output can inject thousands to tens of thousands of tokens;
3. Retry amplification: Failed runs mean paying for the same context again.

Corresponding control strategies:

• Stabilize prompt prefixes: Put system prompts, repository summaries, and other fixed content first to maximize cache hits;
• Truncate and summarize tool outputs: Avoid dumping everything into context;
• Decompose tasks: Let the main agent keep only waypoints, delegate heavy work to sub-agents, and return conclusions rather than full processes;
• Budget guardrails: Set token limits and checkpoint recovery points for each task;
• Tiered model use: Use high-speed or lightweight models for drafting, retrieval, and formatting, and reserve the flagship model for critical decisions.

8. Open Weights and a Developer Adoption Checklist

This generation of flagship models generally commits to releasing full model weights shortly after launch. Open weights make private deployment, domain fine-tuning, and data compliance viable options for teams that handle sensitive data and have stable GPU budgets.

A practical adoption checklist for developers:

1. Evaluate before switching: Compare old and new models on a private task set, looking at both mean and variance;
2. Cost first: Estimate per-task cost before rollout using the formula in section five, and set budget alerts;
3. Cache first: Restructure prompts to keep stable content up front and improve hit rates;
4. Observability: Log per-turn token consumption and tool calls to identify where money is being spent;
5. Weight evaluation: If task volume is stable and data is sensitive, calculate the break-even point for private deployment;
6. Hybrid orchestration: Use the high-speed variant for interaction and fast iteration, and the standard version for final delivery.

Conclusion

The signal from this round of model upgrades is clear: competition is shifting from parameter counts and leaderboards toward the breadth of post-training RL coverage and the maturity of agent infrastructure. For developers, the real priority is not chasing any single release but building their own evaluation systems, cost models, and guardrail mechanisms — so that every jump in model capability can be reliably converted into engineering efficiency.