LLM Capability Leaps: Post-Training RL and Long-Horizon Agent Practices

From the Pretraining Race to the Deep End of Post-Training

Over the past two years, the center of gravity in the large language model industry has visibly shifted. As marginal returns on pretraining data and compute diminish, top teams are directing more resources toward post-training—the stage where a base model is continuously refined through supervised fine-tuning (SFT) and reinforcement learning (RL) to strengthen its reasoning, coding, and tool-use capabilities.

Looking across the flagship models released recently, three highly consistent traits stand out:

1. Post-training compute has grown substantially. Its share of the overall training budget keeps rising, and it has become a key variable in widening capability gaps.
2. RL has expanded from single-purpose alignment to broad domain coverage. It now goes beyond traditional preference alignment into code, math, tool calling, financial research, 3D creation, and hundreds of other downstream domains.
3. Long-horizon agent capability has become a core benchmark. Models are now expected to work autonomously for hours at a time and complete multi-step tasks, not merely answer one-off questions.

Following this main thread, this article breaks down the technical building blocks, evaluation methods, and engineering practices behind these capability leaps, helping developers build a clearer framework for model selection and deployment.

Scaling RL to Hundreds of Domains: Where the Difficulty Lies

Traditional RLHF aims to make a model’s answers “more aligned with human preferences.” The new generation of large-scale RL aims to make a model “actually get the task right.” The two differ by orders of magnitude in engineering complexity, and the main challenges concentrate in four areas:

• Reward signal design. Whether code passes tests or whether a math answer is correct falls under verifiable rewards that can be scored automatically. Subjective tasks—such as the quality of a research report or the aesthetics of a 3D asset—rely on reward models or scoring rubrics, which introduce noise and can even invite reward hacking.
• Task environment quality. An RL system’s capability ceiling is set by its environment. Low-quality, buggy task environments teach the model opportunistic shortcuts, so dedicated environment QA platforms are needed to continuously audit task solvability and judge correctness and to iterate on them.
• Sandbox infrastructure. Code execution, dependency installation, and browser operations must all run inside isolated sandboxes, and the system must support massive concurrent rollouts. This places extreme demands on container scheduling, filesystem snapshots, and network isolation.
• Training efficiency and scheduling. In long-context RL, a single trajectory can span hundreds of thousands of tokens. KV cache memory footprint, the compute ratio between rollout and training, and load balancing across heterogeneous tasks all require a holistic redesign of the training framework and scheduler.

A simplified multi-domain RL training loop looks like this:

# Large-scale multi-domain RL training (illustrative)
for batch in task_stream:             # mixed sampling across hundreds of downstream domains
    traj = agent.rollout(batch.env)   # execute tasks step by step in a sandbox
    reward = verifier.score(traj)     # verifiable reward or reward model scoring
    trainer.update(traj, reward)      # policy gradient update
    env_qa.audit(batch.env, stats)    # continuous environment quality checks

As this shows, RL at scale is fundamentally a “systems engineering upgrade”: beyond model algorithms, sandboxes, scheduling, and environment QA together form the foundation of the capability leap.

Evaluating Coding Agents: From Solving Problems to Doing Real Work

Early code evaluation relied mainly on function-level unit tests: closed problems with single steps, which tend to overstate a model’s real engineering ability. Two evaluation approaches are now more widely accepted:

1. Real terminal environment evaluation. The model gets a genuine shell inside a resource-constrained container and must install dependencies, modify code, and run tests until it passes a dedicated grading script. This measures “given a computer, can you get the job done”—a scenario closely isomorphic to everyday development.
2. Long-horizon code evaluation. Spanning multiple engineering domains, this approach emphasizes multi-step execution, tool calling, and end-to-end problem solving; a single task often requires dozens of tool interactions to complete.

Whichever approach is used, three design points deserve attention:

• Assign independent grading logic to each task so the model cannot “guess the answer”;
• Use environment snapshots and network isolation to keep results reproducible;
• Strictly separate evaluation sets from training data to prevent inflated scores caused by data contamination.

The takeaway for enterprises: public leaderboards are only useful for initial screening. What really determines model selection is an internal evaluation set—extract real issues from your own codebase, build tasks with automatic grading, and treat pass rate as the hard metric.

Long Context and Fast Inference: Balancing Experience and Cost

Million-token context windows are moving from flagship exclusivity toward the mainstream, but the engineering challenges are substantial: the quadratic complexity of attention, KV cache memory pressure, and the extrapolation stability of positional encodings all need to be optimized on both the model architecture and the inference system side.

High-speed model variants take a different productization path: keep the model weights and intelligence unchanged, and trade inference-side optimizations for several times the output speed. Common techniques include:

• Speculative decoding: a small model drafts tokens and the large model verifies them, producing multiple tokens per forward pass;
• Quantization and operator fusion: reduce memory bandwidth pressure and raise per-GPU throughput;
• Continuous batching and Prefill/Decode disaggregation: improve overall cluster utilization.

For selection, a simple principle applies: interactive scenarios (pair programming, real-time Q&A) are sensitive to first-token latency, so prefer the high-speed variant; offline batch processing and deep research tasks are latency-insensitive, so the standard variant is more cost-effective.

Understanding Three-Tier Pricing: How Caching Reshapes Cost Structure

Mainstream LLM APIs today generally use a three-tier pricing structure—cache-hit input, cache-miss input, and output—with cache-hit prices typically around one-tenth of cache-miss prices. The underlying technology is prefix caching: the KV cache for an identical prefix can be reused across requests, eliminating redundant computation.

This directly rewrites prompt engineering best practices: put stable content first and dynamic content last.

[System prompt + project standards + long documents]   ← stable prefix, cache hits across requests
[Current user question + minimal context]              ← dynamic suffix, billed at the cache-miss rate

A common anti-pattern is inserting volatile fields such as timestamps or random IDs at the start of the system prompt, which makes the cache miss every time. In long-context applications, merely reorganizing the order of content can produce several-fold differences in input cost.

Agent Infrastructure: From Calling APIs to Managed Runtimes

A model API solves “one call”; an agent needs “continuous work.” To keep an agent actually running, you need a complete runtime shell: tool registration and invocation, sandboxed execution, state and memory management, task scheduling, and failure retry. The industry trend is to consolidate this shell into managed services that support round-the-clock long-horizon tasks and multi-agent collaboration.

Whether you build your own or adopt managed agent infrastructure, evaluate four dimensions:

• State persistence and resumability. Long tasks must support checkpoints and failure recovery so a single crash doesn’t wipe out all progress;
• Observability. Every tool call, token consumption, and intermediate artifact should be logged and queryable, backed by logging services, monitoring and alerting, and distributed tracing;
• Security boundaries. Sandbox isolation, least-privilege access, and secret management to prevent agents from overstepping their authority;
• Cost guardrails. Budget caps, usage alerts, and automatic fallback policies to keep long-horizon tasks from burning money uncontrollably.

For the deployment architecture, mature cloud-native components can be reused: container platforms to host agent workloads, message queues to decouple task distribution, and object storage to retain intermediate artifacts. On the inference side, you can plug into the LLM services and associated governance capabilities offered by platforms such as Baidu AI Cloud, closing the loop from model invocation to production operations.

Four Practical Recommendations for Developers

1. Validate capabilities against your own task set. Public benchmarks are for reference only; your internal evaluation set is the real basis for selection.
2. Make trade-offs across the capability–speed–cost triangle. Buy speed for interactive scenarios, depth for offline ones.
3. Design your application structure around caching and long context. Stable prefix, dynamic suffix—save wherever you can.
4. Assume agent applications will fail. Checkpoints, idempotent tools, and human approval gates are all indispensable.

Conclusion

The capability leaps of the new generation of flagship models look, on the surface, like a series of impressive benchmark scores. Underneath, they represent a systematic engineering upgrade spanning RL algorithms, sandbox infrastructure, scheduling systems, and environment QA. For developers, understanding this system is far more valuable than memorizing any single score: it shapes how we design evaluations, how we organize prompts, and how we build reliable runtimes for agents. As model capabilities approach the ceiling of various tasks, what truly differentiates applications will be the depth of polish in these engineering details.