From Base Models to Post-Training: How Large-Scale RL Drives Code and Agent Capabilities

The Shift from Base-Model Competition to Post-Training Competition

Looking at recent flagship model releases, several shared characteristics stand out: parameter counts crossing into the trillion-scale range, context windows expanding to one million tokens, the simultaneous availability of “high-quality” and “high-speed” versions within the same model family, and a clear shift in development focus from pre-training to post-training.

This is no coincidence. The marginal returns from pre-training are diminishing—high-quality corpora are finite, and simply scaling up pre-training compute no longer produces meaningful differentiation. Post-training, especially large-scale reinforcement learning (RL), has become the new engine of capability growth. One notable industry signal: models from the same generation can be continuously upgraded through iterative post-training, with release cycles compressing from annual major versions to quarterly updates. At the same time, capabilities in long-horizon agent scenarios such as coding, 3D creation, and financial research continue to improve. Additionally, several teams now release full model weights shortly after deployment, a trend that supports on-premises deployment and downstream development.

For developers, understanding this technical trajectory offers more long-term value than memorizing a particular benchmark score.

Large-Scale RL Post-Training: Where the Broad Gains Come From

The Fundamental Difference Between RL and SFT

Supervised fine-tuning (SFT) is essentially distribution fitting over human demonstration data. Its capability ceiling is bounded by the quality of that data. RL, by contrast, lets the model explore with a result-oriented objective—as long as the reward signal is reliable (code compiles and passes tests, task terminal states can be verified, reasoning conclusions can be checked), the model has the opportunity to discover solutions that surpass human demonstrations. This explains why RL post-training yields the most dramatic improvements in code and agent domains: verification costs are low, reward signals are clear, and the exploration space is vast.

Three Core Challenges in Joint Training Across Hundreds of Domains

RL in a single domain is not difficult. The real challenge is large-scale joint training across hundreds of downstream domains:

• Inconsistent reward scales: Different domains have drastically different reward magnitudes and distributions, which can cause certain domains to dominate gradient updates and crowd out learning in others.
• Uneven difficulty distribution: Easy tasks saturate their reward signals quickly, while hard tasks offer sparse rewards, creating an imbalanced training signal.
• Interference between domains: Multi-objective joint optimization often exhibits a seesaw effect—improving one domain while regressing another.

Common engineering countermeasures include reward normalization and advantage standardization, curriculum scheduling based on task difficulty, dynamic adjustment of domain sampling weights, and regression gatekeeping via benchmark evaluation to enforce a no-regression floor.

Full-Stack Infrastructure Optimization: The Conversion Efficiency of RL Compute

Several teams have recently emphasized that they “optimized every layer of the technology stack to ensure that each unit of additional RL compute translates into stable, efficient returns.” This captures the essence of post-training competition—the marginal benefit of RL is not determined by how much compute you pour in, but by how efficiently your infrastructure converts that compute into effective gradient updates. Breaking this down layer by layer:

• RL framework layer: The training engine and inference engine are decoupled, with rollout generation, reward computation, and policy updates running as three parallel pipeline stages to minimize GPU idle time. Weight synchronization latency must be low; otherwise, a lagging exploration policy can invalidate large numbers of samples.
• Long-context inference layer: A million-token context places enormous pressure on KV cache memory. This requires paged cache management, prefix cache reuse, and pragmatic engineering trade-offs in approaches like sparse attention.
• Training efficiency layer: A combination of data, pipeline, and tensor parallelism, with communication and compute overlap and elastic fault tolerance. A long-running training job that fails halfway and must restart is simply unacceptable.
• Sandbox infrastructure layer: Agent training requires massive numbers of real execution environments. High-concurrency isolation, snapshot and rapid recovery, resource quotas, and network restrictions are all essential. If sandbox throughput becomes a bottleneck, it directly stalls training.
• Scheduling layer: Orchestration of heterogeneous compute, task prioritization and preemption, and efficient utilization of fragmented resources.
• Environment quality platform layer: Task deduplication, solvability verification, and reward hacking detection. RL’s worst enemy is reward hacking—the model learns to exploit loopholes in the reward function instead of actually solving problems. An environment quality checking platform is the mechanism that enforces this boundary.

Million-Token Context and Long-Horizon Agents: The New Competitive Arena

Million-token contexts change the granularity of engineering collaboration: an entire mid-sized codebase, weeks of project evolution records, and dozens of tool-call trajectories can all fit within the model’s working memory. But “fitting” is only a precondition—“using it well” is the hard part. Three practical concerns stand out: reliable retrieval of information embedded in the middle of a long context, state consistency over long trajectories (early constraints are easily forgotten after many steps), and the cost and latency that grow with context length.

The three pillars of long-horizon agent tasks are task decomposition and planning, tool invocation, and cross-step state maintenance. A typical execution loop looks like this:

# Long-horizon agent execution loop (illustrative)
history = []
while not task.finished and step < MAX_STEPS:
    obs = env.snapshot()                        # current state: terminal / codebase / browser
    action = policy.decide(goal, history, obs)  # decide: run command / edit file / call tool
    result = sandbox.run(action)                # execute in an isolated sandbox
    feedback = verifier.check(result)           # verifiable feedback: tests, assertions, terminal checks
    history.append((obs, action, result, feedback))
    if feedback.success and goal_satisfied(history):
        break

Notably, benchmarks based on real terminal environments are gaining in value. These evaluations test coding agents in resource-constrained real environments, with independent acceptance tests for each task—far more aligned with real engineering scenarios than single-turn question-answering benchmarks. Industry feedback indicates that the latest models have narrowed the gap to leading closed-source models to single-digit percentage points on such benchmarks, with substantial improvements over the previous generation in multi-step execution, tool invocation, and end-to-end problem solving.

Code Capabilities: From Fixing an Issue to Delivering a Project from Scratch

The latest models demonstrate simultaneous strength at both ends of the coding spectrum:

• Small-task end: Locating and fixing a bug in a real codebase. This tests retrieval and localization, cross-file understanding, the principle of minimal change, and regression verification awareness. The mature workflow: reproduce → locate → fix → add tests → verify regression.
• Large-task end: Building a complete project from scratch. This tests requirement decomposition, technology selection, scaffolding, dependency management, module division, and end-to-end integration. The model’s output is no longer a code snippet but a complete project with directory structure, build configuration, and test suites.
• Experience end: Generating more polished visual styles for web pages and games, with “tool routing awareness”—knowing when to invoke specialized capabilities like image generation, design systems, or build tools rather than writing everything by hand.

The takeaway for developers: treat the model as a capable engineer and tireless iterator, driven by clear task decomposition and acceptance criteria, rather than using it in a single-turn question-and-answer manner.

How to Scientifically Evaluate a New Model

The reference value of public leaderboards is declining: scores at the top are saturating, evaluation data risks contamination from the training set, and they often misalign with real business scenarios. A scientific evaluation should be layered:

1. Look at benchmark form, not just scores: Real-environment terminal tasks, multi-step engineering benchmarks, and domain-knowledge tasks each examine a different dimension. Only a combination provides a complete picture.
2. A self-built evaluation set is the real decision basis. A minimal viable approach:

# Self-built evaluation set skeleton (illustrative)
for case in cases:                     # sample from real tickets / requirements
    ctx = build_context(case)          # inject real codebase and business constraints
    traj = agent.run(ctx, case.task)   # record multi-step trajectory and tool calls
    ok = run_assertions(case, traj)    # automated assertions + manual spot-checks
    metrics.record(case.id, ok, traj.steps, traj.cost)

1. Monitor process metrics: execution steps, tool-call success rate, rework rate, per-task cost—not just the final pass rate.
2. Run double-blind A/B testing: leaderboard scores determine the shortlist; business A/B tests on real traffic determine whether to go live.

Dual-Version Strategy: Decoupling Intelligence from Latency

Offering a version roughly 2–3 times faster at the same intelligence level typically combines general engineering techniques such as speculative decoding, quantization, KV cache reuse, batch processing, and scheduling optimizations. The strategic significance is decoupling “intelligence” from “latency”: the fast version suits interactive scenarios like IDE autocomplete, high-frequency multi-turn agent loops, and real-time dialogue—where reduced per-step wait time is amplified by the number of steps. The high-quality version suits complex reasoning and deep planning. On the engineering side, model routing should be implemented at the agent gateway layer, dynamically selecting versions based on task complexity, interaction density, and budget constraints—rather than hard-coding the choice in business logic.

Practical Recommendations for Developers

• Progressive rollout: shadow-traffic comparison, gradual traffic scaling, and a retained rollback path—avoid cut-over switching.
• Context management: long context is not infinite context. Pair it with retrieval-based injection, state summarization and compression, and external memory—conversation history and intermediate artifacts can be stored in object storage, with semantic retrieval handled by a vector search service.
• Observability: pipe every agent run’s trajectory, tool calls, and token consumption into logging and monitoring systems. Dead loops and unexpected cost spikes must be detectable.
• Environment as code: use container platforms to standardize the agent execution environment, ensuring consistency across evaluation, training, and production.

These capabilities map to common categories available on mainstream cloud platforms (GPU compute, model serving, container platforms, object storage, logging and monitoring). For example, using the model serving and container platform offerings from Baidu AI Cloud, teams can complete the loop from evaluation environment setup to production deployment without the heavy cost of building infrastructure from scratch.

Conclusion

The center of gravity in large model competition is shifting from “bigger base models” to “higher RL conversion efficiency”: whoever can steadily convert each unit of compute into verifiable capability gains will sustain a widening lead in subsequent iterations. For developers, understanding the technical trajectory of large-scale RL post-training, long context, and long-horizon agents—and building a rigorous self-evaluation system while managing context, cost, and observability in an engineering discipline—is what truly turns model capabilities into production value.