Reinforcement Learning Post-Training: Capability Leaps and the Narrowing Gap Between Open and Closed Models

I. The Engine Switch: From Pre-Training to Post-Training

For the past several years, the capability curve of large models has been driven mainly by scaling laws in pre-training: more data, more parameters, and more compute reliably produced better results. But this engine now faces dual constraints: high-quality public corpora are approaching exhaustion, and the marginal returns of pre-training continue to diminish. The industry’s response has been to shift the center of capability growth to the post-training stage—building on an existing base model, using supervised fine-tuning (SFT) and reinforcement learning (RL) to shape the model’s actual behavior and capabilities.

One clear signal is the changing compute structure: in leading model development, post-training—especially reinforcement learning—accounts for a steadily rising share of compute, evolving from an early auxiliary step into an investment on par with pre-training. The returns are direct: even with an unchanged base model, performance on high-value tasks such as reasoning, coding, and Agents can achieve generational leaps.

The principle is not complicated: pre-training determines the model’s “knowledge base,” while post-training determines “how the model invokes that knowledge.” Once knowledge accumulation reaches a certain scale, the bottleneck shifts from “whether it knows” to “whether it can use it consistently and correctly”—exactly the problem reinforcement learning is designed to solve.

II. Core Mechanisms of Large-Scale RL Post-Training

(1) Verifiable Rewards: Breaking Through the Ceiling of Imitation Learning

Supervised fine-tuning is essentially imitation learning: the model fits human demonstration data. The ceiling of imitation is the ceiling of the demonstration data itself. On tasks with clear correctness criteria, such as coding and math, human demonstrations often contain a large number of suboptimal solutions, so the model learns to “look human” rather than to “achieve the optimum.”

Reinforcement learning with verifiable rewards (RLVR) takes a different approach: instead of telling the model what the correct answer looks like, it provides the problem environment and a set of automatically checkable reward rules—whether unit tests pass, whether the answer is correct, whether the task goal is achieved. Through extensive exploration, the model discovers solutions that surpass human demonstrations, and the capability ceiling switches from “demonstration quality” to “the upper bound of the reward rules.” This is the key mechanism behind the rapid recent progress of reasoning and coding capabilities toward the strongest levels.

(2) Multi-Domain Joint Training: Letting Capabilities Reinforce Each Other

Another key practice is large-scale joint training: rather than optimizing a single capability in isolation, reinforcement learning is conducted jointly across hundreds of downstream domains (coding, math, multi-step reasoning, tool calling, knowledge QA, long-form writing, and more). Its value is threefold:

1. Mitigating capability imbalance. Over-optimizing a single objective can easily damage other capabilities; a mixed multi-domain objective acts as a natural constraint, pushing the model toward generalizable capabilities rather than “test-taking tricks” aimed at benchmarks;
2. Promoting positive capability transfer. The rigorous reasoning habits acquired in coding training can transfer to math and general reasoning, and the task-planning ability formed in tool-calling training can transfer to complex task decomposition; each domain serves as a whetstone for the others;
3. Approximating the real distribution. Real business workloads are highly mixed, and a jointly trained model performs better and more stably on mixed tasks.

(3) The Abstract Form of the Training Loop

A typical policy optimization iteration can be abstracted as follows (illustrative pseudocode, not a real interface):

for iteration in range(num_iterations):
    # 1. Task sampling: draw a batch from a mixed task pool of hundreds of domains
    batch = task_mixer.sample(batch_size)
    # 2. Trajectory rollout: the current policy executes tasks in verifiable environments
    trajectories = rollout(policy_model, batch, max_steps)
    # 3. Reward checking: automatic scoring by rules (tests pass / answer correct / goal achieved)
    rewards = [verifier.check(t) for t in trajectories]
    # 4. Policy update: optimize toward higher rewards with policy-gradient-style algorithms
    policy_model.update(trajectories, rewards)
    # 5. Capability regression: monitor general capabilities on a held-out eval set to prevent "fixing one thing and breaking another"
    assert eval_harness(policy_model) >= baseline

Step 5 is often overlooked but crucial: mature post-training pipelines treat general-capability regression tests as a hard constraint, and any single-point optimization is premised on not harming overall capability.

III. Agent Capabilities: The Track with the Most Significant Gains

If reasoning and coding were the first wave of beneficiaries of RL post-training, Agent capabilities are the second wave with the most significant gains. The essence of an Agent task is turning the model from “a single Q&A” into a “multi-step loop”: understand the goal, make a plan, call tools, observe feedback, revise the strategy, until the task is complete. A weakness in any link gets continuously amplified across the multi-step loop—which also explains why Agent capabilities long lagged behind single-turn Q&A.

Agent tasks are a natural fit for reinforcement learning:

• Verifiable rewards: whether a task is completed and whether the final state meets the standard can usually be checked automatically, for example “the file has been modified according to spec” or “all tests pass”;
• A huge exploration space: multi-step combinations of tool calls far exceed what human demonstrations can cover, which is precisely why the model needs to explore successful strategies on its own;
• Rich failure signals: mid-course tool misuse, ineffective loops, context overflow, and similar events can all be converted into dense process rewards or penalties.

Combined with million-token context windows, the model can maintain memory of the global goal across long-horizon tasks—reading through large codebases, maintaining intermediate state across multiple rounds of tool calls, and completing cross-paragraph reasoning over very long documents. Together, these two capabilities move “the model autonomously completing tasks end to end” from demo to production-ready.

Agent evaluation should likewise use “in-environment completion” rather than static Q&A:

def evaluate_agent(model, task_suite, max_steps=20):
    scores = []
    for task in task_suite:
        env = task.build_env()            # provides the tool set and a verifiable goal
        obs = env.reset()
        for _ in range(max_steps):
            action = model.plan(obs, env.tool_docs())
            obs, reward, done = env.step(action)
            if done:
                break
        scores.append(env.final_score())  # automatic scoring by goal achievement
    return sum(scores) / len(scores)

This evaluation approach is closer to real development scenarios and is the main battleground where leading models currently differentiate themselves.

IV. Open Weights vs. Closed Source: Why the Gap Keeps Narrowing

Judging by the composite intelligence indices published by third-party evaluation organizations in recent years, the gap between leading open-weight models and the strongest closed models is converging rapidly: from an early, clearly double-digit spread to single digits, with some dimensions even matched or surpassed. Three logics underlie the trend:

1. Methodologies becoming public and engineered. Key practices such as policy optimization algorithms, verifiable reward construction, and task mixing ratios have become common knowledge in the research community, placing open-weight teams and closed teams at nearly the same starting line;
2. The late-mover iteration advantage. Open-weight models can target known weaknesses for focused improvement and have more flexible release cadences;
3. Neutral evaluation. Independent evaluation makes capabilities comparable across the board, avoids self-referential claims, and lets real strength be seen.

For the industry, the significance of this trend is no less than the progress of the models themselves: when open-weight models reach or approach the strongest closed-source levels on coding, Agent, and knowledge benchmarks, enterprises gain a viable option to “own the weights, deploy privately, and customize deeply”—data stays in-domain, costs are controllable, and behavior is auditable, which are exactly the core requirements for enterprise deployment.

V. Trillion Parameters and Million-Token Context: Supporting Engineering Evolution

Capability leaps do not happen only at the algorithm layer; the supporting evolution of model architecture and inference engineering is equally critical:

• Mixture-of-Experts (MoE): trillion-level total parameters with sparse activation make “large capacity” and “affordability” compatible, with training and inference costs far below a dense model of equivalent parameter count, and provide a feasible compute basis for large-scale reinforcement learning;
• Long-context engineering: expanding from the hundred-thousand level to the million-token level requires a whole toolset—positional encoding extrapolation, sparse attention, KV cache optimization—and a balance between “long” and “accurate”; however long the window, if mid-section information utilization is insufficient, long context is merely nominal;
• Inference efficiency: general optimizations such as dynamic batching, cache reuse, and speculative generation directly determine real throughput and latency in long-context and Agent scenarios.

For enterprise deployment, these engineering factors often affect real experience more than leaderboard scores: with the same model weights, different inference service optimizations can produce efficiency differences of several times.

VI. Evaluation and Model Selection Practices for Developers

Facing a rapidly evolving model landscape, rather than chasing a single leaderboard, it is better to build your own evaluation framework:

1. Distinguish “composite index” from “task score.” Composite intelligence indices are suitable for judging model tiers, but specific business tasks (such as contract review, internal knowledge QA, log analysis) must be tested on your own task sets;
2. Replace static benchmarks with in-environment evaluation. Build sandboxes with real tools and data to examine task completion rate, average number of steps, and exception recovery ability;
3. Focus on stability and regression. Sample tests at different times and under different loads, and establish version regression baselines for key capabilities;
4. Evaluate total cost of ownership. Consider input/output unit prices, the cost slope of long context, and latency under concurrency, rather than unit price alone;
5. Preserve replaceability. Abstract away model differences through a unified service-calling layer, making “open-weight/closed-source, cloud/private deployment” switchable options rather than architectural lock-in.

At the infrastructure level, enterprises can use general capabilities such as container services, object storage, logging, and monitoring/alerting provided by platforms like Baidu AI Cloud to quickly build evaluation sandboxes and inference services, concentrating effort on evaluation set construction and business validation.

VII. Conclusion

The focus of large-model competition has shifted from “whose base is bigger” to “whose post-training is smarter.” Large-scale reinforcement learning uses verifiable rewards as a lever to pry generational leaps in high-value capabilities such as reasoning, coding, and Agents; and the diffusion of methodologies is driving the gap between open-weight and closed models to converge toward near parity. For developers, the real question is no longer “whether to use large models” but “how to build your own evaluation system to continuously select the most suitable model amid a rapidly evolving capability supply.” Make evaluation solid and architecture open, and the dividends of this capability leap can truly be converted into productivity.