Post-Training Trillion-Parameter Models: Engineering Practices and Evaluation for Large-Scale RL

1. From “Stacking Parameters” to “Stacking Post-Training”: Three Signals of a Paradigm Shift

The evolution logic of flagship large models has been undergoing a far-reaching change in recent years: the marginal returns of parameter competition during pre-training are diminishing, while reinforcement learning (RL) compute investment during post-training has become the primary source of capability leaps. This shift can be confirmed through three signals.

Signal 1: Diminishing marginal returns from pre-training. High-quality corpora are increasingly scarce, the compute cost of further expanding parameters and data scales rises exponentially, and the capability gains from simply “stacking pre-training” have noticeably slowed.

Signal 2: The share of post-training compute has risen sharply. New-generation flagship models are generally positioned as “large-scale post-training versions of base models,” with post-training investment significantly increased over the previous generation. Reinforcement learning with verifiable rewards (RLVR) in particular delivers outstanding returns in domains where results can be automatically validated, such as code, mathematics, and agents. To support large-scale joint training across hundreds of downstream domains, leading teams are no longer just “adding more GPUs” but systematically rebuilding training infrastructure.

Signal 3: A shift in evaluation focus. Single-turn knowledge Q&A benchmarks are gradually saturating. New-generation evaluations focus on “long-horizon” capabilities: multi-step execution, tool calling, and end-to-end problem solving. Trillion-level parameters and million-level context are becoming baseline configurations for flagship models, but what truly differentiates user experience is post-training quality rather than parameter scale itself.

2. Compute Is Not Equal to Returns: Four Core Challenges of Large-Scale RL

A common misconception is that “once post-training compute goes up, capabilities will naturally follow.” Practice yields the opposite conclusion: if the technical stack does not evolve in tandem, more than half of the added compute will be swallowed by inefficient rollout, environment defects, and training instability. The challenges concentrate in four areas:

1. Reward signal quality. Rewards for agent tasks are naturally sparse and delayed, and carry the risk of “gaming.” Defects in the environment can be exploited by the policy, producing inflated scores (reward hacking).
2. Long-horizon credit assignment. A single task may involve dozens of tool interactions, with success or failure revealed only at the final step, making it very difficult to assess the value of intermediate actions.
3. Throughput bottlenecks. Agent-style rollouts often contain contexts of tens of thousands of tokens and dozens of rounds of tool calls. Inference workloads and training workloads have heterogeneous characteristics and compete for resources within the same cluster, and load imbalance directly causes GPU utilization to plummet.
4. Training stability. Large-scale concurrent environments introduce non-determinism; issues such as KL constraint failure and policy entropy collapse are amplified under large-batch updates, and training can easily “fly off the rails.”

Therefore, scaling post-training is fundamentally a systems engineering effort: the goal is not to “use more compute” but to ensure that every additional unit of compute translates into stable, efficient returns.

3. Supporting Post-Training at Scale: Six Key Layers of the Technical Stack

To run large-scale RL successfully, the following six layers are indispensable—any weak layer will drag down overall efficiency.

1. RL training framework. An asynchronous architecture that decouples the inference engine from the training engine: policy updates and sample generation (rollout) overlap in execution, combined with streaming rollout and experience replay mechanisms to significantly improve cluster utilization.
2. Long-context inference. Million-token contexts dramatically increase KV cache memory pressure. Prefix caching, paged attention management, and chunked prefill are needed to support long-horizon tasks while keeping rollout throughput acceptable.
3. Training efficiency. A combination of data parallelism, tensor parallelism, pipeline parallelism, and sequence parallelism, combined with operator fusion, mixed precision, and communication-compute overlap; long-sequence training especially depends on a careful trade-off between sequence parallelism and activation recomputation.
4. Sandbox infrastructure. Agents must execute code and call tools in securely isolated environments: container snapshots ensure reproducibility, while network isolation and CPU/memory/network quotas ensure security and fairness. This is the foundation of Agent training environments.
5. Scheduling system. Heterogeneous co-location of training and inference nodes, elastic scaling by training stage, fault detection, and checkpoint resumption determine whether thousand-GPU, long-duration tasks can run continuously and stably.
6. Environment quality assurance platform. A unified platform for task validity verification, environment determinism checks, and data leakage and contamination detection. Environment quality directly determines reward quality—this is the last mile of “converting compute into returns” and the most easily overlooked layer.

What these six layers have in common: none of them changes the algorithm itself, yet they determine how fast the algorithm can run, how stably it can run, and how much return it can generate.

4. Long-Horizon Agent Capability: The Core Observation Dimension for Evaluating New Models

For new models, the most noteworthy capability dimension is long-horizon agent capability—the ability to “reliably complete a task through multi-step execution and tool calling in a real environment.” It can be observed across five sub-dimensions:

• Multi-step planning and execution: whether tasks can be reasonably decomposed and advanced step by step, rather than one-shot “generative guessing”;
• Tool calling accuracy: whether parameter assembly, invocation timing, and exception handling are reliable;
• Context utilization: the ability to integrate information across documents and files within million-level contexts;
• Error recovery and self-correction: whether the agent can diagnose causes and adjust its approach after a failed execution;
• End-to-end completion rate: whether the final deliverable passes strict validation rather than merely “looking correct.”

Correspondingly, new-generation benchmarks are also evolving: terminal-style benchmarks let agents complete terminal tasks in real, resource-constrained environments, with each task equipped with strict task-specific tests; long-horizon engineering benchmarks cover multiple engineering domains and focus on multi-step execution, tool calling, and end-to-end problem solving.

How should scores be interpreted? Four lessons are worth referencing:

1. Absolute scores are far from saturated. On the hardest terminal-style benchmarks, top models’ completion rates generally remain around 60%, and a huge gap remains between “can run a demo” and “can deliver reliably.”
2. The gap among leaders is narrowing. Top models often differ by only a few percentage points, and rank alone is of limited significance; pay attention to evaluation variance and stability across multiple runs rather than leadership in the decimal places.
3. Look to long-horizon benchmarks for generational gains. Public benchmarks saturate easily and are prone to training data contamination; internal long-horizon engineering benchmarks better reflect true generational gains. From recent public information, top models’ generational improvements on such benchmarks often reach the five-percentage-point level, which is more informative than fractions of a point on mature public leaderboards.
4. Also examine stability metrics. In long-horizon tasks, “occasionally succeeding” and “consistently succeeding” have completely different value. Where conditions permit, examine the pass rate of the same task across multiple runs (e.g., pass^k) rather than single-run pass rate (pass@1).

5. Practical Guide: Building a Long-Horizon Evaluation Set for Your Own Business

No matter how impressive vendor leaderboards are, the answer that truly matters for your organization is “how well does it perform on your own tasks.” Consider building your own long-horizon evaluation set according to the following principles:

• Domain coverage: tasks span multiple engineering domains (backend services, frontend interaction, data processing, ops scripts, test completion, etc.) to avoid a single question type;
• Forced multi-step: tasks are designed so that a single-turn answer cannot complete them, requiring a full loop of environment exploration, tool calling, and result validation;
• Strict validation: each task comes with automated task-specific tests, with test passing as the sole success criterion—no manual scoring;
• Deterministic environment: container snapshots, fixed random seeds, and network isolation ensure reproducibility;
• Contamination prevention: tasks are kept private and rotated periodically to avoid entering any model’s training corpus.

Below is an illustrative evaluation framework skeleton demonstrating the complete loop of “environment reconstruction—multi-step execution—strict validation—stability statistics”:

class AgentTask:
    """Abstract long-horizon engineering task: environment + tools + strict validation"""
    def __init__(self, name, env_setup, tool_registry, verifier, max_steps=50):
        self.name = name
        self.env_setup = env_setup      # Environment init: container snapshot, dependency list, random seed
        self.tools = tool_registry      # Allowed tools: Shell, files, HTTP, etc.
        self.verifier = verifier        # Task-specific tests: one automated check per task
        self.max_steps = max_steps      # Maximum allowed interaction steps
def run_eval(agent, tasks, runs_per_task=5):
    """Statistics across multiple runs for stability: only 'consistent success' has value in long-horizon tasks"""
    report = {}
    for task in tasks:
        passed, total_steps = 0, 0
        for _ in range(runs_per_task):
            env = task.env_setup()                # Rebuild a clean sandbox environment each time
            traj = agent(env, task.tools, task.max_steps)
            result = task.verifier(env, traj)
            passed += result.passed
            total_steps += result.steps
        report[task.name] = {
            "pass_rate": passed / runs_per_task,     # Stable pass rate
            "avg_steps": total_steps / runs_per_task # Average interaction steps (efficiency)
        }
    return report

Evaluation is not a one-time procurement action but a continuous asset: recording complete execution trajectories and performing root-cause analysis on failed samples (planning failure, tool calling failure, or validation failure) can both guide model selection and feed back into prompt and workflow design.

6. The Speed-Intelligence Trade-off: Understanding “High-Speed” Model Variants

Recent flagship models often offer both a standard version and a “high-speed” version: with intelligence levels roughly on par, output speed is increased by about 2–3x. This is typically achieved through inference-side optimizations—speculative decoding, operator and batch scheduling optimizations, service-oriented quantization strategies, and so on—rather than by cutting model capability. The basis for selection is straightforward:

• For interactive coding, multi-turn Agent dialogue, and similar scenarios, time-to-first-token and token output speed directly affect experience, and the high-speed version offers clear benefits;
• For offline batch processing, overnight batch jobs, and similar scenarios, throughput and per-task cost carry more weight, and the standard version or self-hosted deployment can be prioritized.

In terms of access form, the industry-standard practice is to simultaneously open multiple channels: chat interfaces, office collaboration endpoints, coding assistant endpoints, and APIs. Enterprises should focus on API compatibility, structured output stability, and function calling accuracy. In addition, many vendors progressively release full model weights after launch, allowing enterprises to weigh API access against self-hosting based on data compliance, scaling costs, and custom fine-tuning needs.

7. Enterprise Adoption: An Actionable Selection Checklist

1. Context requirements: Million-level context has practical value for whole-repository code understanding, long-document analysis, and long-conversation memory; but don’t just look at the length number—measure the “effective utilization” of cross-document reasoning and long-horizon information integration.
2. Agent reliability: Validate with your own long-horizon benchmarks, focusing on stable pass rate and average steps rather than the single best result.
3. Speed and cost: Calculate based on “per-task cost” rather than “per-thousand-token price,” and also examine caching strategy and concurrency capability.
4. Tools and integration: Function calling accuracy, structured output stability, and the integration and rework cost with existing R&D pipelines.
5. Infrastructure support: Landing an agent system requires general cloud service foundations such as container platforms, object storage, message queues, logging, and monitoring. Enterprises can leverage such general capabilities provided by cloud service providers like Baidu AI Cloud to quickly build elastically scalable evaluation and inference platforms, concentrating resources on task design.
6. Evolution tracking: Pay attention to methodological transparency in technical reports—whether the training pipeline, environment construction, and evaluation details are clearly explained often predicts a model’s engineering maturity better than marketing language.

8. Conclusion: Turning “Evaluating Models” into Systems Engineering

Returning to the initial question: how should a newly released flagship model be evaluated? After the analysis above, the answer can be distilled into three questions:

First, how does it turn compute into capability? Look at the depth of post-training engineering—whether the RL framework, long-context inference, sandbox, scheduling, and environment quality assurance form a closed loop. Second, is it stable on long-horizon tasks? Look at stable pass rates on terminal-style and long-horizon engineering benchmarks, and the magnitude of generational gains. Third, how does it perform in your scenarios? Use a self-built evaluation set to give your own answer.

Competition in large models has moved from “whose parameters are bigger” to “whose compute conversion efficiency is higher and whose environment and evaluation systems are more solid.” For developers, self-built evaluation should be treated as a first-class asset; for enterprises, selection should demand reproducible evidence rather than leaderboard screenshots. Understand these three points, and no matter how elaborate the wording of the next launch event, you can quickly grasp the signals that truly matter.