Post-Training RL and Long-Horizon Agent Engineering: What Actually Drives Capability Gains in Trillion-Parameter Models

    This article breaks down the engineering questions behind the current generation of flagship model releases: where marginal returns on post-training compute come from, what infrastructure large-scale joint RL training requires, how long-horizon agent capability should be evaluated, and how enterprise teams should make selection and deployment decisions. It covers the shift from pretraining scaling to alignment and reinforcement learning, the compute structure of a large RL loop (sampling and inference engines, sandboxes and execution environments, scheduling, environment quality checks), the real engineering implications of million-token context windows, why code, 3D creation, and financial research capabilities advanced in the same period, how high-speed inference tiers built on speculative decoding and aggressive batching should be routed, and a practical four-step evaluation framework for technical teams.

    Model capability is going through a clear shift in where the gains come from. Pretraining still sets the boundary of knowledge and the quality of representations, but whether a model can actually finish a complex task depends more and more on the depth of post-training. Recent industry movement reflects this: trillion-parameter base models, million-token context, agent capabilities aimed at long-horizon work, and high-throughput inference tiers on the same base are becoming a standard configuration for flagship releases.

    This article is not a benchmark of any specific product. Instead, it abstracts the technical problems behind these releases and discusses four questions from an engineering perspective: where does the marginal return on post-training compute come from, what infrastructure is needed to support large-scale joint training, how should long-horizon agent capability be evaluated correctly, and how should enterprise teams make selection and deployment decisions accordingly.

    1. The Center of Competition Has Moved: Capability Gains Increasingly Come from Post-Training

    Over the past two years, the main driver of capability improvement was the parallel expansion of parameter count, data quality, and training compute. Once base models reached the hundreds of billions to trillions of parameters, however, pushing scale along a single dimension became dramatically more expensive while returns started to show clear diminishing marginal gains. The emerging consensus in the industry is to direct more incremental compute into the alignment and reinforcement learning stages that come after pretraining.

    The reasoning is straightforward. Pretraining is essentially fitting the conditional distribution of the next token. It makes a model “know a lot,” but it does not inherently guarantee that the model will “get a job done.” Real development and research tasks tend to look like this: read some code, run a test, fix based on the error, run it again, and repeat dozens of times. That kind of behavior needs explicit reward signals to shape it, and the reward should not simply favor answers that look correct; it should favor processes that actually advance the task to a verifiable completion state.

    This leads to a practical rule for developers. When evaluating a model, look beyond scores on knowledge benchmarks and examine its behavior on long-horizon tasks: when a task requires dozens or even hundreds of tool calls to complete, how quickly does the success rate decay as the number of steps grows? That curve predicts real-world engineering usability far better than any single point score.

    2. Marginal Returns on Post-Training Compute Depend on Every Layer of the Stack

    “A large increase in post-training compute” sounds simple. In practice it is a systems engineering problem, because the compute profile of reinforcement learning is structurally different from pretraining. Pretraining is a large-scale, unidirectional, predictable compute stream. Post-training is a closed loop of generate, execute, score, update, containing large amounts of unpredictable inference requests, real environment interactions, and result verification.

    A typical large-scale RL training loop looks roughly like this:

    1. for step in range(total_steps):
    2. prompts = sampler.sample(batch_size)
    3. trajectories = actor.generate(prompts, num_samples_per_prompt=8)
    4. rewards = verifier.batch_verify(trajectories)
    5. advantages = grpo_advantages(rewards)
    6. loss = policy_objective(trajectories, advantages, ref_policy=ref)
    7. optimizer.step()

    Every line in this pseudocode sits on top of an independent infrastructure challenge.

    Sampling and the inference engine. The RL stage needs massive rollouts, and inference throughput directly determines the effective output per unit of compute. In engineering terms this usually relies on general techniques such as continuous batching, paged KV cache management, and prefix cache reuse, which raise GPU utilization in the generation phase from single-digit percentages to a level that can actually scale.

    Sandboxes and execution environments. If the reward comes from real execution (compiling, running tests, executing scripts, rendering assets), there must be an isolated execution layer that can be replicated quickly. The metrics that matter here are cold start latency and concurrency density. Compressing cold start from tens of seconds to the order of a second can often shorten total training time by an order of magnitude.

    The scheduling system. Training jobs and inference jobs want different things from resources: the former needs stable long-term occupancy, the latter needs elasticity and tolerance for bursts. Mixed scheduling, priority preemption, fast failure recovery, and resume from checkpoint are the baseline capabilities that keep expensive compute from idling.

    Environment quality checks and a shared platform. This is the most overlooked and most critical link. The ceiling on how trustworthy a reward signal can be equals the quality of the feedback the environment provides. If sandbox images drift, dependency versions are inconsistent, or the test scripts themselves are defective, the model does not learn capability; it learns to fit noise. A shared platform is therefore needed to maintain task environments consistently, verify that tasks are actually solvable, track anomalous shifts in the reward distribution, and manage contamination protection for evaluation sets.

    This explains a frequently misread phenomenon: the same compute budget produces wildly different results in different teams. The difference is usually not in the model architecture but in the product of three factors: sampling efficiency, environment stability, and reward quality.

    3. What Long Context Means in Engineering: From “It Fits” to “It Works Well”

    Million-token context is now a common configuration in flagship models, but a common misconception needs correcting: context window length is not the same as long-task capability. Three things actually determine whether long tasks succeed.

    First, the quality of the long-range attention implementation. Under very long contexts, the compute and memory access cost of attention dominates overall latency, so engineering practice commonly combines strategies such as sparse attention, a mix of sliding-window and global tokens, and chunked prefill. These optimizations improve throughput while potentially weakening recall of distant information, which is why long-distance dependency recall needs dedicated evaluation.

    Second, caching and scheduling strategy. In long conversations and long documents, prefix cache hit rate directly determines cost per request. For an agent that repeatedly reads the same large codebase, the cache hit rate often affects the total bill more than the choice of model does.

    Third, context engineering, meaning the techniques outside the model itself: layered retrieval and on-demand reads, writing intermediate results to storage and feeding back a summary, and isolating subtasks to keep the main context from growing. This layer can often deliver meaningful gains without changing the model at all, which makes it worth investing in first.

    4. Code, 3D Creation, and Financial Research: Why Three Capability Areas Improved Together

    Three capability directions recur in recent industry observation, and they happen to cover different kinds of capability construction.

    The core of code capability is verifiability. Compilers, static analysis, and unit tests provide judgments that are close to objective, which makes code one of the highest value-per-compute domains for post-training RL. For evaluation, agent benchmarks in a terminal environment are worth attention: inside a resource-constrained container, the model must install dependencies, read and write files, run tests, and fix problems, with the verdict produced by preinstalled task-specific test scripts rather than human grading. This kind of evaluation is much closer to real working conditions than “write a function.”

    3D creation sits at the intersection of multimodal generation and procedural modeling. The difficulty lies in geometric consistency and editability: a generated result has to be readable, modifiable, and reusable by downstream tools, not just a good-looking image. Evaluation should include structural validity checks (mesh closure, normal orientation, topology sanity) as well as a human preference evaluation.

    Financial research layers long-document understanding, numerical reasoning, and traceability. The evaluation focus here is not whether the answer is correct, but whether citations are verifiable, whether definitions and accounting conventions stay consistent throughout, and whether the output distinguishes facts from inferences. A recommended approach is to trace each key conclusion in the answer back to a specific passage in the source text and compute a traceability rate.

    It is worth noting that these three capability areas are exactly the task families where the “long-horizon” characteristic is most pronounced. This confirms the earlier judgment: the incremental value of post-training compute shows up mainly as the ability to push multi-step tasks to a completed state.

    5. “Same Intelligence, Higher Speed”: Why Inference Tiers Are Becoming a Delivery Option

    Another engineering practice worth noting is a single base model offered at two inference tiers, described officially as the same intelligence with output speed several times that of the standard tier. The technical paths behind this kind of offering are general and mature, and mainly include:

    • Speculative decoding: a lightweight draft model or shallow autoregressive head proposes candidates, the main model verifies them in parallel, and a high acceptance rate significantly reduces the number of serial decoding steps.
    • Continuous batching and more aggressive scheduling policies: higher throughput and better tail latency stability for online serving.
    • Quantization and sparsification: lower memory access cost with controllable accuracy loss, which typically has limited impact on output quality for generative tasks.
    • Output length and inference budget scheduling: tightening generation length for structured tasks to avoid wasted tokens.

    The practical implication for users: scenarios sensitive to time-to-first-token, such as interactive code completion and real-time retrieval question answering, are a good fit for the high-speed tier, while complex refactoring and long-document reasoning are better served by the standard tier in exchange for greater inference depth. A practical approach is to build a tier routing policy that selects automatically based on task complexity rather than using one tier for everything.

    1. routing:
    2. - match: {task: code_completion, latency_slo_ms: 800}
    3. tier: highspeed
    4. - match: {task: repo_refactor, complexity: high}
    5. tier: standard
    6. - match: {fallback: true}
    7. tier: standard

    6. Open Weights and Deployment Shapes: What They Mean in Practice for Enterprises

    Releasing full model weights on a published schedule matters to engineering teams for more than “free models are now usable.” It means three concrete things: the model can be deployed privately to satisfy data residency compliance requirements; it can be quantized and distilled on in-house hardware to bring cost into a range a specific use case can bear; and it can be fine-tuned on domain data to cover long-tail knowledge that pretraining data misses.

    On deployment architecture, enterprises typically use container platforms to host inference services, object storage to archive models and evaluation data, message queues to smooth request peaks, and logging services with monitoring and alerting to make the call chain observable. General cloud capabilities such as containers, storage, and monitoring and alerting offered by Baidu AI Cloud are one possible foundation for workloads of this kind. The point to emphasize is that the choice of deployment shape should be driven by compliance boundaries and the cost model, not by a preference for simply “running it on our own machines.”

    7. Selection and Evaluation Advice for Technical Teams

    The final part is an evaluation framework that can be executed directly. First, build your own task set. A size of 100 to 300 items is enough. The source must be the team’s real historical tickets, covering three categories: high-frequency, long-horizon, and error-prone samples, and each item should retain an automatically verifiable acceptance script. Second, run a blind comparison, standardizing sandbox images and dependency versions so that environmental factors do not contaminate the conclusions. Third, record four metrics at the same time: task success rate, average number of steps, token cost per task, and end-to-end latency. Fourth, use tier routing to control cost, and run quality regression on the high-speed tier, because whether the acceleration techniques introduce a perceptible quality drop has to be measured rather than assumed.

    A pragmatic conclusion: in a market where capability gaps have narrowed to single-digit percentage points, what determines whether a project succeeds is often not which model you pick, but the quality of the task set, the stability of the sandbox, and the reproducibility of the evaluation process. Get those three things solid, and the returns from model iteration will actually land on business metrics.