Post-Training Scaling: How to Evaluate LLM Iterations Built Entirely on Post-Training

    This article examines a new frontier coding model whose developers claim that every capability gain over its predecessor came from post-training alone, while the base model, architecture, total parameter count, and activated parameters stayed fixed. It breaks down the three technical pillars behind the approach—long-horizon environment scaling, automated verifiers, and RL infrastructure—and proposes a four-layer evaluation framework covering attribution, task-level verification, cost-effectiveness, and safety and compliance. It also outlines engineering practices for adopting these capabilities, along with limitations and risks that practitioners should watch for.

    1. The Core Takeaway: Capability Gains Can Come Entirely From Post-Training

    A Chinese AI team recently released a frontier coding model positioned around “built for coding, prepared for cyber defense.” The team publicly claims that every capability improvement over the previous generation came from the post-training stage alone. The base model, architecture, total parameter count, and activated parameters per token were all left untouched. The only variable was roughly one month of post-training that scaled the long-horizon environment by tens of times, paired with a larger compute investment in reinforcement learning (RL).

    This is not simply a matter of “skipping pre-training.” It is a carefully designed controlled-variable experiment:

    • Fixed variables: base weights, network structure, parameter count and activated parameters, context capacity.
    • Sole variable: the scale of the post-training environment, the quality of automated verifiers, and RL infrastructure efficiency.

    Because the variable set is so tightly controlled, “same base, higher capability” can be treated as direct evidence of what post-training contributes, giving us a clean sample for studying post-training scaling.

    • Across roughly 20 capability evaluations, 14 held steady or improved versus the previous generation.
    • In the low-confidence region, gains concentrated on interactive long-horizon tasks: the terminal-interaction benchmark rose from 4.6 to 28.3 (about 6x), and the real-world software engineering benchmark improved by 20.7 points.
    • On the cybersecurity benchmark, the model ranked first at 84.5%; exploit-oriented benchmarks more than doubled versus the previous generation.
    • At the same time, 6 metrics regressed: code editing 2.0 dropped 14.9 points, task reproduction 0.3 dropped 12.0 points, and reasoning 0.4 dropped 4.0 points.

    The uncomfortable inference here is worth facing directly: post-training scaling is not a uniform “capability amplifier.” It delivers strong returns on interactive tasks with ample environment coverage and clear verification signals, but it can cause capability drift on older tasks that the new environments do not cover well. To evaluate this kind of iteration, you cannot rely on leaderboard scores alone. You have to return to two basic questions: whether the base capability has been sufficiently preserved, and whether the newly added capability is verifiable.

    This article builds a four-layer evaluation framework around those questions:

    1. Attribution layer: confirm whether the variables were truly isolated, and whether capability gains can be credited to post-training.
    2. Task-evidence layer: distinguish benchmark scores from real-world task performance.
    3. Cost-effectiveness layer: assess compute and engineering cost, and judge scalability.
    4. Safety and compliance layer: evaluate the risks and responsibility boundaries that come with stronger capability.

    The article closes with an actionable infrastructure checklist.

    2. Release Context: From a Parameter Race to a Post-Training Race

    2.1 Competitive Pressure During an Intense Iteration Cycle

    In the second half of 2026, leading model developers in China entered a period of dense iteration:

    • One vendor released a flagship model with 2.4 trillion parameters.
    • Another team released a model with 2.8 trillion parameters.
    • A general-purpose flagship model shipped a stable-version update.

    Against this backdrop, “don’t stack parameters, only scale post-training” is a differentiated choice. The signal it sends is that on interactive tasks, the marginal gain in model capability need not come from larger parameter counts, but may instead come from better training signals and richer execution environments.

    2.2 Why Open Weights Are Decoupled From Safety Evaluation

    The model was first released as an API and a subscription coding service. The team plans to complete safety evaluation and hardening roughly two weeks after launch, then open the full weights.

    This sequencing is not an ad hoc decision; it is tied directly to the direction of capability improvement. RL reinforces exactly the high-risk skills: terminal control, vulnerability discovery, and exploitation. Once the weights are public, potential users are no longer constrained by the service provider, and the gap between defensive capability and offensive capability widens in lockstep. “Evaluate first, release later” is therefore about decoupling model capability from safety responsibility: the capability ceiling is set by post-training, while the release cadence is set by risk assessment.

    2.3 Tool Ecosystem: When Prompt Protocols and Efficiency Move Together

    The model also integrates several mainstream AI coding tools. A common misreading is worth flagging here: better tool compatibility improves usability; it is not the same as model capability.

    The real engineering value lies in the efficiency dividend created when “the model and tool protocols co-evolve.” When a model’s tool calls are more stable and its long-context usage is fuller, external agent frameworks need to do less of the “backstop” work (retries, corrections, truncation compensation), and the token consumption of the entire call chain drops accordingly.

    The takeaway for developers: when evaluating a coding model, don’t just ask “how many points does it score on the leaderboard.” Also ask “how much rework can it eliminate in my toolchain.” The latter is the real return when the model interacts with an engineering system.

    3. The Capability Base: Understanding the “Controlled Variable” Premise

    To evaluate a claim that “the only remaining variable is post-training,” you first need to understand the base that was held fixed.

    The base belongs to a sparse Mixture of Experts (MoE) architecture:

    • Total parameters of about 753 billion, with only about 40 billion parameters activated per token.
    • A network depth of 78 layers.
    • The attention layer combines Multi-head Latent Attention (MLA) with sparse attention, and introduces an index-sharing mechanism.
    • A context window of up to millions of tokens, with the API supporting a single continuous output of up to about 128K tokens.
    • Existing engineering capabilities including chain-of-thought reasoning, streaming output, function calling, context caching, structured output, and tool protocols.
    Capability How long-horizon post-training acts on it Why the base must stay fixed
    Parameter capacity Sets the ceiling for knowledge and reasoning Changing parameter count introduces confounds that cannot be attributed
    Activated parameters Determine per-step compute and latency Affects the fairness of comparing training vs. inference cost
    Attention and index design Determine retrieval efficiency in long context Long-horizon tasks depend on precise localization within long history
    Context and output length Determine the state and action sequence a single task can carry Insufficient output length truncates long tasks mid-run
    Engineering interfaces (tool calling, etc.) Determine whether the model can reliably plug into agent frameworks Unstable interfaces amplify noise in post-training gains

    The “large total parameters, small activated parameters” characteristic of the sparse architecture gives the base ample knowledge capacity while keeping per-token compute cost low. This matters especially for post-training. When running RL on an MoE model, every sample requires repeated forward inference to generate trajectories, and the scale of activated parameters directly determines sampling throughput and memory pressure. In other words, the base architecture itself forms the “cost foundation” for post-training scaling.

    4. Why Choose Post-Training Scaling

    4.1 Data Supply: Text Is Depleting, Interaction Scales

    The growth rate of internet text data has clearly slowed, and simply expanding the corpus is hard-pressed to deliver sustained returns.

    Training data for interactive tasks has an entirely different nature: a single executable environment (an operating-system terminal, a code repository, a cybersecurity range) can host nearly unlimited task instances. Tasks can be generated automatically through:

    • Template instantiation
    • Programmatic generation
    • Adversarial expansion

    Each trajectory is itself a training sample with execution feedback. As a result, the scarcity shifts from “corpus” to “environment.”

    4.2 A Controlled Experiment: The Cognitive Value of a Single Variable

    If the base and post-training both change, no capability gain can be confidently attributed to either one. The controlled-variable design amounts to a natural experiment: with the base held fixed, environment scale and RL compute become the only independent variables, so the attribution chain for capability gains becomes clear.

    4.3 Marginal Returns: The Leverage Effect of Post-Training

    The input-output relationship of post-training is not linear. It exhibits a logarithmic pattern resembling the classic scaling law:

    1. Score(S) ≈ a · log(S / S0) + b

    Here, S is a combined measure of the effective environment scale and compute investment during the RL stage, and S0 is the corresponding magnitude from the previous generation’s post-training. The team claims that “post-training investment scaled by tens of times,” which lets us estimate the capability gain per order of magnitude:

    Benchmark Previous → New Gain Gain per order of magnitude (theoretical ceiling)
    Terminal interaction 4.6 → 28.3 +23.7 about 16.5 points
    Real-world software engineering +20.7 points over previous +20.7 about 14.4 points
    Cybersecurity reached 84.5% (no previous baseline) Unknown Not estimable
    Exploitation more than doubled over previous +100% or more Only a multiple given; cannot be back-derived

    Three points deserve emphasis:

    1. These values are only theoretical ceilings. In practice, the curve is often steeper in data-sparse regions and flatter in saturated ones.
    2. The cybersecurity and exploitation results lack a previous-generation baseline or raw figures, so they cannot be folded into a unified comparison.
    3. If the actual gain falls well below the estimate, it means the new environment contains a large number of “ineffective samples”—tasks that cannot distinguish model behavior. In that case, improving verifier quality should take priority over piling on more volume.

    More interesting still is the architectural leverage: on an MoE base, activated parameters account for only about 5% of total parameters, which means the same sampling budget can generate more trajectories and cover more tasks. Post-training scaling capability is therefore amplified by both the base architecture and the RL infrastructure.

    5. The Three Pillars: A Technical Breakdown of Post-Training Scaling

    5.1 Long-Horizon Environment Scaling: Turning the Real World Into Trainable Tasks

    The core of long-horizon environment scaling is expanding the task environment from “a handful of curated samples” into a “scaled, executable, verifiable” training ground. It involves at least three layers of design:

    1. Environment diversity. Covering heterogeneous scenarios such as operating-system command execution, defect fixing and feature implementation in real repositories, and vulnerability discovery and penetration validation, so the model repeatedly practices under different toolchains and constraints.
    2. Long-horizon nature. A single task may require tens or even hundreds of interaction steps. Context caching and chain-of-thought let the model maintain long-horizon memory, but they also bring error accumulation and context bloat. Long-horizon RL trains precisely the ability to “maintain goal consistency within long sequences.”
    3. Verifiability. Each environment must carry executable verification conditions, such as:
      • Unit-test pass rate
      • System-state assertions
      • Whether a vulnerability was successfully exploited
      • Whether a defensive measure took effect

    An environment without verification conditions cannot produce an effective reward signal, and scaling it up only introduces more noise.

    5.2 Automated Verifiers: Scaled Assurance of Reward Quality

    The reliability of the verifier directly sets the ceiling on RL effectiveness. A model will precisely optimize whatever the reward measures, which may not be what the designer actually wants.

    • Too weak a reward signal: the model learns to “pass by chance” rather than to solve the problem correctly.
    • A manipulable reward signal: for example, directly editing test files, bypassing validation logic, or exploiting test-data leakage. The model quickly finds “shortcut” strategies.

    For this reason, a reliable verification system typically includes four complementary signal types:

    • Functional correctness: unit tests, integration tests, hidden test sets.
    • Behavioral constraints: process exit codes, the scope of filesystem changes, a network-access allowlist.
    • State consistency: the system state after a task must satisfy its assertions.
    • Anti-manipulation checks: auditing the model’s own modifications to detect whether it bypassed the verification mechanism.

    In cybersecurity scenarios, the verifier also acts as a double-edged filter: whether an exploit was reproduced and whether a defensive hardening is effective are both capability measures and risk-assessment inputs. Once automated attack verification scales up, it must be paired with equivalent defensive and compliance review.

    5.3 RL Infrastructure: Determining Output per Unit of Compute

    With the same compute, different infrastructure efficiency can yield several times the difference in obtainable training signal. Long-horizon RL faces three engineering challenges:

    • Sampling bottleneck: long trajectories generate slowly and require real execution environments (sandboxes, containers, network isolation), with resource costs far higher than pure text generation.
    • Weight synchronization: after the policy model updates, weights must be synced to the inference cluster promptly, so sampling and training policies don’t drift apart.
    • Reward delay: the reward for a long task often arrives only after hundreds of interaction steps, making credit assignment difficult.

    To address these problems, industry commonly uses several approaches:

    • Asynchronous sampling: decouple inference and training into a pipeline, with the inference cluster continuously producing trajectories and the training side consuming them in batches.
    • Environment pooling: maintain a pool of environment instances that can be created, destroyed, and reset quickly, shortening task-preparation time.
    • Group-normalized advantage: sample multiple trajectories for the same task and compare relative quality within the group, reducing sensitivity to absolute reward values.
    • Curriculum learning: dynamically adjust task difficulty based on historical pass rates, prioritizing tasks the model can “just barely” solve.
    • Failed-trajectory feedback: distill key error nodes from failed trajectories into new training samples, closing the loop.
    1. # Long-horizon RL training loop (pseudocode)
    2. for step in range(total_steps):
    3. # 1. Sample from the task pool by difficulty distribution, adjusting the curriculum dynamically
    4. tasks = env_pool.sample(batch_size, difficulty_policy)
    5. # 2. Asynchronous concurrent execution: inference + environment interaction
    6. rollouts = engine.async_rollout(model, tasks, max_steps=128)
    7. # 3. Multi-source verifier scoring, producing structured rewards
    8. rewards = verifier.score(rollouts) # unit tests + state assertions + anti-manipulation checks
    9. # 4. Filter out unverifiable samples to avoid noisy rewards
    10. rollouts, rewards = filter_invalid(rollouts, rewards)
    11. # 5. Compute group-normalized advantages, update the policy
    12. advantages = group_normalize(rewards, group_size=8)
    13. model.update(rollouts, advantages)
    14. # 6. Sync weights to the inference cluster to keep sampling and training consistent
    15. engine.sync_weights(model)

    Each step in this flow maps to an engineering decision: asynchronous execution determines throughput, environment pooling determines task-preparation overhead, group normalization determines training stability, and weight synchronization determines policy consistency. Getting all of these right is the real engineering meaning of “post-training scaling.”

    6. How to Evaluate: A Four-Layer Framework

    6.1 Layer One: Attribution and Control

    The first question is “where does the capability actually come from.” I recommend checking from three angles:

    • Are the variables isolated? Did the team explicitly state that the base, architecture, and parameter count are unchanged, and are the post-training variables quantifiable (e.g., environment scale in multiples, RL compute)?
    • Is the evaluation on the same footing? Were the previous and new generations evaluated on the same hardware and the same inference configuration (temperature, max output length, tool-call limit)?
    • Is the evaluation independent? Do the vendor’s self-tests and third-party re-tests agree, and is there any targeting optimization specific to the benchmarks?

    6.2 Layer Two: Benchmark Selection and Task-Level Verification

    Not every weight improvement matters equally. To judge a benchmark’s value, look at what kind of question it answers:

    Benchmark type Main question it answers Information value
    Traditional knowledge and reasoning Is the model’s static knowledge sufficient? The previous generation was usually near saturation, so it discriminates poorly
    Interactive terminal Can it complete multi-step operations in a real command-line environment? Directly reflects long-horizon execution ability
    Software engineering Can it fix defects in real repositories? Highly relevant to real-world engineering adoption
    Cybersecurity Can it perform vulnerability discovery, exploitation, and hardening? Reflects high-risk capability and requires paired risk assessment

    “Scores near the ceiling” often signal falling information value. For example, a reasoning benchmark where the previous generation already reached 74.4 and the new generation simply held steady provides almost no incremental information. It is therefore better to prioritize gains on “low-confidence” benchmarks: they sit closer to the real capability frontier and are more valuable for distinguishing models.

    Two traps are also worth avoiding:

    • Static-score dependence. It cannot reflect how a model performs “while interacting with tools.” The same model can produce very different results in “autonomous completion” versus “collaboration within an agent framework.”
    • Leaderboard contamination. Once an interactive benchmark is public, the training environment may unintentionally cover similar tasks, inflating scores. Task-level verification is a necessary complement here.

    Task-level verification is best done with bucketed analysis:

    Bucket Definition What to watch
    Pass All hidden tests pass and anti-manipulation review is cleared Whether success relies on environmental accident (e.g., test leakage)
    Partial pass Main tests pass but edge cases fail Whether defect localization is accurate
    Fail Not completed, or triggered a safety limit Whether the failure mode is a capability gap or over-refusal

    Notably, tasks that pass nearly all buckets on the first attempt often hide test-coverage blind spots. For such tasks, add adversarial samples rather than counting them directly toward capability statistics.

    6.3 Layer Three: Cost and Engineering Economics

    To judge whether a model is “worth using,” you need to look at capability gains alongside cost:

    1. Inference cost. The sparse architecture lowers per-token compute, but long-horizon tasks often require many rounds of calls, so total token consumption can grow several-fold.
    2. Toolchain overhead. Retries, error correction, and context compression in external agent frameworks all consume extra tokens.
    3. Operational cost. The extra resource overhead from long context, tool calling, and sandbox execution environments needs to be folded into the overall budget.
    4. Task success rate. “Tasks completed per unit cost” is a better final metric than a raw leaderboard score, because it is closer to business value.

    6.4 Layer Four: Safety, Regression, and Compliance

    Capability gains must be evaluated in step with risk:

    • High-risk skill audit. Terminal control and exploitation are inherently dual-use. Before use, confirm the output meets the safety requirements of the deployment scenario, and validate it in a controlled environment.
    • Over-refusal risk. Over-strong safety alignment can block legitimate development tasks, so an evaluation should include a “false-refusal rate on normal tasks” metric.
    • Capability regression. The 6 regressing metrics signal the need for regression testing on key business scenarios to confirm the new model does not degrade on existing tasks.
    • Timing of open weights. Opening weights means capability can be deployed and fine-tuned arbitrarily, so the release cadence and scope should be decided together with the safety-evaluation conclusions.

    7. Engineering Practice: How to Adopt Post-Training Capability

    For enterprise developers, a model’s capability gains ultimately come down to “how to use it reliably and stably.” Long-horizon post-training and long-horizon agent applications share the same set of infrastructure needs, so start with the following:

    1. Environment isolation. Provide containerized sandboxes for model execution, restrict network egress and filesystem permissions, and prevent model-generated code from affecting the host environment.
    2. Tool-call protocols. Describe available tools with standardized function calling and tool protocols, ensuring model output can be parsed and replayed reliably.
    3. Context and cache management. Long-horizon tasks depend on effective context compression and caching strategies to reduce redundant inference and token waste.
    4. Observability. Build logs and tracing for every tool call, sandbox execution, and model output, making it easier to locate failure causes and account for cost.
    5. Evaluation pipeline. Fold task-level evaluation into a CI process so regression tests run automatically on every model version update.

    8. Limitations and Risks

    1. The boundary of attribution. The conclusion about post-training scaling depends on the premise that “the base is completely unchanged.” Once the base is updated, the existing conclusion cannot be transferred directly.
    2. Diminishing returns at scale. After environment scale grows by tens of times, whether gains persist depends on whether the new environment brings “verifiable new information.” If a large share of tasks cannot distinguish model behavior, further scaling only adds noise.
    3. Non-uniformity of capability. The regressing metrics remind us that post-training is not a monotonic gain. For critical scenarios, establish a “no-regression” baseline so that an overall score increase does not mask local capability drops.
    4. Timeliness of safety evaluation. Safety-evaluation conclusions have a time window; the stronger the capability and the wider the release, the more you need continuous monitoring and response mechanisms.
    5. Timeliness of benchmarks. Once an interactive benchmark is public, its discriminative power decays over time, so the task set needs periodic refresh or expansion.

    9. Conclusion: Post-Training Scaling Is a New Lever for Capability Growth

    The real significance of this release is not any single score or ranking. It is that it provides a clear sample: without changing the base, scaling post-training alone can deliver order-of-magnitude capability gains on interactive tasks such as terminal control, real-world software engineering, and cybersecurity.

    This path rests on three conditions:

    • A scalable and verifiable environment.
    • High-quality automated verifiers.
    • Efficient RL infrastructure.

    None can be missing.

    For developers, the right way to evaluate this kind of model iteration is to return to the task itself: can it reliably complete long-horizon tasks in my environment, is the cost manageable, and is the risk acceptable? On that basis, building an engineering loop of “environment—verification—infrastructure” is what turns model capability growth into business value growth.