Post-Training Scaling for Coding Agents: A Methodology for Capability Leaps

    This article examines a recent model release that kept the base model, architecture, total parameters, active parameters, and tokenizer unchanged, and instead scaled post-training environments and reinforcement learning compute. It explains why controlled post-training experiments matter, how to expand long-horizon environments, why automated verifier quality determines the reward signal, and which RL infrastructure choices affect exploration throughput. It also outlines evaluation approaches for terminal operation, software engineering, and cybersecurity capabilities, along with a practical implementation order and key risks.

    1. A Signal Worth Attention: Capability Growth Is Starting to Come From Post-Training

    A recent release from a leading model team in China introduced a frontier model aimed at coding and cybersecurity. What drew attention in technical circles was not its parameter count, but a methodology the team made public: the base model, architecture, total parameters, and active parameters were all unchanged. The only variable changed was post-training. Over roughly a one-month period, the team scaled up long-horizon interactive environments by tens of times and expanded reinforcement learning compute in parallel.

    This matters because it turns the question “where does capability improvement actually come from?” into something close to a textbook-style controlled experiment. The model is a mixture-of-experts architecture with roughly 700 billion total parameters and about 40 billion active parameters per token. It also includes multi-layer attention compression, sparse indexing mechanisms, and a million-token context window. These conditions were identical between the previous and current generations. There was no additional pretraining and no tokenizer change. Therefore, the reported results—roughly sixfold gains on terminal-task benchmarks, a roughly 20-point improvement on real software-engineering benchmarks, and a cybersecurity benchmark result that placed the model near the top globally—can only be attributed to changes in the data and compute structure of the post-training stage.

    For any team building its own model, the implication is broader than a single benchmark score. When the marginal return from improving the base model is diminishing, investing in a pipeline made up of executable environments, verifiable rewards, and high-throughput RL infrastructure is one of the more cost-effective paths to capability growth today.

    2. Why Controlled Post-Training Experiments Matter

    Many teams iterate on models by changing the base data mix, SFT datasets, and preference-optimization strategy at the same time. When capabilities improve, the source of the improvement cannot be clearly attributed. A controlled-variable approach to post-training iteration has three practical engineering benefits:

    • Clear attribution: changes in capability can be tied to a specific stage of post-training, such as insufficient environment diversity, excessive verifier noise, or insufficient rollout throughput.
    • Low regression cost: if the base model remains unchanged, the tokenizer, inference stack, quantization scheme, and context-management logic can all be reused, and the evaluation system does not need to rebuild baselines.
    • Short iteration cycles: pretraining is a month-level investment, while a post-training scaling experiment can validate a hypothesis in a week-level cycle and roll back quickly if it fails.

    The tradeoff is equally clear: the upper bound of post-training is constrained by the base model. If the base model has structural weaknesses in long-context reasoning or code syntax understanding, post-training can only expose existing capabilities; it cannot create capabilities from nothing. The correct positioning of this approach is to fully unlock the potential of the base model, not to replace pretraining.

    3. Long-Horizon Environment Expansion: From Writing Problems to Building Environments

    Traditional post-training data is mostly static question-answer pairs, but agent capabilities must be learned through interaction. Long-horizon environment expansion is essentially three things:

    1. Environments are data. A training sample is no longer just “question plus answer.” It is a complete executable context: an operating system image, a code repository with dependencies installed, test cases with deliberately injected defects, available network and services, and explicit success criteria. The information density of a sample comes from the richness of the environment, not from lengthy text.

    2. Tasks must be long-horizon. The strategies learned from short tasks, such as writing a single function, differ greatly from those required for long tasks, such as locating a defect in an unfamiliar repository, refactoring across multiple files, or retrying after failure. Long-horizon tasks force the model to learn planning, state tracking, error recovery, and tool-call orchestration. These are central to terminal control and real software engineering capabilities.

    3. Environments must be produced automatically. Manually constructing environments cannot scale. A practical approach is to start from real repositories and issues, automatically generate task descriptions, automatically prepare reproducible images, and automatically generate test pairs that fail before the fix and pass after the fix. The key constraint is reproducibility. Under the same image and random seed, bringing up the environment at any time should produce an identical initial state. Otherwise, the reward signal will be polluted by environment noise.

    The experience-based criterion for environment scale is not quantity, but a coverage matrix: programming languages, dependency complexity, repository size, task types such as defect fixing, refactoring, performance optimization, and security hardening, context-length distribution, and failure-mode distribution. If any dimension collapses, the model will overfit to the corresponding scenarios.

    4. Automated Verifiers: Reward-Signal Quality Determines the Upper Bound

    There is a repeatedly confirmed lesson in reinforcement learning: models optimize the verifier, not the task itself. If the verifier is unreliable, training systematically amplifies errors. Verifiers should be designed in layers:

    Level Implementation Advantages Risks
    Executable checks Unit tests, integration tests, compilation checks Objective and requires no human effort Incomplete test coverage may let incorrect implementations pass
    State consistency Artifact diffs against expected constraints, performance thresholds, static checks Quantifiable and relatively low cost Metrics can be optimized in a targeted way
    Rules and heuristics Allowlists for dependency changes, denylists for dangerous calls Blocks obvious cheating Rule maintenance costs
    Model review Small models scoring code quality and maintainability Covers subjective dimensions Noisy and easier to exploit

    There are three engineering disciplines that must be followed:

    First, separate the verifier from the policy model. Letting the model being trained also act as the scorer makes it very easy for training to degenerate into “learning to please itself.” The scoring model should be independent, rotated regularly, and alert to adversarial text in the policy model’s outputs.

    Second, build an evaluation set for the verifier itself. Use a batch of human-labeled success and failure samples to continuously measure the verifier’s accuracy and recall. If verifier accuracy is below a usable threshold—commonly considered below 90%—the team should fix the verifier first instead of increasing RL compute.

    Third, denoise rewards. Use majority voting or the median across multiple runs, fix random seeds for stochastic test tasks, and classify timeouts and resource exhaustion as a separate reward category to prevent “timeout” from being misread as “failed but partially correct.”

    5. Reinforcement Learning Infrastructure: Throughput Determines Exploration Breadth

    Whether post-training scaling can be put into practice depends heavily on engineering efficiency. In practice, four bottlenecks often need to be addressed:

    • Rollout throughput: Agent trajectories are long-sequence, multi-turn interactions, so single-sample collection is expensive. Practical strategies include decoupling asynchronous sampling from training, action-level caching, shared tool-execution pools, and batch scheduling for similar tasks.
    • Long-tail task isolation: A small number of tasks can consume most of the time, such as compilation, dependency installation, and running full test suites. They should be scheduled in tiers based on estimated runtime, with hard timeout limits.
    • Trajectory and artifact persistence: Code artifacts, logs, and test results produced during training need to be stored in highly reliable storage for replay, auditing, and secondary construction of evaluation sets. A common combination is using containerized clusters to host environments, object storage to hold trajectories, and logging and monitoring systems to track sampling success rates.
    • Stability and reproducibility: Random seeds should be fixed, and the data version and environment version of each experiment should be recorded. Otherwise, it is impossible to determine whether a gain comes from the data, the algorithm, or random fluctuation.

    Below is an illustrative skeleton for explaining the structure. It is pseudocode and does not represent a real API:

    1. from dataclasses import dataclass
    2. from typing import Callable, List
    3. @dataclass
    4. class Task:
    5. env_id: str # Reproducible image identifier
    6. prompt: str # Task description with a long-horizon goal
    7. verify: Callable[[str], float] # Artifact -> reward in [0, 1]
    8. timeout_s: int = 1800
    9. cost_penalty: float = 0.0 # Penalty for steps or duration
    10. def collect_batch(agent, tasks: List[Task], k: int) -> List[dict]:
    11. """Sample k trajectories for the same task to enable group-normalized advantages."""
    12. trajs = []
    13. for t in tasks:
    14. for _ in range(k):
    15. trace = agent.run(t.prompt, env=t.env_id, timeout=t.timeout_s)
    16. if trace.timed_out:
    17. reward = 0.0 # Count timeouts separately as zero; do not mix with partial credit
    18. else:
    19. reward = t.verify(trace.artifact) - t.cost_penalty * trace.steps
    20. trajs.append({"env": t.env_id, "reward": reward, "trace": trace})
    21. return trajs
    22. def train_step(agent, tasks, verifier_audit, k=8):
    23. trajs = collect_batch(agent, tasks, k)
    24. adv = group_normalize(trajs) # Within-group relative advantage to reduce variance
    25. adv = filter_unreliable(adv, verifier_audit) # Downweight samples with low verifier confidence
    26. loss = policy_loss(trajs, adv) + kl_penalty(agent.ref, trajs)
    27. agent.update(loss)
    28. log_metrics(trajs, adv) # Log sampling success rate, average steps, and reward distribution

    The filter_unreliable and group_normalize steps are easy to overlook, but they directly determine whether training converges or oscillates.

    6. Capability Map: How to Evaluate the Three Main Lines

    Terminal and system-operation capability measures how well the model can compile, configure, troubleshoot, and orchestrate services in a real shell environment. The difficulty lies in state management: the side effects of one command change all subsequent decisions, and the model must maintain an accurate mental model of the environment. Evaluation should include multi-step dependency tasks and failure-recovery scenarios.

    Real software-engineering capability measures the ability to work in an existing codebase: reading code, locating problems, and making changes without breaking existing behavior. Key metrics include not only task pass rates, but also whether artifacts are minimally modified, whether regression tests pass, and whether unnecessary dependencies are introduced.

    Cybersecurity capability is the line that requires the most caution. Defensive capabilities—vulnerability localization, configuration auditing, patch generation, and attack-surface review—have clear engineering value and deserve focused investment. Offensive capabilities are inherently dual-use. A reasonable approach is to use isolated sandboxes and self-built ranges for all training environments, perform only structured assessment of artifacts rather than retaining directly exploitable end-to-end chains, and complete security assessment and hardening before opening weights, following common practices among publishers. This “assessment first, progressive openness” rhythm is becoming an industry consensus.

    7. A Suggested Implementation Order for Reproduction

    1. Build an evaluation baseline before training. Fix a private evaluation set that does not overlap with public benchmarks, and make the automated evaluation pipeline work. Without a trustworthy baseline, all post-training gains are impossible to judge.
    2. Fix verifiers before expanding environments. Measure verifier quality with human-labeled samples, push accuracy into a usable range, and only then scale environment size. Otherwise, compute will be consumed by reward noise.
    3. Expand environments in batches by dimension. First validate at small scale that each dimension can be trained, then scale it up. Monitor the curve of sampling success rate as environment scale grows; a sharp drop indicates an imbalanced task-difficulty distribution.
    4. Prioritize infrastructure throughput. Doubling rollout throughput roughly doubles exploration breadth under the same budget, which often delivers greater returns than tuning algorithm hyperparameters.
    5. Build anti-cheating and regression systems. Periodically scan artifacts with a cheating-detection set—deleted tests, tampered assertions, hard-coded expected values—and run full regression on every post-training version to prevent capabilities from trading off against one another.
    6. Complete security assessment before releasing weights. This is especially important for models involving offensive and defensive capabilities. Misuse-risk assessment and hardening should be completed in isolated environments before deciding the scope and pace of openness.

    8. Boundaries and Risks: Post-Training Is Not a Panacea

    Four limitations need to be clearly recognized. First, overfitting to evaluation sets is the most common trap. If training tasks and evaluation tasks come from the same distribution, a score increase does not necessarily mean improved generalization. Out-of-distribution tasks and human blind evaluations should be used for cross-validation. Second, reward hacking persists over the long term. Models will look for verifier weaknesses rather than truly solving the problem, so verifiers require continuous adversarial iteration. Third, the base model sets an upper bound. Post-training cannot compensate for structural defects in language understanding or instruction following. Fourth, capability and risk arise together. Improvements in coding and security capabilities will also increase the possibility of misuse, so governance mechanisms must be built alongside training.

    9. Conclusion

    The technical message conveyed by this release is a clear engineering judgment. In a stage where base models are converging and the returns from parameter races are diminishing, the variables that determine agent capability are shifting from “how large the model is” to “how good the environments are, how accurate the rewards are, and how fast sampling is.” Building post-training as an independent, attributable, regressionable, and scalable engineering system—with automated environment production, layered and trustworthy verifiers, high-throughput RL infrastructure, and closed-loop evaluation and governance—is more likely to produce sustained capability growth than chasing a single leaderboard score. For teams preparing to build models from scratch or optimize existing models, the transferability of this methodology is far greater than any one specific score improvement.