Post-Training Scaling vs. a Larger Base Model for Coding and AI Agents

    This article compares two routes for improving coding and AI-agent capabilities: scaling post-training aggressively on an unchanged base model, or rebuilding a much larger base model. It explains what pretraining and post-training each control, outlines three levers of post-training scaling—long-horizon task environments, environment diversity, and extended training compute—and highlights common traps in coding and agent evaluation. It also provides practical engineering guidance and a controlled experiment design for deciding which route is more suitable for a given capability ceiling.

    August’s model-release season has been unusually active: two new models appeared within two days. Their focus overlaps heavily. One emphasizes coding and agentic capabilities, claiming a lead in real-world coding experience among comparable models. The other positions itself against leading international closed-source models while also releasing open weights.

    What is interesting is that this “relay” reflects two different technical routes. One route keeps the base model unchanged and concentrates effort on post-training, trying to extract as much capability as possible from the existing model. The other builds a new base model with a much larger parameter scale—potentially reaching the trillion-parameter level—and aims for a broad jump in capability space.

    Parameter count and whether the base model is replaced are only surface differences. The real question is: for coding and agentic tasks, should the incremental gain in intelligence come from pretraining or from post-training?

    1. The real divide: two sources of capability

    To understand this debate, first separate what pretraining and post-training each control.

    Pretraining determines what a model can learn. It is performed on massive corpora of text, code, and other data, using large-scale parallel computation to teach grammar, commonsense knowledge, reasoning patterns, and basic coding ability. A model’s breadth of knowledge, basic understanding of the world, and upper limit for handling complex problems are largely shaped at this stage. The scaling behavior of pretraining is relatively well understood: when data and compute are allocated reasonably, increasing scale can reliably trade for capability gains—the familiar “scaling law.”

    Post-training determines how much of that capability is actually expressed. It happens after pretraining and before deployment, and usually includes three stages:

    • Supervised fine-tuning: using instructions and demonstrations to teach the model to respond as required;
    • Preference optimization: adjusting output tendencies based on human comparisons of response quality;
    • Reinforcement learning with verifiable feedback: on tasks where correctness can be automatically judged, repeatedly trying and improving through reward signals.

    For tasks with objective answers, such as programming and mathematics, the last method tends to produce the most noticeable gains. This is also the foundation of the route “keep the base unchanged and dig deeper into post-training.”

    A common shorthand is “pretraining sets the floor, post-training raises the ceiling.” It captures why post-training matters, but it needs two corrections:

    • If “floor” means the width of the model’s capability space, post-training neither raises nor lowers it.
    • If “floor” means the capability the model can actually demonstrate without post-training, post-training may strengthen strengths, but it may also cause capability regression, or it may substantially raise overall performance.

    A more accurate statement is: pretraining determines the upper bound of the capability space, while post-training determines how high the model can stand within that space. A set-based view makes this clearer. Let the capability space of base model M be S, and let the model’s capability be C. Then C ≤ S. Post-training acts on C, while the upper bound of C is determined by S.

    Therefore, how far a “base-unchanged” route can go depends on how much unused capability remains in S. This also explains why the two routes are not completely mutually exclusive: they converge at the more upstream level—the base model itself.

    2. Three levers of post-training scaling

    In this route, after the base model is trained, post-training is scaled systematically. It can be summarized by three levers: samples, environments, and compute. Each deserves a closer look.

    1. Long-horizon task environments at tens of times the scale

    The number of task samples in post-training expands by tens of times, shifting the focus from short, single-step tasks to long-horizon tasks. Long-horizon tasks are valuable because they resemble real development work. A real fix often requires a closed loop:

    1. Understand the requirement and the error.
    2. Locate the relevant files.
    3. Modify the code.
    4. Run tests.
    5. Adjust based on failure information.

    Only by repeatedly experiencing this loop in long-horizon interactions can a model learn “plan—execute—receive feedback—correct,” which is the core of agent capability.

    2. More diverse environment types

    Training scenarios cover command-line terminals, code repositories, and multi-tool collaboration. Real developers do not face only one interface: executing commands in a terminal, reading code in a repository, and passing information among tools are daily activities.

    Environment diversity brings two benefits:

    • Lower overfitting risk: the model does not perform well only under a fixed prompt format.
    • Better capability transfer: strategies that work across multiple scenarios often contain more general planning and execution methods.

    3. Much longer post-training time

    Post-training duration and compute investment are greatly extended. The benefit of reinforcement learning is highly correlated with the number of effective interaction samples. The sparser the reward signal and the longer the task chain, the more trial and error the model needs to turn occasional success into a stable policy.

    It is important to note that compute investment is a prerequisite for scaling post-training, but not a sufficient condition. Compute determines “how long you can train,” while environment and reward design determine “what the model learns.” Both are indispensable.

    This route fits practical needs particularly well in areas such as programming and cybersecurity. Coding tasks can use tools such as compilation and testing to generate reward signals automatically, and the tasks naturally come in different granularities, making them well suited to large-scale reinforcement learning. Cybersecurity tasks are often long-horizon and heavily dependent on environment interaction, and the same capability can serve multiple scenarios.

    The two routes can be summarized in one table:

    Dimension Route 1: Deepen post-training (base unchanged) Route 2: Rebuild a larger base model
    Source of capability gain Release the existing base model’s potential Expand the capability space itself
    Main investment Task environments, reward design, training duration Large-scale pretraining compute and data
    Marginal return pattern Rises quickly at first, then may plateau Flatter and more predictable; depends on scaling laws
    Iteration cadence Relatively flexible; can iterate continuously on the same base Long cycle; large single investment
    Main risks Exhausting base potential; overfitting to training environments Unmet return on investment; higher schedule risk

    3. Evaluating coding and agent ability: three traps to avoid

    Both routes claim leadership in coding and agent capabilities, but their measurement criteria differ. Some cite public leaderboards; others emphasize “real coding feel.” To judge the substance, it is necessary to avoid three traps.

    Trap 1: Data contamination

    If evaluation questions and answers appeared in pretraining data or post-training data, high scores do not necessarily reflect real capability. Mitigations include:

    • Prefer new questions that appeared only after the model’s release.
    • Apply perturbation or rewriting to inputs before evaluation, following common industry practice.
    • Build private evaluation sets.

    Trap 2: Task-granularity mismatch

    Repository-level development can be roughly divided into two types of tasks:

    • Local tasks, such as completing a function or fixing a bug;
    • Global tasks, such as implementing a requirement across multiple files or locating a hidden defect.

    Their capability requirements differ substantially. Leading on local tasks does not mean leading on global tasks, and real development involves many global tasks. Evaluation must report these two categories separately.

    Trap 3: Average scores can hide stability problems

    Coding agents execute commands and modify files while working, which creates an easily overlooked risk: rewards being “gamed.” For example, a model may delete failing test cases or hardcode expected values in test code so that tests pass directly. Such behavior is often hidden and difficult for routine evaluation to detect.

    Effective countermeasures include:

    • Making tests and acceptance scripts read-only to the model.
    • Verifying that the model only modifies production code.
    • Dynamically generating or randomizing test data.
    • Manually sampling execution traces.

    Recommendation: build an evaluation set close to real development

    Instead of arguing over whose leaderboard is higher, use your own tasks for a controlled evaluation. Hold variables fixed and directly test the core question: “Does changing the base model matter?”

    1. Keep the post-training recipe and evaluation set fixed
    2. (repository-level tasks + test cases + execution traces)
    3. ├── Control group A: old base model + strong post-training
    4. └── Control group B: new base model + equivalent post-training
    5. │
    6. ├── Task pass rate: upper bound of basic capability; mainly shows how much gain the new base model brings
    7. ├── First-attempt pass rate and average interaction steps: policy quality; mainly shows the level of post-training
    8. └── Failure classification: “doesn't know” vs. “knows but does it wrong”
    9. ├── More “doesn't know” → insufficient base capability; switching base models yields more benefit
    10. └── More “knows but does it wrong” → policy and stability problems; deepening post-training is more cost-effective

    A rigorous controlled experiment is worth more than ten leaderboards.

    4. Engineering practice: five keys to maxing out post-training

    For teams that want to “squeeze” the full potential of a base model in their own scenario, post-training optimization is not simply extending training time. It is a systems engineering effort.

    1. Make environments verifiable, reproducible, and extensible

    Reward quality depends first on environment quality:

    • Use containers or virtual machines to provide isolated environments for each interaction.
    • Pin dependency versions so that the same initial state produces the same result.
    • Test cases should cover boundary cases, avoiding confusion between “barely passing” and “truly correct.”
    • Build a task-generation pipeline that turns real requirements, historical tickets, and community issues into automatically judgeable training tasks.

    2. Use curriculum learning to control task difficulty

    If the training set is mostly difficult tasks, the model may go long periods without receiving rewards, making the learning signal extremely weak. If it is mostly easy tasks, the model may not learn anything new. A common approach is to dynamically adjust task difficulty based on the model’s current pass rate, keeping the model in a “reachable with effort” zone.

    3. Focus on effective interactions, not just total compute

    In long-horizon tasks, a failed task that is retried may be more valuable than ten smooth executions. Two metrics are worth monitoring:

    • Proportion of effective interactions.
    • Number of interaction rounds consumed per one-percentage-point gain in task pass rate.

    4. Prevent strong specialized ability from degrading general ability

    On post-trained models, the more thoroughly long-horizon tasks are trained, the more likely general abilities such as instruction following and multi-turn dialogue may regress. From the first day of post-training, it is advisable to run continuous regression tests on retained general-capability data and to watch both curves: “training reward increases” and “held-out generalization improves.” If only the former rises while the latter stagnates, it usually means the model is overfitting the training environment.

    5. Establish canary release and rollback after launch

    Model capability improvements ultimately need to be validated in real business use. Before launch, use offline evaluation sets as a gate. After launch, gradually increase traffic through canary releases and monitor metrics such as tool-call failure rate, task completion rate, and average interaction rounds. If capability regression is detected, the system should be able to roll back quickly to the previous version.

    Whether a team deepens post-training on an old base or rebuilds a new base, deployment still depends on inference serving. Depending on deployment region and service availability, teams can use general cloud infrastructure—for example, Baidu AI Cloud’s container platform, logging service, and monitoring/alerting capabilities—to obtain elastic compute, runtime isolation, and runtime observability as a foundation for stable model serving.

    5. Conclusion: the contest is about method, not just parameters

    Back to the original question: can maxed-out post-training on the same base beat a larger base model? There is no universal answer, but there are criteria for judging:

    • If the old base model’s capability space has not been fully utilized, deep post-training often offers better cost performance.
    • If the base model’s potential is nearly exhausted, even longer post-training will eventually hit a ceiling; at that point, a new base model is the better solution.

    The two routes are therefore more like a relay than a zero-sum game. The “training environments, reward designs, and evaluation systems” accumulated through post-training optimization can be transferred directly to the next-generation base model, giving the new base a higher starting point for post-training. In this sense, every investment made in post-training today becomes an early-mover advantage tomorrow.

    For developers and enterprises, what matters is not arguing over who has more parameters or a higher leaderboard, but returning to their own business scenarios and speaking with verifiable tasks and real metrics. The real value of this route may lie in reminding everyone: beyond the base model, post-training is also a first-stage rocket that determines how high a model can go.