Post-Training Scaling Laws: Improving Models Without Changing the Base

    This article examines a growing trend in model development: improving a model through post-training while keeping its base model, architecture, total parameters, and active parameters unchanged. It explains why parameter count is only one part of model scaling, how post-training can push a model up the capability-cost-latency Pareto frontier without increasing inference cost, and why benchmark rankings need careful interpretation. It also covers chain-of-thought length, practical evaluation methods, and the role of executable environments in advancing code-focused models.

    In recent weeks, the AI community has been discussing a set of claims about a model vendor that took an already released flagship model and kept the base model, network architecture, total parameter count, and active parameter count unchanged. By applying roughly one month of additional post-training and reinforcement learning, the vendor reportedly matched another vendor’s flagship model on an authoritative third-party public benchmark. Once the news spread, an old question resurfaced: should teams keep adding parameters, or should they extract more capability from the parameters they already have?

    For many people, the first reaction to “continued training on the same base” is that the model was optimized mainly for leaderboard scores. But when the training pipeline is broken down, this iteration touches a part of Scaling Law that is often overlooked.

    1. Scaling Law Is About More Than Parameters

    The common simplification is that “bigger models are stronger.” However, a large body of practice and public discussion points to a more complete picture: model size is only one independent variable. At least four factors need to be considered together.

    1. Training data. The ratio between parameters and data, along with domain coverage, difficulty distribution, and noise level, often matters more than simply increasing parameter count when determining what a model has seen and learned.
    2. Training duration and compute budget. The number of training tokens determines how thoroughly each parameter is trained. The industry consensus is that if parameter count is raised too aggressively while data is severely insufficient, the extra capacity will remain idle. Many early models were effectively too large and underfed.
    3. Inference cost. Looking only at the cost of a single training run while ignoring long-term inference overhead is a common planning mistake. After launch, a model may be called millions of times per day, and every cent saved on inference can be amplified by call volume.
    4. Model architecture. Dense models activate all parameters at every computation step, while sparse mixture-of-experts (MoE) models activate only a small subset of experts for each token. Under the same compute budget, MoE can store more resident knowledge, but it also increases memory usage, communication overhead, and engineering complexity.

    These four factors are difficult to optimize simultaneously. In practice, they form a multi-objective optimization problem. The more useful question is not simply which model has the most parameters, but which configurations are efficient across capability, training cost, inference cost, and response latency.

    2. What “Not Changing the Base” Means

    “Not changing the base” means retaining the pretrained base model and the overall architecture, then improving capability through post-training. This usually includes:

    • Supervised fine-tuning: teaching the model the expected output format for target tasks, such as tool-call formatting, structured output, or code style;
    • Preference optimization: adjusting output tendencies based on preference rankings from human or model reviewers;
    • Reinforcement learning with verifiable rewards: using automated feedback such as unit tests, compilation results, or answer validators as reward signals.

    Pretraining determines the upper bound of capability. Post-training determines how consistently and efficiently that capability can be invoked. After pretraining, many capabilities already exist in a latent form; the model simply may not “produce” them well on specific tasks. Post-training adjusts attention allocation, reasoning paths, and output formats to a more suitable state.

    Post-training is not free of costs:

    • Diminishing returns: repeatedly stacking the same strategies tends to produce smaller gains over time;
    • Catastrophic forgetting: if the training distribution is too narrow, the model may sacrifice general capability;
    • Reward hacking: if the optimization target is poorly defined, the model may learn to exploit gaps instead of solving the real problem;
    • Reproducibility: continued training requires careful management of data versions, random seeds, and evaluation baselines.

    Therefore, interpreting “continued training on the same base works” as “the base does not matter” is a misreading. A more accurate statement is that on a base model with sufficient capacity and a solid foundation, the marginal benefit of post-training has been underestimated.

    3. A Rare and Well-Controlled Comparison

    From a methodological perspective, the value of this iteration lies in the fact that it resembles a controlled experiment.

    Variable Changed? Explanation
    Base model No Pretrained weights remain the same
    Network architecture No Layer count, attention mechanism, expert structure, and other details are unchanged
    Total parameter count No Model capacity ceiling remains the same
    Active parameter count No Per-token inference compute remains the same
    Post-training data and reinforcement learning About one month The only actively changed variable

    Keeping other conditions fixed and changing only one variable is more informative than a holistic upgrade that changes the base model, architecture, and data at the same time. It makes the contribution of post-training itself easier to observe.

    The comparison is not perfect. The composition and quality of the post-training data are not public, and different vendors may not use identical leaderboard submission settings. Even so, the fact that a model can approach a stronger competitor using only post-training while keeping the same parameter count and architecture supports one conclusion: the relationship between parameter scale and task performance is not linear.

    4. Occupying a Pareto Frontier Point

    Continued iteration on the same base model brings another benefit that is often overlooked: every time capability improves by one level, response speed and per-call cost remain almost unchanged. In the three-dimensional space of capability, cost, and latency, this means the model moves up one step in capability without moving along the cost or latency axes.

    The Pareto frontier is the set of options that are not dominated by any other option across all metrics. In plain terms, options on the Pareto frontier have no replacement that is simultaneously cheaper, faster, and stronger. The significance of the frontier is not the ranking itself, but the fact that candidates on it deserve consideration no matter whether the priority is cost, latency, or capability.

    The landing point of this iteration is especially notable. It is not at the expensive top end of parameter count and cost, but in the mid-to-high capability range where competition is intense and demand is concentrated.

    • Top-end models: highest capability ceiling, but also highest inference cost and latency; suitable for a small number of high-value, high-difficulty scenarios.
    • This range: can cover most production workloads, including daily coding, document processing, and data analysis.

    When a near-top model lands near the inflection point of the cost curve, its total cost of ownership advantage can be significant. In other words, its value is not just a few extra points on a leaderboard, but the fact that these capabilities appear in the range where most teams actually spend money.

    5. Chain-of-Thought Length: Trading Time for Parameters

    According to real-world testing feedback, this model did not exhibit abnormally long chains of thought. This contrasts with another phenomenon worth observing: some smaller-parameter models are using longer reasoning processes to compensate for limited capacity.

    Why Longer Chains of Thought Can Trade Time for Parameters

    The computational depth of a Transformer is determined by its number of layers, and a single forward pass can complete only a limited amount of computation. When model capacity is insufficient, a natural compensation is to let the model “think longer.” Research on test-time compute scaling, such as the approach proposed by OpenAI, also shows that investing more compute at inference time can improve accuracy to a meaningful extent.

    The reason is that every reasoning step generated by the model writes an intermediate result back into the context. Later computation can read these results directly, which is similar to writing extra drafts outside the fixed computational depth. The longer the draft, the more complex multi-step reasoning the model can complete. This is also an important reason why smaller-parameter models perform well on tasks such as code and mathematics, where progress can be verified step by step and errors can be corrected within the draft.

    However, “trading time for parameters” has clear costs:

    • Serial generation increases latency and degrades user experience;
    • As the context grows longer, caching, memory usage, and cost all increase;
    • Incorrect conclusions in the draft may be repeatedly referenced, causing the model to drift further off course;
    • On simple tasks, overly long reasoning may introduce unnecessary interference.

    Therefore, a model’s real performance depends on three things: parameter capacity, chain-of-thought length, and whether the task requires multi-step verification.

    What This Observation Suggests

    This model matched a stronger competitor under the same architecture at a 700B-class parameter scale without relying on an unusually long chain of thought. There are at least two possible interpretations:

    1. The 700B-class parameter scale may already provide sufficient capacity for the target tasks, and post-training released that capacity more fully;
    2. From another perspective, models that often produce very long chains of thought may also be trading latency and cost for accuracy.

    This leads to a cautious judgment: for tasks such as coding that can be automatically verified, a 700B-class parameter scale may already be sufficient. After that point, the main cause of the remaining gap may no longer be parameter count, but whether enough high-quality executable environments can be produced for training. For more complex long-horizon planning, multimodal understanding, or open-domain reasoning, whether the capacity threshold is still higher requires more experiments.

    6. Reading Benchmark Leaderboards: Six Common Mistakes

    Benchmarks are useful signals, but using them directly as the basis for model selection can easily lead to mistakes:

    1. Aggregate scores are weighted averages. Leaderboards usually provide a composite index covering several capability dimensions with assigned weights, but a business may rely mainly on only one or two dimensions.
    2. Version drift. The same leaderboard may update scores over time, so rankings captured at different dates are not necessarily comparable.
    3. Evaluation setup differences. Whether thinking mode is enabled, how sampling parameters are configured, and how many tool calls are allowed can significantly affect results, and leaderboards may not fully reflect your actual usage pattern.
    4. Serving differences. The same model can show very different latency, throughput, and stability after deployment configurations, inference backends, and quantization schemes on different cloud platforms.
    5. Data overlap. Overlap between public test questions and training data cannot be completely ruled out, and evaluation sets also need continuous updates.
    6. Offline scores are not online experience. There is often a clear gap between offline pass rates and success rates on real online tasks.

    7. Practical Validation: How to Tell Whether a Model Fits You

    An experienced tester reported that after using a new-generation model in an AI coding tool, there was no obvious difference compared with the previous generation. The tester also acknowledged that the tasks may not have been difficult enough to reach the model’s capability ceiling.

    This highlights the limitations of personal testing: small samples and everyday tasks make it hard to distinguish whether a model is stronger or simply better packaged. To make validation more convincing, subjective impressions can be turned into a reproducible evaluation process:

    • Task set: select 50 to 200 representative tasks from real team work, such as bug fixes, cross-file refactoring, and API integration, with clear acceptance criteria for each task;
    • Unified conditions: use the same system prompts, context scope, and sampling parameters, and run each task multiple times to distinguish occasional success from stable success;
    • Multi-dimensional records: record success rate, average time, token consumption, and the number of human interventions, then convert them into a comprehensive cost per successful task;
    • Blind evaluation: hide model identifiers and have multiple people score independently to reduce subjective bias;
    • Regression testing: rerun the same task set after each upgrade to observe whether capability has degraded.

    Beyond capability, engineering observability is also important. Using general cloud capabilities such as logging, monitoring, and alerting, teams can continuously collect success rates, latency, and token consumption for different models. This turns model selection from a one-time comparison into an ongoing, data-driven decision.

    8. The Next Bottleneck: Executable Environments

    Why is the next bottleneck for code tasks executable environments? Because code has a scarce property: it can be automatically verified by machines at low cost and with reasonable accuracy. Whether a feature works can be checked by running unit tests, and whether it passes type checks or static analysis can be determined by compilers and analyzers. These signals are exactly the fuel needed for reinforcement learning.

    Conversely, if the training environment quality is poor, reinforcement learning will quickly fail. Common problems include:

    • Dependencies cannot be reproduced, causing high reward noise;
    • Test coverage is insufficient, so the model may actually fix the issue but receive no reward;
    • Reward design has loopholes, so the model learns to modify tests or take shortcuts to “score” rather than solve the problem;
    • Tasks are disconnected from real engineering, so the model performs well on toy repositories but struggles with large-scale projects.

    A minimum viable data loop can be summarized as follows. It comes from real R&D data, with version control, access control, and data anonymization throughout the process.

    1. # Pseudocode: Build trainable executable environments from real development data
    2. def build_rl_tasks(repos, issue_tracker, days=90):
    3. tasks = []
    4. # 1. Collect only real changes that have already been validated
    5. for pr in merged_pull_requests(repos, since=days_ago(days)):
    6. if not (pr.tests_added and pr.ci_passed):
    7. continue
    8. # 2. Roll back the change to construct the "pre-fix" code state
    9. env = snapshot(repo=pr.repo, commit=pr.base_commit)
    10. tests = select_tests(pr.tests_added) # Select tests related to the change
    11. tasks.append({
    12. "prompt": compose_issue(pr.title, pr.description),
    13. "env": env,
    14. "verify": lambda e: run_tests(e, tests) and run_lint(e),
    15. "reference_patch": pr.diff, # Used only for offline evaluation
    16. })
    17. # 3. Filter: reproducible, verifiable, and of moderate difficulty
    18. tasks = [t for t in tasks if reproducible(t) and not trivially_solvable(t)]
    19. return tasks
    20. # Reward = functional correctness - resource cost; tests run in an isolated environment, and access to the reference solution is prohibited
    21. def reward(task, rollout):
    22. sandbox = isolated_env(task.env) # Locked dependencies, controlled network
    23. passed = verify(sandbox, rollout) # Tests + static checks
    24. return (1.0 if passed else 0.0) - token_cost_penalty(rollout)

    The key to this loop is not the code itself, but three principles:

    • Automatically verifiable rewards: tests run in an isolated environment, and access to the reference solution is prohibited to prevent reward hacking;
    • Gradable difficulty: difficulty levels are divided according to the pass rate of a baseline model sampled multiple times, avoiding tasks that are all correct or all incorrect and therefore provide no useful gradient;
    • Scalable difficulty: as model capability improves, harder tasks can be continuously mined from larger and more complex real repositories.

    When these three points are done well, every real R&D activity can be converted into new training resources. The stronger the model, the more difficult tasks it can handle; once more difficult tasks are verified, they flow back as stronger training signals. This is the real meaning behind the statement that the remaining gap depends on executable environments.

    9. Conclusion

    The discussion around post-training on the same base leaves at least three lessons:

    1. Parameter count is not the only variable. Data mixture, training investment, inference cost, and model architecture jointly determine model efficiency. Scaling laws are a set of trade-offs, not a monotonic relationship.
    2. Post-training is an underestimated lever. On a base model with sufficient capacity and a solid foundation, continuous and controlled post-training can move the model up the Pareto frontier without increasing inference cost.
    3. Evaluation must return to your own scenario. Leaderboards are useful for discovering candidates, but final decisions still require evaluation sets close to the business, reproducible conditions, and blind scoring.

    For teams, asking “which model has now matched which other model” is less valuable than investing in three fundamentals: building your own evaluation set, understanding cost and latency along the call chain, and turning real R&D experience into high-quality executable environments. Competition among models will continue, but what ultimately determines application outcomes is often these fundamentals.