From Scaling Wars to Post-Training Value Reassessment: The Technical Divergence Behind a Benchmark Tie

    This article explores the technical divergence in large language model development, contrasting massive parameter scaling in pre-training with deep reinforcement in post-training. It analyzes how a 700-billion-parameter model matched a 2.8-trillion-parameter model through post-training optimizations alone, examining the evolution of scaling laws, the separation of total and activated parameters, and why post-training is transitioning from an alignment tool to the primary engine for model intelligence.

    A Benchmark Tie: The Path Dispute Beyond the Scores

    Recently, the latest intelligence index published by an AI evaluation platform revealed that an open-source model with approximately 700 billion total parameters tied in overall intelligent performance with a leading open-source model boasting 2.8 trillion total parameters. Both now share the top spot among open-source models and are approaching the first tier of proprietary flagship models. This result itself is not surprising; the rapid iteration of open-source models over the past year has been evident. What truly warrants industry attention is the implicit difference in implementation paths hidden behind these two sets of numbers.

    Even more intriguing is the competition score in a specific sub-dimension: the 700-billion-parameter model achieved a significantly higher score than the comparison model, jumping to second place overall, trailing only behind the top proprietary model. This change was not incremental; it leaped directly from the 1,500-point range to the 1,770-point range, an improvement of over 240 points in a single iteration.

    If this jump were merely the natural progress brought by parameter stacking, it could be interpreted as yet another example of “brute force.” But the reality is quite the opposite—the base model of this iteration is entirely identical to its previous generation. The architecture remained unchanged, the parameters were not increased, and no additional compute was invested in the pre-training phase. All the leaps in scores came entirely from the expansion of the post-training phase.

    This raises a core question: When the pre-training scale no longer grows, how much further can a model’s intelligence go?

    Two Distinct Technical Paths

    In the current competitive landscape of large models, there are two distinctly different technical routes.

    Path One: Massive Parameter Expansion in the Pre-training Phase. This is a “parameter stacking” strategy characterized by larger total parameters, more expert modules, and higher training compute. These models typically possess trillion-level total parameters, activating hundreds of billions of parameters per inference. The core logic is: through massive data ingestion and parameter fitting during pre-training, the model “memorizes” more knowledge and patterns, thereby acquiring stronger generalization capabilities.

    Path Two: Frozen Base, Deep Reinforcement in Post-training. This approach keeps the pre-trained base model unchanged, investing all optimization resources into post-training phases such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback (RLHF), and especially Reinforcement Learning (RL). By constructing richer task environments, longer decision chains, and larger-scale trial-and-error signals, the model learns to “think better” and “reason more effectively” based on its existing knowledge reserves.

    The collision of these two paths constitutes the most observable divergence in current large model evolution. The former is like expanding a library’s collection—the more books there are, the more comprehensive the materials that can be consulted. The latter is like training a reader’s reading methodology—the collection remains unchanged, but the reader learns to reason, argue, and cross-verify, capable of drawing deeper conclusions from the same books.

    Real-world data comparisons have already provided a preliminary answer: under conditions where the total parameters differ by nearly four times, the post-training-driven model not only matched the former in overall intelligence but also significantly led in specific dimensions involving complex reasoning. This indicates that the intelligence ceiling of a model is not entirely determined by its parameter count.

    Evolution of Scaling Laws: From Parameters to Data, to Inference Costs

    To understand the phenomenon above, it is necessary to review the three major revisions of scaling laws over the past five years. This is not just an evolution of academic discussion, but a shift in the resource allocation focus of the industry.

    Phase One: Parameter Priority (2020). Early scaling law research yielded an influential conclusion: the growth rate of model parameters should be significantly faster than the growth rate of training data, with a recommended ratio of about 2.7:1. This meant that when preparing to double the data volume, the parameter scale should increase by nearly three times. This conclusion directly spawned a batch of super-large models with hundreds of billions or even trillions of parameters, as the industry immersed itself in the naive belief that “bigger parameters mean stronger capabilities.”

    Phase Two: Data Matching (2022). Subsequent researchers overturned the above conclusion through finer experiments, pointing out that the optimal parameter-to-data ratio should be approximately 20 training tokens per parameter. This meant that many previous models were actually “underfitting” the data—parameters increased too quickly while data ingestion was insufficient, leading to underutilized model capacity. This revision shifted the industry’s focus to the dual improvement of data quality and scale, spawning a batch of more “balanced” models.

    Phase Three: Inference Cost Perspective (Present). However, the previous two generations of scaling laws shared an implicit premise—they optimized the static cost of “train once, evaluate once.” But today’s large models are deployed in production environments, called billions of times daily. The compute consumption during the inference phase is the real cost driver. If this variable is factored in, the optimal solution is no longer “maximum parameters” or “most comprehensive data,” but rather “achieving the target intelligence level with the minimum activation cost, provided inference needs are met.” This leads to a new paradigm of “small model, long training”: the base need not be excessively large, but sufficient compute must be invested in the post-training phase to “squeeze out” every bit of the model’s potential.

    This shift in perspective is fundamentally disruptive: it challenges the intuition that “model capability is strongly bound to parameter count.”

    Hierarchical Scaling: Separating Total and Activated Parameters

    A new core insight suggests that model scaling should not be a chaotic, monolithic process, but should be deconstructed into two independent dimensions corresponding to different capability levels:

    First, total parameters determine “knowledge capacity.” The total parameter count of a model characterizes how much knowledge and long-tail information it can store. A larger total parameter count means it can remember more factual content, rarer linguistic patterns, and finer distribution features. This is the “raw material library” for all complex reasoning.

    Second, activated parameters and effective depth determine “reasoning capability.” Under the Mixture of Experts (MoE) architecture, the number of activated parameters actually involved in computation during each inference, along with the computational chain depth formed by these parameters, determines how many steps of causal deduction the model can carry. It characterizes the model’s ability to “think far and deep” given a certain knowledge reserve.

    These two can be separated physically and logically. A model with massive total parameters but only a few expert modules activated during inference might excel at memorization but be clumsy at multi-step reasoning. Conversely, a model with limited total parameters but a high activation ratio and deep computational chains might demonstrate stronger performance in complex logical tasks.

    Further quantitative analysis points out that under the constraint of a fixed total compute budget, blindly pushing up total parameters will actually dilute the compute density of a single activation, thereby harming reasoning capability. Conversely, architectural optimization that allows more expert modules to be activated during a single inference can steadily improve the model’s performance on complex reasoning tasks.

    This explains the aforementioned evaluation results: in that specific score, the post-training-driven model was able to lead significantly precisely because it learned, through reinforcement learning in the post-training phase, how to more effectively utilize its “effective depth” within a limited activated parameter budget—compressing, refining, and combining intermediate steps in multi-step reasoning chains, thereby deriving better answers under the same or even less inference overhead.

    Post-Training: From “Alignment Tool” to “Capability Engine”

    For a long time, post-training was viewed as a “decorative” step after pre-training—primarily used for aligning human preferences and improving the politeness and formatting of responses. But a new generation of practices is completely reversing this perception: post-training is becoming the true engine of capability growth.

    There are three key elements to this:

    First, the richness of task environments. Traditional reinforcement learning relied on a few fixed Q&A or instruction environments, making models prone to overfitting with limited generalization space. New-generation approaches expand the post-training environment to dozens of types: from multi-step reasoning, mathematical proofs, and code execution, to strategic gaming, tool calling, and long-horizon planning. Each environment forces the model to establish new connections between “existing knowledge” and “action strategies,” fundamentally reshaping the operational methods of its internal representations.

    Second, the depth of long-horizon tasks. Early post-training tasks were typically only a few steps away, requiring the model to make locally optimal decisions. Current post-training introduces a large number of long-horizon tasks requiring tens or even hundreds of continuous decision steps. Such tasks require the model not only to “think correctly” but also to “think far”—envisioning subsequent multi-step feedback, corrections, and re-reasoning in advance. The result of long-term trial and error is the formation of deeper causal chain representation capabilities within the model, which is the essence of complex reasoning.

    Third, the density and quality of reinforcement signals. Judging whether a model “answered well” cannot stop at the binary judgment of “whether it answered correctly.” New-generation post-training introduces mechanisms like verifiable reward signals, process reward models, and multi-model cross-discrimination, refining the feedback granularity of reinforcement learning from “whether the whole paragraph is reasonable” to “whether each step is reasonable.” This dense, high-resolution feedback signal allows the model to precisely locate and repair weak links in its own reasoning chain, much like a human expert reviewing a game.

    The combination of these three fully activates the existing parameter capacity—akin to “reprogramming” through post-training on the same base, reorganizing the collaboration mode among parameters from “loosely associated memory” to “rigorous reasoning network.” This is the intrinsic reason why model capabilities can achieve such leaps without any change in total parameters.

    Economic and Technical Perspective: When Training Costs No Longer Matter Compared to Inference

    If the above analysis is an academic judgment, the accounting of economic costs provides a more realistic basis for decision-making.

    According to public industry data, training a top-tier model from scratch can require compute costs in the tens of millions of dollars for the pre-training phase alone. The subsequent full-lifecycle inference costs are often amplified by an order of magnitude or more—because training is a “one-time investment,” while inference is a “daily ongoing investment.”

    This leads to an important resource allocation logic: Rather than spending high budgets repeatedly expanding parameter counts, it is better to invest more compute into the post-training phase to continuously optimize the model’s inference efficiency and capability ceiling. This is not only the optimal technical solution but also an economically wise choice.

    At the same time, this is a subtle variable in the competition between open-source and proprietary models: once model weights are made open, the community can even conduct independent post-training research on the same base. This means that the developers of the base model and the developers of post-training capabilities can be two different groups. Post-training technology thus becomes a public capability unit that can be independently developed and freely combined. This will, to a certain extent, break the monopoly narrative that “model capabilities are entirely determined by pre-training investment,” allowing more lightweight teams to play a substantial role in the evolution of model intelligence.

    Future Competition Focus: From “Building Skyscrapers” to “Internal Cultivation”

    This benchmark tie event should not be simply understood as a victory for a specific model vendor, but should be viewed as a landmark node of the industry’s cognitive shift on the question of “where does model intelligence come from.”

    The main thread of competition over the past two years has been “whose model is bigger” or “whose base is stronger.” In the future, the divergence that truly determines victory or defeat may become: who can maximize the excavation of the model’s potential intelligence on top of an existing base at the lowest inference cost.

    This will bring a series of chain reactions:

    • Reconstruction of evaluation metrics: The practice of only reporting total parameter count and pre-training data scale will gradually lose its reference value; metrics such as activated parameters, effective depth, and inference-time compute density will receive more attention.
    • Shift in engineering focus: Post-training supporting technologies such as reinforcement learning infrastructure, task environment simulators, and automated evaluation systems for long-horizon tasks will become the R&D focus for new-generation models.
    • Differentiation of the open-source ecosystem: The gap in base models may converge, while post-training recipes will become a point of differentiated competition—they may even hold more strategic value than the base model itself.

    Returning to the evaluation data at the beginning of the article. What is truly worth remembering is not the result of “who tied with whom,” but the fact that “they tied despite the total parameters differing by nearly four times.” It shows that the intellectual potential of models is far greater than we imagine; the key is how to release it in the correct dimensions. When compute budgets no longer expand infinitely and inference costs become a constraint, post-training—once considered an “auxiliary step”—is becoming the core battlefield defining the capability boundaries of next-generation models.

    For developers, this trend also provides an actionable guideline: when facing a base model whose performance “seems to have frozen,” do not rush to switch to a larger base. Instead, try exploring the possibilities of post-training in depth—there may lie additional intelligence based on existing parameters that has yet to be unlocked.