Post-Training Scaling vs. New Foundation Models: Two Paths to LLM Capability Leaps

    This article compares two strategies for advancing large language model capabilities: deepening post-training on an existing foundation model, and training a new, larger foundation model from scratch. It explains why post-training still has substantial headroom, outlines the main engineering levers behind post-training scaling, and analyzes the logic and costs of rebuilding the base model. It also provides a practical framework for evaluating model releases through verifiable tasks, custom task distributions, long-horizon performance, and total cost of ownership. The article concludes that the real debate is about resource allocation, and that the most reliable signals to track are the improvement curve for verifiable long-horizon tasks and the decline curve for unit intelligence cost.

    Introduction: Two Routes Meet Head-On

    Over the past year, the iteration cadence of Chinese LLMs has clearly accelerated. A striking comparison keeps appearing in the community: some teams choose to dig deeper into an existing foundation model and devote all their effort to post-training, while others choose to start over and train a new foundation model with a larger scale and a newer architecture. The former is like applying extreme extraction to an oil field that has already been explored; the latter is like drilling a new and deeper well.

    These two routes represent one of the central debates in current LLM engineering: as the marginal returns from pretraining scaling gradually slow, where should the next leap in intelligence come from—from “being bigger” or from “being taught better”? This article does not evaluate any specific model. Instead, it abstracts the two routes and examines them across three dimensions: principles, engineering practice, and cost structure. The goal is to help developers build their own judgment framework.

    1. “Extreme Extraction” Without Changing the Foundation: What Post-Training Scaling Actually Does

    1.1 Why Post-Training Still Has So Much Room

    For a long time, the industry has held an implicit consensus: the intelligence ceiling of a model is mainly determined by parameter scale and corpus scale during pretraining, and post-training is only fine-tuning to “align the taste.” But practice over the past two years has repeatedly shown that this judgment underestimates the ceiling of post-training. The reasons are:

    • Pretraining corpora are “static text.” What the model learns is “what the next token is,” not “how to solve a multi-step real problem.”
    • Real engineering tasks—fixing a cross-file defect or delivering an end-to-end requirement—require long-chain planning, trial and error, and backtracking. Such signals are extremely sparse in static corpora.
    • Therefore, using reinforcement learning in environments to let the model “do things” and receive outcome feedback essentially supplements a data dimension that is structurally missing during pretraining.

    1.2 Three Main Levers of Post-Training Scaling

    The technical route represented by “keep the foundation model and deepen post-training” usually concentrates engineering investment in three areas:

    First, expanding long-horizon task environments by orders of magnitude. Rather than having the model answer single-turn questions, put it into a real engineering task chain: read requirements, locate code, modify it, run tests, and iterate based on errors. The longer the task chain, the more the model is forced to practice “state management.” Expanding the number of environments from hundreds to tens of thousands is a typical feature of this route.

    Second, diversifying environment types. A single type of environment makes the model lopsided. A common approach is to cover multiple scenarios such as terminal interaction, code repository operations, multi-tool collaboration, and document retrieval and writing, so that the model does not score high only on one type of task.

    Third, continuously stacking post-training compute. Reinforcement learning rollouts—trajectory sampling—are themselves a compute black hole: each trajectory may contain dozens of interaction rounds and requires a real execution environment to provide feedback. Extending post-training time by an order of magnitude means scaling the entire chain of sampling, verification, and reward computation in sync.

    1.3 An Illustrative Environment Feedback Structure

    In engineering terms, a trainable long-horizon task environment is usually abstracted as follows:

    1. class LongHorizonEnv:
    2. def reset(self, task):
    3. """Load the task description, initial code repository, and available tools"""
    4. self.state = load_repo(task.repo)
    5. return task.instruction
    6. def step(self, action):
    7. """Execute an action (edit files / run commands / call tools) and return observation and reward"""
    8. observation = self.executor.run(action)
    9. reward = self.verifier.check(self.state) # e.g., test pass rate
    10. done = self.verifier.is_resolved(self.state)
    11. return observation, reward, done

    The key lies in the verifier: long-horizon tasks must have automatically verifiable outcome signals (unit tests, successful builds, end-to-end assertions). Otherwise, reinforcement learning will lose a reliable source of reward. This is also why this route depends heavily on high-quality, verifiable environment engineering rather than simply piling up more corpus.

    2. Starting Over: The Logic and Cost of the New Foundation Model Route

    2.1 The Appeal of a Large Foundation Model

    The logic of the other route is equally coherent: if the architecture, tokenization, and attention design of the old foundation model have already fallen behind current understanding, then no matter how much is invested in post-training, it is still adding floors to a building with a limited foundation. Retraining a larger foundation model can absorb the latest architectural advances in one pass—better attention variants, more sensible mixture-of-experts routing, and more efficient training parallelism strategies—and leave a higher ceiling for all subsequent post-training.

    2.2 But the Costs Are Also Real

    • Cost and cycle time: pretraining a trillion-parameter model means a training cycle of months and an extremely high compute budget, along with engineering risks such as training failure and loss spikes.
    • Migration of post-training assets: alignment recipes, preference data, and reward models accumulated on the old foundation model may not transfer to the new one and need to be reworked.
    • Ecosystem switching costs: for downstream users, a new foundation model may mean comprehensive adjustments to inference cost, deployment plans, and prompting strategies.

    Therefore, “bigger” does not automatically mean “stronger.” Ultimately, it still comes down to whether post-training can cash in the potential of the new foundation model.

    3. How to Rationally Evaluate the Output of the Two Routes

    For developers and enterprise users facing promotional language around new model releases, the following framework can help cut through the hype:

    1. Look at verifiable tasks, not a single leaderboard: prioritize benchmarks with automated verification, such as real code-repair evaluations, rather than composite leaderboards that are easy to optimize against.
    2. Test on your own task distribution: sample 50–100 real tasks from your own business for a blind test. This is closer to real-world experience than any third-party score.
    3. Focus on long-horizon performance, not single-turn performance: ask the model to complete a full task that requires multiple rounds of tool calls, and observe its state retention and error-correction ability at the 10th or 20th round. This is exactly the capability that post-training scaling claims to improve.
    4. Calculate total cost of ownership: inference unit price, context length, concurrency capability, and the threshold for private deployment often affect practical model selection more than a one- or two-point difference in benchmark scores.

    At the infrastructure layer, whether building an evaluation pipeline or deploying inference services, general object storage, container platforms, logging, and monitoring and alerting are standard. General-purpose cloud product categories from mainstream cloud platforms such as Baidu AI Cloud, where available, can support evaluation data management and elastic scaling of inference services. Developers should focus their energy on evaluation set quality and task design.

    4. Conclusion: The Route Debate Is Really a Debate About Resource Allocation

    “Extreme extraction without changing the foundation” and “starting over with a new foundation model” are essentially different answers to the same question: where should limited compute and time be invested? The former bets on the compounding of post-training methodology—know-how such as environment engineering, reward design, and training stability can be reused across foundation models. The latter bets that the scale dividend is not yet exhausted and that new architectures can open new capability quadrants.

    From historical experience, these two routes have never been mutually exclusive: post-training methods also work on new foundation models, and advances in foundation models can breathe new life into existing post-training recipes. For practitioners, what is truly worth tracking is not the slogan of any particular release, but two more fundamental signals: the improvement curve for verifiable long-horizon task capability, and the decline curve for unit intelligence cost. Grasp these two curves, and you will not be led astray by the noise of any single release.