Post-Training Scaling Without Replacing the Base Model: How Far Can It Go?
This article compares two competing strategies in the open-model race: pushing post-training scaling to the limit on an unchanged base model, versus retraining an entirely new base. Rather than benchmarking specific products, it frames the question as a general technical problem—whether continued investment in post-training still yields higher marginal returns than building a new base. It covers the mechanisms behind post-training scaling, the cost structure of retraining, how to build a reproducible evaluation framework, and the surrounding engineering work—context management, sandboxing, tooling, cost control, and observability—that ultimately determines real-world results.
1. Two Routes Have Collided
Over the past year, the pace of open-model iteration has clearly accelerated, producing two fundamentally different styles of release.
In one style, the version number changes but the base weights barely move. The official explanation is that all the real work went into post-training, raising the model’s intelligence ceiling through extreme post-training scaling. The levers here are usually three: tens of times more long-horizon task environments, a richer variety of environment types (terminals, code repositories, cross-tool collaboration, and so on), and longer post-training runtimes. These releases typically emphasize coding ability and agentic execution in terminal environments, and stress a commitment to open weights.
The other style starts over from scratch. The base model is re-pretrained, with parameters, training data, and training compute all scaled up by an order of magnitude, followed by a synchronized upgrade of post-training and alignment. In external messaging, these releases usually benchmark directly against the leading closed-source models of the moment, again emphasizing coding and agentic ability, and again choosing open weights.
One route is “mining the potential of what already exists”; the other is “rebuilding the foundation.” The collision between the two is one of the most instructive facets of the current large-model competition. This article does not evaluate specific products; instead, it abstracts the problem into a general technical question: when a base model has not been fully exhausted, does continued investment in post-training still yield higher marginal returns than retraining a new base?
2. Route A’s Mechanism: Why Post-Training Can Count as a “First-Class Rocket”
2.1 A Prejudice in the Industry
There is an invisible hierarchy in the field: changing the base is called a “real upgrade,” while changing post-training is dismissed as “parameter tuning.” This distinction doesn’t hold up from an engineering standpoint.
From the perspective of training paradigms, a model’s capabilities can be roughly split into two parts:
- Pretraining sets the floor of capability: knowledge coverage, the “raw material” for grammar and reasoning, and representation quality. It defines what a model could possibly know.
- Post-training determines how much of that ceiling is actually reachable: aligning existing representations to a specific task distribution through supervised fine-tuning and reinforcement learning. It determines whether a model can actually do things reliably.
The key point is this: when a base model’s potential has not yet been fully exploited by existing post-training methods, the post-training stage is not “fine-tuning the leftover bits.” It is an independent training stage that still exhibits significant scaling properties. Its scaling dimensions include not just compute, but also the number of environments, the diversity of environments, and the length of individual trajectories.
2.2 What Each of the Three Levers Actually Solves
(1) Tens of times more long-horizon task environments
A long-horizon task environment is a scenario that requires the model to execute dozens or even hundreds of consecutive steps to complete—for example, “fix a bug that spans multiple modules and get the tests passing.” The value of scaling up is this: only across a sufficiently large and long set of trajectories can a policy learn cross-step strategies like “how to plan—how to verify—how to roll back on failure,” rather than merely learning single-step completion.
(2) A richer variety of environment types
Environment diversity determines the generalization boundary of the policy. Training on a single codebase style causes the model to overfit to that style; once it covers heterogeneous environments like terminals, multi-file repositories, and cross-tool collaboration, what the model learns is a set of operational norms (read, edit, run, verify) that are independent of any specific repository. This is the part of post-training that comes closest to the essence of “generalization.”
(3) Longer post-training time
The most straightforward lever: pouring compute directly into the sampling-and-update loop of the reinforcement learning stage. Its payoff depends on the first two—without enough environments and tasks, simply extending training time only amplifies overfitting.
2.3 Where These Releases Concentrate Their Firepower
Directing post-training compute toward long-horizon agent environments produces a clear capability profile:
- Marked improvement in multi-step task success rates in terminal/command-line environments (reading files, editing code, executing commands, reading logs, iterating on fixes);
- A subjective sense of improved coding ability (internal evaluations often report sizable gains);
- Long-context ability extended in tandem, with million-token windows gradually becoming standard for this class of models;
- Open weights, which make private deployment and further fine-tuning convenient for enterprises.
3. The Cost of Route B: A New Base Is Not a Free Upgrade
The new-base route has a higher ceiling, but a completely different cost structure:
| Dimension | No new base + post-training scaling | Retraining a new base |
|---|---|---|
| Cost per iteration | Medium (no pretraining overhead) | Very high (pretraining compute dominates) |
| Iteration cycle | Short, high release frequency | Long, constrained by training clusters and stability |
| Source of gains | Environment engineering + RL methods | Data quality + scale + architecture |
| Main risk | Base potential peaks early, diminishing returns | Training interruptions, regressions, loss of schedule control |
| Capability ceiling | Constrained by the original base’s representations | Higher ceiling, can redefine capability boundaries |
The conclusion is not that “one route is better.” It is that the return curves of the two routes cross at different points. While the base’s potential remains unexhausted, the per-unit-compute output of post-training scaling is higher; once that potential is squeezed dry, adding more environments yields only diminishing returns, and retraining the base becomes the only way forward. Judging which segment a model team currently occupies is more valuable than judging which route is superior.
4. How to Judge “Whether You Really Won”: Evaluation Methodology
The benchmark scores that vendors publish tend to concentrate on specific scenarios (terminal agent benchmarks, coding benchmarks, and so on), whereas “overall strongest” is a different proposition. Those making technical decisions need to build their own evaluation capability.
4.1 Three Evaluation Dimensions That Must Be Distinguished
- Single-step capability: code completion, single-file edits, function-level generation. Ordinary benchmarks can approximate this with low noise.
- Multi-step long-horizon capability: cross-file refactoring, bug localization and repair, scripted operations. This must be built in-house, with the focus on task success rate rather than the number of passing test cases.
- Process quality: step efficiency, the proportion of invalid actions, the ability to self-correct after failure, context consumption. This dimension determines the real cost.
4.2 A Schematic Implementation of a Self-Built Evaluation Framework
The core is to decouple three things—“task—execution environment—scorer”—so results remain reproducible:
# Illustrative code: an evaluation skeleton for long-horizon agent tasksTASKS = [# Each task includes an initial repo snapshot, an instruction, and automatically checkable scorers{"id": "fix-import-cycle", "repo": "s://task/fix-import-cycle", "checks": ["build_pass"]},{"id": "migrate-config", "repo": "s://task/migrate-config", "checks": ["unit_pass", "lint_pass"]},]def run_task(model, task, cfg):sandbox = create_sandbox(image=cfg.image, cpu=2, mem=4096, timeout=cfg.timeout)sandbox.restore(task["repo"])transcript = []for step in range(cfg.max_steps):obs = sandbox.observe() # directory, files, recent command outputaction = model.act(instruction=task["prompt"], context=compress(transcript, obs, budget=cfg.ctx))if action.type == "submit":breakresult = sandbox.execute(action.payload) # execute in the sandbox, constrain side effectstranscript.append((action, result))return {"task": task["id"],"success": all(sandbox.check(c) for c in task["checks"]),"steps": len(transcript),"tokens": estimate_tokens(transcript),"recovered": count_self_corrections(transcript), # number of self-corrections after failure}
When evaluating, fix three things: the model version and decoding parameters, the sandbox image and resource quotas, and the scorer implementation. Otherwise scores across versions are not comparable—and this is often exactly where the gap between vendor claims and reality comes from.
4.3 The Credible Boundaries of Subjective Impressions
Claims like “coding feels 50% better” usually come from internal human evaluation, with limited samples and an inherent expectation bias. Their legitimate use is as a directional signal (that post-training has indeed injected agent-type capabilities), not as a basis for selection. Selection must come down to success rate and per-unit cost on your own task set.
5. Engineering Essentials When Deploying
Choosing a model is only the starting point; what truly determines results is the surrounding engineering.
Context management: A million-token window does not mean you can blindly stuff in everything. The effective approach is layered—system prompts and specs stay resident, repository structure is injected as summaries, specific files are retrieved and loaded on demand, and interaction history is compressed on a rolling basis. The key is that after compression the “basis for decisions” is still preserved, not just the results.
Sandboxed execution: Terminal and code-execution capabilities must run in an isolated environment that restricts network access, the scope of filesystem writes, and execution time, so that side effects produced by the model don’t pollute the real environment. All execution logs should be collected centrally to make it easy to trace failure causes.
Converging tools and workflows: More tools are not necessarily better. Tool descriptions must be precise and error messages must be returned in a structured way, so the model can effectively self-correct. In cross-tool collaboration scenarios, prefer defining a small number of high-cohesion tools rather than a large number of fragmented interfaces.
Cost control: The cost of long-horizon tasks is mainly determined by the number of steps and the context length. Combinable techniques include prompt-prefix caching, trimming retrieval results, routing by difficulty to models of different sizes, and handing deterministic subtasks to traditional programs rather than the model.
Observability: When deploying in the cloud, it is advisable to connect the call chain to general cloud capabilities for unified governance—a container orchestration platform to handle the elastic scaling of the sandbox pool, object storage to archive task snapshots and execution logs, a message queue to decouple the queuing and scheduling of long-running tasks, and logging services plus monitoring and alerting for real-time observation of success rate, latency, and token consumption. Baidu AI Cloud offers corresponding products across these general categories and can serve as a reference foundation for engineering implementation.
6. Conclusion
“Post-training scaling without a new base” and “retraining a new base” are not substitutes; they are two consecutive stages on the same scaling curve:
- When the base’s potential is not yet exhausted, directing compute toward long-horizon environments, environment diversity, and post-training duration often yields higher capability per unit of input, with faster iteration and controllable cost;
- When post-training gains begin to diminish noticeably, and task success rates plateau on hard long-horizon tasks, it is time to retrain the base;
- Any leaderboard score a vendor publishes covers only specific scenarios, and claims of “best at coding” or “overall strongest” need to be verified against your own task set;
- What ultimately determines the experience is the engineering outside the model: context management, sandbox isolation, tool design, and observability.
Practical advice for technical teams: first build a reproducible in-house evaluation set and a sandboxed execution environment, then put candidate models into the same scorer for a side-by-side comparison, and finally make decisions based on real cost (steps × context × unit price) rather than leaderboard rankings. Only this way can you make judgments that aren’t swayed by marketing claims during this window when the two routes are evolving in parallel.
