LLM Upgrades: Post-Training Exhaustion vs. New Base Model Retraining
This article compares two main paths for upgrading large language models: freezing the base model and aggressively scaling reinforcement-learning post-training around verifiable environments, or retraining a new base model with more parameters. It explains why these paths have diverged, how interactive environments become post-training data, why long-horizon trajectories are the real bottleneck, how to evaluate coding and agent capabilities, and what enterprises should check before choosing a model. It also emphasizes practical metrics such as pass rate, cost, latency, safety violations, tool-call stability, and regression testing.
1. Why the upgrade paths have diverged
Over the past two years, the main way to measure whether a large language model has become stronger was fairly narrow: more parameters, more pretraining data, and longer context windows. But this curve is flattening. Growth in high-quality text corpora is not keeping up with compute expansion, and simply scaling pretraining is producing clearly diminishing marginal returns.
As a result, the industry has split into two response strategies.
Path A: Freeze the base model and scale post-training. This view assumes that the current base model has not yet exhausted its potential. Resources are concentrated on reinforcement learning, environment construction, and trajectory selection. The usual vendor narrative is: “The base model is unchanged, but capabilities improve substantially.”
Path B: Retrain the base model and expand parameters. This view assumes that post-training can shape capabilities but cannot grow them beyond the existing capacity ceiling. The model capacity must be increased first, then the full pipeline of pretraining, mid-training, and post-training must be repeated.
The key to understanding both paths is a basic judgment:
Pretraining mainly determines the lower bound of the model’s capability distribution. Post-training determines the upper limit that can be reliably unlocked within that distribution.
The former is the foundation. The latter is the deed that records which parts of the foundation can be reliably activated.
2. Path A: Treat post-training as a first-stage rocket
2.1 The environment is the data
In traditional post-training, the data unit is usually a question-answer pair. In the agent era, the data unit becomes an interactive environment: a task container that can be executed, verified, and used to produce objective feedback.
Common forms include:
- Terminal environments: Given a set of shell command capabilities, the model explores the filesystem, diagnoses failures, and builds projects.
- Code repository environments: A real repository plus unit tests and hidden tests, where the test pass rate can serve as a reward signal.
- Cross-tool collaboration environments: The model must switch between search, file reading and writing, script execution, and subtask dispatch.
- Multi-turn long-horizon environments: Tasks contain dozens or even hundreds of dependent steps, and mid-task failures are difficult to roll back.
The value of these environments is that rewards are verifiable. Whether code compiles, whether tests pass, whether a service starts, and whether data remains consistent are objective signals. Gradients can be formed without human annotation. This is the prerequisite that makes post-training scale viable.
2.2 The real scarce resource is long-horizon trajectories
Scaling up the number of environments is relatively easy. Scaling up long-horizon task trajectories is much harder.
A single-step task, such as completing one function, may produce a trajectory of only a few hundred tokens. The model mostly learns local patterns. A real engineering task chain—diagnosing a production timeout, locating a cache failure, modifying code, adding tests, and running regression tests—may span tens of thousands of tokens and include extensive exploration, trial and error, and backtracking.
Reinforcement learning faces two hard constraints when using long trajectories:
- Credit assignment is difficult. The final outcome may provide only a sparse reward. Attributing that outcome to one intermediate decision creates high variance.
- Training cost grows superlinearly with length. Longer trajectories require more sampling, storage, and backpropagation. Early errors also keep contaminating later decisions through the context.
Therefore, claims such as “increasing the number of environments by tens of times” usually do not mean simple expansion. They usually require supporting trajectory filtering and curriculum design: keep only trajectories that contain useful exploration and can ultimately be verified as successful; grade tasks by difficulty so the model moves from short-horizon tasks to long-horizon tasks.
2.3 Time itself is compute
The third lever in Path A is the least exciting: putting more post-training time and compute directly into the system.
The rationale is that reinforcement learning is far less sample-efficient than supervised fine-tuning. A single policy update may require hundreds or thousands of samples. This also explains why Path A demands strong infrastructure: many concurrent containers for environments, high-throughput storage for trajectories, and complete log retention for failed cases so that they can be reproduced.
2.4 A simplified post-training loop
Ignoring framework-specific details, the training skeleton of Path A looks roughly like this:
for step in range(total_steps):# 1. Sample tasks from the environment pool and adjust difficulty according to the curriculumtasks = env_pool.sample(batch_size, curriculum=difficulty(step))# 2. Run full trajectories with the current policy in real environments (may include tool calls)rollouts = []for task in tasks:traj = rollout(policy, task, max_turns=MAX_TURNS)reward = verify(task, traj) # compile / unit tests / end-to-end assertionsif is_degenerate(traj): # loops, idle spinning, or privilege misusereward = 0.0rollouts.append((traj, reward))# 3. Trajectory-level filtering: discard all-correct and all-wrong samples; keep middle segments with useful gradientsbatch = filter_by_advantage_variance(rollouts)# 4. Policy update with KL constraint to avoid drifting too far from the base distributionloss = policy_loss(batch) + beta * kl(policy, reference)loss.backward()optimizer.step()
This pseudocode hides three engineering points that are easy to overlook:
verifymust be reproducible and impossible for the model to fake.is_degeneratemust detect behaviors where the model “exploits loopholes” in the environment.- The KL coefficient needs to be adjusted dynamically during training. Otherwise, the model may overfit to a small number of tasks, causing general capabilities to regress.
3. Path B: A new base model and parameter expansion
Path B has its own logic. If a task requires knowledge density or compositional reasoning depth beyond the current base model’s capacity, post-training optimization alone is only shifting the model within a limited space.
Parameter expansion provides relatively certain gains:
- A higher knowledge capacity ceiling, with more stable coverage of rare frameworks, edge APIs, and domain-specific terminology.
- Greater compositional reasoning depth, making it easier to consider long dependencies across files and modules at the same time.
- Better instruction following and format stability, especially at larger scales, which matters for tool-calling protocols.
The costs are also clear. Pretraining is a one-time, large investment. It takes months and cannot be effectively stopped mid-way. Unit inference cost is higher, and memory and latency pressure become more obvious in long-context scenarios.
4. Comparing the two paths
| Dimension | Scaling post-training (Path A) | Retraining a new base model (Path B) |
|---|---|---|
| Iteration cycle | Short; weekly releases are feasible | Long; months as a baseline |
| Investment per iteration | Moderate; can be added in batches | High; not readily reversible |
| Capability ceiling | Constrained by base model capacity | Higher ceiling |
| Main risk | General capability regression; reward hacking | Imbalanced pretraining data mixture |
| Diminishing returns | Returns drop quickly once the base model is exhausted | Also limited once the corpus dividend is exhausted |
| Best-fit scenarios | Coding, agents, tool use | Knowledge-intensive tasks, complex reasoning |
One conclusion deserves emphasis: these two paths are not an either-or philosophical choice. They are two sequential events on the same roadmap.
When a base model is first released, post-training is far from saturated, and Path A usually offers much better cost efficiency. As environments are reused repeatedly and advantage samples are exhausted, marginal returns decline quickly. At that point, retraining the base model becomes necessary to reset the ceiling.
5. How to judge whether coding experience has truly improved
Claims of “better coding experience” are hard to trust directly. However, they can be broken down into verifiable metrics through evaluation methods.
5.1 Why terminal and CLI scenarios have become the main battlefield
In real development workflows, models increasingly work through command lines, filesystems, and test frameworks rather than only outputting code text. Evaluation in these scenarios is naturally objective: run the task and check whether it passes.
When designing such evaluation sets, pay attention to:
- Tasks must be automatically verifiable. Use test assertions rather than manual scoring.
- Long-horizon dependencies must be included. For example, changing component A may cause tests for component B to fail.
- Failure and rollback must be allowed. Otherwise, the model’s ability to correct mistakes cannot be measured.
- Context contamination must be controlled. Avoid mixing evaluation set content into training data.
5.2 Pass rate alone is not enough
At minimum, evaluation should cross three or more dimensions:
eval_matrix:- name: functional_correctness # hidden test pass rateweight: 0.5- name: cost_per_solved_task # average token consumption per solved taskweight: 0.25- name: latency_p95 # end-to-end time, including tool-call waitingweight: 0.15- name: safety_violations # number of unauthorized actions or dangerous command executionsweight: 0.10
Many models improve pass rate while consuming twice as many tokens. In practice, this can make engineering costs higher. The ratio of pass rate divided by average tokens often reflects usability better than accuracy alone.
5.3 Regression testing cannot be omitted
The biggest hidden risk in Path A is capability trade-offs: coding may become stronger, but multi-turn instruction following, structured output, and reasoning in non-code domains may regress.
A stable approach is to maintain a regression suite covering general capabilities. Every version update should run the full suite, so regressions are caught before release.
6. An enterprise selection checklist
Setting aside the debate over paths, actual model selection can converge in the following order:
Run small-scale A/B tests using your own tasks. Select 30–50 real internal tasks rather than public evaluation sets. Let candidate models execute them in a unified container sandbox. Compare only pass rate and cost per task.
Prioritize tool-calling protocol stability. Parameter format errors, instruction drift in very long contexts, and consistency across multiple turns are often more damaging than raw “intelligence” in real pipelines.
Treat context length as a budget, not a selling point. A million-token window means higher per-request overhead. Retrieval and segmented summarization may be needed to compress the effective context.
Pay attention to engineering support, not just the model itself. Sandbox isolation, concurrent scheduling, log retention, and trajectory replay are usually implemented through general cloud capabilities such as container platforms, object storage, message queues, logging services, and monitoring/alerting. The model is only one part of the system.
Clarify the operational cost of open-weight models. Self-hosting requires elastic scaling of inference clusters, staged rollout, and effectiveness monitoring. A managed API shifts some of this responsibility to the provider, but it also involves trade-offs in controllability and data boundaries.
7. Common misconceptions
Misconception 1: If the base model has not changed, the upgrade is meaningless.
The opposite is true. If post-training can reliably unlock existing capabilities, it suggests that the model’s previous intelligence ceiling was systematically wasted.Misconception 2: Larger parameter counts are always better.
Parameter scale determines the theoretical ceiling. Practical usability depends on inference cost, latency, and call stability.Misconception 3: Public leaderboard rankings can be directly converted into productivity.
Leaderboard task distributions often differ significantly from real engineering task distributions, especially between hidden test sets and internal repositories.Misconception 4: Post-training can be squeezed infinitely.
When the advantage variance of verifiable tasks in the training set approaches zero, reinforcement learning has no useful gradient left. This is the natural endpoint of Path A.
8. Conclusion
“Keep the base model and exhaust post-training” and “start over and expand parameters” are essentially two investment modes for the same goal at different stages.
For developers and enterprises, the right posture is not to pick a side. The right posture is to take evaluation back into your own hands. Use your own tasks, a unified sandbox, and reproducible verification scripts to measure every upgrade. Track four curves: pass rate, cost, latency, and safety violations.
Vendors decide the model roadmap. But which version truly fits your system can only be answered by your own evaluation matrix.
