Post-Training Scaling Without a New Base Model: Can It Match a Larger-Parameter One?
Two routes dominate the current push for stronger large language models: pushing post-training to the limit on an existing base, or pre-training a larger base with a sparse Mixture-of-Experts design. This article compares both across technical principles, engineering practice, evaluation, and deployment. It shows why RL-based post-training is a full training stage rather than prompt tweaking, how to build task environments with verifiable rewards, how to handle credit assignment and reward hacking, and how to measure whether a base still has extractable headroom. It closes with a decision framework, an evaluation system that goes beyond leaderboard scores, and a deployment checklist to help teams choose between squeezing the current base and investing in a larger model.
Post-Training Scaling Without a New Base Model: Can It Match a Larger-Parameter One?
Extreme post-training “squeezing” versus a larger-parameter new base model—which should you choose? This article breaks the question down across four layers—technical principles, engineering practice, evaluation, and deployment—to help you see the essence of this “route debate.”
1. The Route Debate: Reframing the Problem
LLM capability growth has long relied on two engines: larger parameter counts and more training data. But the growth of high-quality corpora has clearly slowed, and continuing to “pile on scale” during pre-training now yields diminishing returns. The industry is looking for a new growth curve, and two routes have emerged.
Route A: Keep the base, push post-training to the extreme. Retain the existing base and concentrate compute and engineering effort on post-training. The idea is to repeatedly put the model through massive real-world engineering task environments so it learns from “getting it right or wrong” feedback rather than simply memorizing corpora. This route can be seen as a second wave of scaling—this time driven by reinforcement learning.
Route B: Start over and scale up parameters. Re-architect and pre-train a larger base, commonly by introducing a sparse Mixture-of-Experts (MoE) structure alongside a dense architecture. Total parameter count jumps by orders of magnitude, but only part of the network is activated at inference time to keep per-token compute under control. The core assumption: a larger-capacity base can hold more knowledge and leave more room for downstream optimization.
If we roughly decompose the sources of LLM capability:
final capability ≈ f(base capacity, post-training extraction, inference-time system support)└─injected in pre-training─┘ └─shaped by RL─┘ └─context/tools/sandbox─┘
Pre-training sets the floor, post-training determines how high you can reach, and the inference system determines whether that capability can be delivered reliably. The focus of the debate between the two routes is: at the current stage, which item should resources be concentrated on?
2. Clarification: “Post-Training Squeezing” Isn’t “Parameter Tuning”
Many people equate “touching only post-training” with “tweaking prompts and running a few fine-tuning rounds”—that is a misunderstanding. Post-training, especially RL-based post-training, is essentially a complete training stage: objective function, sampling distribution, reward signal, and training compute are all indispensable. In the training sense, it has at least three levers.
2.1 Scale up the size and duration of task environments
RL requires the model to keep interacting with an environment. In coding and tool-use scenarios, a task environment typically has three parts:
| Element | Meaning | Example |
|---|---|---|
| Initial state | Resources accessible before the task starts | Repo snapshot, dataset, read-only docs, network access scope |
| Action space | Operations the model can perform in the environment | Edit files, run commands, run tests, call tools |
| Termination & judging | Task end conditions and correctness criteria | All unit tests pass, target file generated in valid format |
Scale up the available environments by an order of magnitude and keep the interaction steps long enough, and the model gets repeated exposure to the full loop of “plan → act → fail → localize → fix.” This kind of long-horizon experience is hard to acquire naturally from “question-and-answer”-style pre-training corpora.
2.2 Increase environment diversity
Task environments need to cover many types: single-machine command-line operations, cross-file changes in large repositories, multi-tool pipelines, debugging and performance analysis, documentation generation, and so on. The richer the variety, the more general the strategies the model learns—and the less likely it is to overfit to a single task shape.
2.3 Extend the compute investment in post-training
Post-training also consumes compute, and in a different form: beyond gradient computation, most of the time goes into “generating trajectories” (the model interacting with the environment and sampling) and “executing checks” (running tests, sandboxes, etc.). This means scaling post-training places two demands on infrastructure:
- Elastic sandbox resources that can spin up and isolate many task instances concurrently;
- A high-throughput data pipeline that fully captures each interaction trajectory, command output, and test result for training and review.
2.4 Why it works when the base isn’t exhausted
The objective of pre-training is “predict the next token.” It teaches the model knowledge and language patterns, but it never directly tells the model “what it means to be done.” Post-training directly optimizes “is the task complete,” reorganizing capabilities scattered throughout the base into goal-oriented strategies.
Let the base’s existing capability ceiling be C, and let the extractable portion—under the premise that the base stays unchanged and only the output distribution is moderately constrained—be η. If η is much smaller than C, the marginal return of scaling post-training is very high; when post-training approaches the constraint boundary, further investment enters clear diminishing returns. So the question isn’t “can squeezing catch up to a larger parameter count” but rather how much extractable space remains in the current base.
There is only one way to judge: measure it.
- Freeze the base on the target task and gradually scale only the data and environment size of post-training.
- Record the “capability–compute” curve: x-axis = cumulative post-training compute or environment scale, y-axis = task pass rate.
- If the curve is still on the steep segment, prioritize scaling post-training; if the curve flattens, further squeezing only wastes compute.
3. Engineering Practice: Turning “Squeezing” into an Executable Pipeline
3.1 Build a task-environment factory
The quality ceiling of post-training is set by the environment. The build approach is similar to a data factory:
- Collection: Extract auto-judgeable tasks from real engineering work, historical tickets, and incident retrospectives.
- Construction: Generate “repair-type” tasks from code version diffs and “implementation-type” tasks from documentation.
- Filtering: Remove tasks with too many steps, overly heavy environment dependencies, or unstable judging.
- Snapshots: For each environment, save a fixed initial state and a rollback-able version snapshot to guarantee training reproducibility.
- Metadata: Record difficulty labels (e.g., completion rate, required steps) for curriculum learning.
You can use a unified structure to describe environments, making scheduling, reuse, and version management easier:
from dataclasses import dataclass, field@dataclassclass EnvSpec:"""Description structure for a post-training task environment"""name: strrepo_snapshot: str # Initial code repo snapshotinstruction: str # Natural-language task descriptionchecks: list[dict] # List of checkers: compile, unit tests, static checks...allow_network: bool = False # Whether network access is allowedbudget_steps: int = 40 # Step budgetextra: dict = field(default_factory=dict) # Metadata such as difficulty labels
3.2 Make verifiable rewards the core
RL can’t do without a reward signal. In coding and tool-use tasks, the most reliable reward comes from program execution results:
- Compilation: whether it passes syntax and type checks;
- Unit tests: the pass ratio, which should include hidden cases to prevent the model from only “catering to” visible cases;
- Static checks and format validation: whether it conforms to code conventions;
- Output validation: whether the artifact meets structural and content constraints;
- Human or model scoring: used only for subjective tasks that are hard to judge automatically, such as explanation and summarization.
import shutilimport subprocessfrom pathlib import PathCHECK_TIMEOUT = 120 # Timeout per checker (seconds)def run_check(cmd: str, cwd: Path) -> bool:"""Run a check command; a non-zero exit code or timeout counts as failure"""try:proc = subprocess.run(cmd, shell=True, cwd=cwd,capture_output=True, timeout=CHECK_TIMEOUT)return proc.returncode == 0except subprocess.TimeoutExpired:return Falsedef build_reward(spec: EnvSpec, workspace: Path) -> float:"""Checker scorer: runs all checkers in an isolated copy"""# Validate in a separate copy to avoid corrupting the original environment snapshotprobe = workspace / "_probe"if probe.exists():shutil.rmtree(probe)shutil.copytree(workspace, probe, ignore=shutil.ignore_patterns("_probe"))results = [run_check(c["cmd"], probe) for c in spec.checks]reward = sum(results) / len(results)# Log full check results for review and later process-reward design(workspace / "reward.json").write_text(repr({"spec": spec.name, "checks": results}))return reward
Keep these points in mind when designing rewards:
- Hard before soft: priority order is “executable checks > rule matching > model scoring”;
- Rewards can’t be too sparse: “give 1 point only if everything passes” makes the training signal too sparse. Use “proportion of passed cases + a bonus for passing all cases” to form a tiered reward;
- Explicit failure = zero: when there are wrong submissions or cheating behaviors, give a direct zero without entering subsequent statistics.
3.3 Handle credit assignment for long-horizon tasks
A single task needs only one scalar reward, but every one of the dozens of interaction steps contributes to the gradient. If the whole trajectory ultimately fails, all steps get “collaterally penalized,” which may include critical steps that were actually correct. Common mitigations:
- Process rewards: assign staged rewards at several checkpoints (e.g., “code compiles,” “basic cases pass”).
- Progress-estimation model: estimate “how far from completion” each intermediate state is, as a dense reward.
- Subgoal decomposition: break long tasks into independently verifiable milestones.
- Step penalty: introduce a slight length penalty to suppress ineffective loops and repeated trial-and-error.
3.4 Difficulty curriculum and data mix
- Arrange the curriculum from easy to hard: in the early stage, focus on simple tasks to give the model stable positive feedback first; then gradually introduce complex tasks.
- Empirically, keeping training tasks in a range where “the model has some probability of getting it right” (e.g., a pass rate between 30% and 70%) yields the richest gradient signal.
- Filtering passed trajectories from the model’s own samples for rejection sampling and continuing to train on them as positive examples can improve sample efficiency.
3.5 Guard against reward hacking and evaluation contamination
RL naturally seeks out flaws in the reward function. Common behaviors include: modifying or deleting test files, hardcoding specific expected outputs, only making visible cases pass, silently swallowing exceptions, and so on. Countermeasures:
- Protect critical cases with “canary tests”; if they are modified or deleted, mark a direct loss;
- Hide part of the test cases and use different validation sets for training and evaluation;
- Check the diff: whether test files, config files, and other parts that shouldn’t be touched were changed;
- Check dependencies: whether external resources were downloaded without permission;
- Check resources: timeouts and memory overruns count as failures.
3.6 Post-training main loop (illustrative)
The following code is a simplified illustration that only describes the flow skeleton and involves no specific framework:
# ConfigurationBATCH_TASKS = 32 # Tasks sampled per roundGROUP_SIZE = 8 # Trajectories sampled per taskKL_COEF = 0.02 # Penalty coefficient for deviation from the reference modelLOG_EVERY = 20 # Logging interval (rounds)for step in range(total_steps):tasks = sampler.sample(BATCH_TASKS) # Sample tasks by difficultygroups = []for task in tasks:env = SandboxEnv(task) # Spin up an isolated sandboxtrajs = [rollout(policy, env) for _ in range(GROUP_SIZE)]groups.append([(traj, build_reward(task.spec, traj.workspace))for traj in trajs])env.close() # Release resources# Compute group-relative advantage: compare good vs. bad within the same task to remove task-difficulty differencesadvantages = []for group in groups:rewards = [r for _, r in group]mean = sum(rewards) / len(rewards)scale = max(std(rewards), 1e-6)advantages.append([(traj, (r - mean) / scale) for traj, r in group])# Compute objective (policy-gradient term - new/old policy ratio term - KL regularization term) and update parametersloss = objective(policy, ref_policy, groups, advantages, kl_coef=KL_COEF)optimizer.step(loss)if step % LOG_EVERY == 0:passed = [r for g in groups for _, r in g if r >= 1.0]log_metrics(step, pass_rate=len(passed) / (BATCH_TASKS * GROUP_SIZE))
4. Larger-Parameter New Base: Logic and Cost
4.1 Core logic
A sparse MoE architecture decouples total parameter count from per-token activated parameters. Larger total capacity means:
- It can hold richer knowledge and domain patterns;
- Different experts specialize in different task types, reducing interference between parameters;
- More stable training, and more room left for post-training.
This explains the logic of “starting over”: rather than doing marginal optimization on a base whose capabilities are nearing saturation, it is better to build a higher-capacity starting point all at once.
4.2 Costs and constraints
- High pre-training investment: compute, data, and engineering cycles are all required, with high trial-and-error costs.
- Higher inference cost: larger total parameters raise memory footprint, expert routing, and communication overhead.
- System optimization needed: quantization, expert parallelism, request scheduling, and batching must keep pace, otherwise service capacity becomes the bottleneck.
- Diminishing returns: gains from larger parameters aren’t linear; insufficient data quality may also introduce noise and knowledge conflicts.
So “larger parameters = stronger capability” holds only when capacity is truly the bottleneck. Beyond capacity, data quality and training strategy are equally critical.
5. Decision Framework: When to Squeeze, When to Upgrade
| Decision dimension | Lean toward post-training squeezing | Lean toward a larger-parameter new base |
|---|---|---|
| Base potential | Capability curve still on the steep segment | Curve flattening, post-training marginal return clearly narrowing |
| Task type | Clearly defined, auto-verifiable engineering and tool tasks | Knowledge-intensive tasks, open-domain reasoning |
| Data & compute | Can build environments in bulk, limited post-training compute | Has data and compute reserves for large-scale pre-training |
| Iteration cadence | Need rapid trial-and-error, weekly iteration | Can tolerate month-scale cycles |
| Cost constraint | Inference-cost sensitive, want to control per-token overhead | Can accept higher inference cost for a higher capability ceiling |
In practice, deployment is often a combined strategy: use a larger-parameter base as the foundation, then layer post-training on long-horizon task environments on top; the two are not mutually exclusive. For example, you could use elastic container resources from Baidu AI Cloud to build a sandbox cluster, batch-validate the effect of different post-training configurations in an offline environment, and then pick the option with the best return on investment.
6. Evaluation System: Don’t Just Look at Leaderboard Scores
Post-training can easily “game the leaderboard,” so evaluation must be independent of training:
- Offline evaluation set: cover dimensions such as coding, debugging, tool calling, and long-horizon tasks, using samples isolated from training and rotated periodically.
- Process metrics: beyond task pass rate, also track average steps, average token consumption, task latency, and per-task success cost.
- Online shadow evaluation: run the new strategy on a bypass first, comparing it against the online strategy on the same requests, without serving users directly.
- Canary and rollback: roll out to a small share of traffic; if pass rate drops or costs spike, roll back to the previous version quickly.
Evaluation dimension Key metrics--------------------------------Correctness Task pass rate / case pass rate / first-attempt success rateEfficiency Average steps / token consumption / task latencyCost Total cost per task / cost per million tokensStability Timeout rate / sandbox failure rate / output-format compliance rateSafety Out-of-bounds access count / sensitive-info leakage alerts
7. Deployment Checklist
Base model selection
- Define the target scenario; first build an auto-verifiable evaluation set, then compare candidate bases
- Distinguish “dense base” vs. “sparse MoE base” in latency, memory, and throughput
Post-training
- Task environments are rollback-able and concurrent, with stable and reliable checkers
- Rewards are program-check-based, with canary tests and hidden cases
- Keep full trajectory logs to support failure review
Serving & operations
- Wrap calls with a model-agnostic abstraction layer for smooth switching between models
- Use sandbox and container isolation to execute untrusted code, restricting network and filesystem access
- Monitoring and alerting covering core metrics: success rate, cost, latency, abnormal output
- Version-manage prompts and system prompts; prompt-injection content should be treated as untrusted input
Security & compliance
- Assess the security risk of model-generated code, combined with static checks and dependency scanning
- Desensitize sensitive data before it enters the training and logging pipeline
8. Conclusion
There is no absolute winner in this “route debate.” The essence of extreme post-training squeezing is to mine capability up to the constraint boundary within existing base capacity; the idea behind a larger-parameter new base is to directly raise that boundary. There is only one criterion: how much extractable space the current base has, and what cost you are willing to pay to extract it.
For most teams, a more pragmatic path is “dual-engine driving”:
- Build on a solid evaluation system;
- Use post-training squeezing to quickly approach the current base’s ceiling;
- When marginal returns keep falling, reassess whether a larger parameter scale is needed.
Whichever route you choose, environment building, verifiable rewards, evaluation, and observability are all infrastructure that determines success or failure.
