How to Improve LLM Capabilities Without Changing the Base Model: Engineering Practices for Post-Training
This article examines how large-scale post-training and reinforcement learning can deliver substantial capability gains without increasing model size, inference cost, or context-window limits. It explains why MoE architectures are well suited to targeted refinement, how SFT, preference alignment, and verifiable reinforcement learning serve different purposes, how to measure improvements in task success, safety, context use, and cost, and when teams should prioritize post-training over replacing the base model.
1. Reframing the Question: Can a Model Improve Without Growing?
The industry widely follows the Scaling Law: model capabilities improve predictably as parameters, data, and compute increase. This principle shaped the development strategies of earlier generations, making “use a larger base model and add more parameters” a standard approach to version upgrades.
However, scaling alone does not answer a more practical engineering question:
When the base model, parameter count, and inference cost remain unchanged, can a model still achieve a major capability improvement?
A recent model upgrade provides a useful case study. It retained the existing MoE architecture with 744B total parameters and 40B active parameters. It did not expand the context window or change the model size. Instead, it applied large-scale, targeted post-training and reinforcement learning to address weaknesses in engineering tasks, safety, and long-horizon workflows.
The reported results included a significant reduction in the number of tokens required to complete tasks of comparable difficulty, along with a substantial improvement in overall user experience. The model was also expected to be open-sourced after a two-week safety review, supporting research, private deployment, and commercial use cases.
This article does not focus on the detailed specifications of any particular product. Instead, it treats this upgrade as a methodological example and explores four questions:
- Does meaningful upgrade headroom exist when the base model remains unchanged?
- Is large-scale reinforcement-learning post-training practical at the current stage?
- Can post-training capability gains be measured objectively?
- How should teams proceed when capability, cost, and deployment constraints all matter?
2. Why the Approach Is Reasonable: The Base Model May Already Support Targeted Refinement
2.1 The Relationship Between Parameter Scale and Capability Is Showing Diminishing Returns
Over the past several years, the industry has primarily improved model capabilities in two ways: increasing parameter count and extending the pretraining corpus.
The first approach is constrained by memory and compute. The second is constrained by the availability of high-quality data. Once models move from the billion-parameter range into the hundreds-of-billions range, the capability gained from each additional parameter begins to decline. Training costs rise nonlinearly, while inference latency and memory consumption also increase. Meanwhile, task completion increasingly depends on the quality of the model’s final behavior:
- Does it follow formatting requirements?
- Does it retain critical facts from the context?
- Does it produce code that actually compiles?
- Does it complete a multi-step task reliably?
In other words, the base model provides knowledge and representation capacity, while production engineering requires behavioral capability. The latter is closely related to strategy and alignment, which are the primary strengths of post-training.
2.2 Sparse MoE Activation Makes It Possible to Add Training Without Increasing Inference Cost
The 744B total-parameter and 40B active-parameter configuration is representative of a practical MoE design. Its engineering significance is that the model contains a large parameter capacity, while only about 5% of the parameters are activated during each forward pass.
This produces two important effects.
First, it creates room for extensive training. Total parameter count determines knowledge capacity, while active parameters determine the computation required for an individual pass. Large-scale reinforcement-learning post-training can reorganize existing knowledge into more reliable task behavior without changing the model’s underlying architecture.
Second, inference costs remain largely controlled. Because the active parameter count stays around 40B, the capability gains from post-training do not increase per-request FLOPs or peak memory in the same way that replacing the base model might.
For teams operating private deployments, this can mean that the model version changes while the existing hardware remains sufficient.
2.3 A Million-Token Context Window Turns Engineering Context into Usable Input
The model discussed in the source material continues to support a context window of one million tokens. This makes it possible to provide an entire codebase, a complete set of technical documents, or large volumes of logs without splitting them into disconnected fragments.
The real value of long context is not simply the amount of text that can be inserted. It is whether the model can access the complete task context.
Typical use cases include:
- Full-repository understanding: Provide the source tree, module dependencies, and interface definitions together, then ask which callers may be affected by an API change.
- Long-document review: Contracts, specifications, and audit reports often require cross-section comparisons. Summarizing each section independently can destroy references and dependencies.
- Log and trace analysis: Give the model a continuous stream of logs for anomaly clustering without breaking the temporal context.
Long context is not free, however. It improves information completeness, not necessarily cost efficiency. Longer inputs increase prefill computation and first-token latency. A practical strategy is to reserve long contexts for tasks that genuinely require complete information, while using retrieval or summarization for optional content.
3. Why Reinforcement-Learning Post-Training Is a Controllable Source of Capability Gains
3.1 The Different Roles of SFT, Preference Alignment, and RL
Modern post-training pipelines generally contain three layers with distinct responsibilities.
| Stage | Objective | Data format | Typical cost |
|---|---|---|---|
| SFT (supervised fine-tuning) | Teach the model what a correct output looks like | Human- or model-generated instruction-response pairs | Low; thousands to tens of thousands of examples may be sufficient |
| Preference alignment, such as RLHF or DPO | Make outputs better match human preferences | Pairwise comparison data | Medium; strongly dependent on annotation quality |
| Verifiable reinforcement learning (RLVR) | Improve capabilities using automatically checkable rewards | Test cases, compilation results, unit-test pass rates | High; requires large-scale sampling and reward design |
The key difference is whether the reward signal can be verified.
SFT teaches the model to imitate answers that appear correct. Preference alignment relies on human judgments and is difficult to scale to objective properties such as code correctness. RLVR can directly use signals such as whether code compiles, whether unit tests pass, or whether an output follows a required format.
These signals do not require manual labeling for every example and can provide dense training feedback. As a result, they are particularly effective for large-scale post-training.
3.2 Why Reinforcement Learning Works Well for Engineering and Safety Tasks
Engineering capability has a useful property: correctness is often testable.
A generated program can be compiled and executed. A refactoring can be evaluated through regression tests. These environments can produce rewards directly, making them naturally suitable for reinforcement learning.
Security tasks are different. They often involve adversarial reasoning. Given suspicious code or an untrusted input, the model must identify an injection point, assess exploitability, and propose a repair.
This capability is difficult to obtain through memorization alone. It can instead be developed through large volumes of attack-and-defense examples combined with verifiable graders. This helps explain why security capabilities may appear to “emerge”: after sufficient training on security tasks, the model can move rapidly from producing mostly unusable answers to generating practical conclusions.
3.3 What a Reduction in Token Usage Actually Means
The source material reports that the upgraded model uses significantly fewer tokens to complete tasks of similar difficulty. This matters because token usage directly affects operating cost.
A reduction in tokens usually comes from three sources:
- Fewer ineffective reasoning steps: The model explores less, avoids repeated trial-and-error, and produces less irrelevant preamble.
- Fewer retries: The model is more likely to generate executable code on the first attempt, reducing edit-and-run cycles.
- Less wasted context: The model avoids repeatedly including irrelevant files and generating redundant explanations.
Together, these changes reduce costs in token-based billing environments. For teams operating their own inference clusters, they can also increase throughput and reduce queueing time.
4. Measuring Post-Training Improvements Objectively
Capability improvements should not be evaluated through impressions such as “the model feels better.” Teams need measurable indicators to determine whether an upgrade is worthwhile.
A useful evaluation framework covers three dimensions.
4.1 Task-Level Evaluation: End-to-End Success Rate
The goal should not be to measure whether an answer looks plausible. It should be to measure whether the task was completed successfully.
For programming tasks, a reasonable evaluation unit is:
def evaluate_solution(candidate_code, test_suite):"""Evaluate code-generation quality using verifiable signals."""compile_result = try_compile(candidate_code) # Compilation or syntax checkif compile_result.failed:return {"passed": 0, "total": len(test_suite), "compiled": False}passed = 0for case in test_suite:if run_and_compare(candidate_code, case):passed += 1return {"passed": passed,"total": len(test_suite),"compiled": True,"pass_rate": passed / len(test_suite),}
The central principle is that the reward function should match the evaluation function. When training and production acceptance use the same verifiable signals, capability changes become easier to attribute.
4.2 Security Evaluation: Break “Emergence” into Observable Metrics
Security capability should not be represented by a single aggregate score. It is better to evaluate several measurable categories:
- Vulnerability identification: Can the model identify the specific risk in a code fragment, such as unparameterized SQL construction?
- Repair correctness: Does the proposed patch actually remove the vulnerability without breaking the original behavior?
- Adversarial robustness: Can the model resist prompt injection and unauthorized instructions?
- False-positive rate: Does it over-report risks in safe code? This is often overlooked but has a major impact on usability.
4.3 Context and Cost Evaluation: Quantifying “Lossless” Context
Long-context capability requires dedicated tests. A common approach is a “needle-in-a-haystack” evaluation: insert several facts into a one-million-token context and measure whether the model can accurately retrieve them.
Teams should also record operational metrics such as:
CONTEXT_BUDGET = {"max_context_tokens": 1_000_000,"prefill_latency_ms": 8000, # First-token latency for a very long context"task_token_cost": 12_000, # Average token consumption per task"retry_rate": 0.08, # Initial failure rate}
With these measurements, teams can answer practical questions such as whether a model upgrade actually reduces operating costs.
5. Reusable Engineering Practices for Model Selection and Deployment
5.1 Constraint One: Latency-Sensitive Workloads
Online interaction and IDE completion are highly sensitive to first-token latency. In these scenarios, a million-token context may be a liability rather than an advantage.
A practical approach is to:
- Use retrieval to reduce context to thousands or tens of thousands of tokens.
- Summarize very long inputs before sending them to the model.
- Move heavyweight tasks such as full-repository analysis to an asynchronous workflow rather than synchronous completion.
5.2 Constraint Two: Cost-Sensitive Workloads
For batch code generation and document processing:
Cost = input tokens × input price + output tokens × output price
The value of lower token consumption is amplified in these workloads. Teams should run A/B tests on a fixed task set and compare task_token_cost and retry_rate before and after the upgrade.
def compare_versions(model_a, model_b, task_set):"""Compare cost and quality on the same task set."""rows = []for task in task_set:res_a = run(model_a, task)res_b = run(model_b, task)rows.append({"task": task.id,"cost_a": res_a.input_tokens + res_a.output_tokens,"cost_b": res_b.input_tokens + res_b.output_tokens,"ok_a": evaluate_solution(res_a.code, task.tests)["pass_rate"],"ok_b": evaluate_solution(res_b.code, task.tests)["pass_rate"],})return summarize(rows) # Report cost per successful result, not token count alone
The final line is important: teams should compare cost per successful result, not token count alone. A model that uses fewer tokens but produces more errors may ultimately cost more.
5.3 Constraint Three: Private Deployment and Data Compliance
For organizations that cannot send code or other sensitive data outside their internal network, local deployability is a decisive requirement.
Relevant considerations include:
- How model weights are distributed
- Whether the license permits commercial use
- Whether memory requirements match the existing cluster
- Whether quantized inference is supported
The planned safety review before open-sourcing also illustrates that open source does not mean “release immediately.” Safety checks and license reviews are typically required. For enterprise users, this can provide a clearer path for commercial adoption and deployment.
5.4 Constraint Four: High-Security Environments
Financial services, government, healthcare, and other high-security environments should treat the model as an untrusted component.
All outputs should pass through validation layers. Critical operations should require human review, and prompt injection should be treated as a real attack surface.
Improved model-level security is only the first layer. Defense in depth at the system level remains necessary.
6. Choosing the Right Pace: When to Change the Base Model and When to Post-Train
Taken together, this upgrade suggests a priority order rather than an argument that parameters no longer matter.
- Start with targeted post-training. If the base model already has sufficient knowledge and representation capacity, while its weaknesses are concentrated in engineering conventions, security reasoning, or long-horizon stability, large-scale RL post-training is likely to provide the best return without changing inference cost.
- Then optimize context and inference strategies. Context layering, retrieval compression, and batch scheduling often offer better cost efficiency than replacing the model.
- Replace the base model last. A new base model becomes necessary when the problem is missing knowledge or insufficient fundamental reasoning ability. At that point, teams must also reassess memory, latency, and licensing requirements.
A practical rule is:
If failures are caused by incorrect formatting, missed constraints, or an inability to complete the task correctly on the first attempt, prioritize post-training. If the model fundamentally lacks the required knowledge, consider changing the base model.
7. Conclusion
“Keep the base model unchanged while doubling capability” may sound like marketing language. At the engineering level, however, it describes a reproducible technical path:
- The base model provides knowledge and representation.
- Post-training develops behavior and task strategies.
- Verifiable rewards turn answers that merely look correct into answers that can be tested as correct.
- Long context preserves the complete engineering context when a task requires it.
- Open-source distribution and licensing turn capability into a deployable asset.
For developers, the most important takeaway is not a particular model version. It is the decision-making framework:
- Identify the task’s failure mode.
- Decide which layer needs to change.
- Measure verifiable outcomes before adopting an upgrade.
- Calculate the cost per successful result before choosing a deployment model.
When progress no longer depends entirely on parameter competition, and instead depends on post-training and evaluation engineering, teams regain more control over model selection and deployment.
