LLM Post-Training: From Answering Questions to Performing Work
This article explains how post-training moves large language models beyond question answering toward executing tasks. It examines four technical pillars: sparse mixture-of-experts architecture for capacity and cost, reinforcement learning with verifiable rewards for turning knowledge into behavior, long context and native multimodality for broader task coverage, and hallucination governance for business trust. It also covers evaluation practices, deployment considerations, and common pitfalls.
Introduction: The same base model, post-training moves models from answering questions to performing work
The main line of capability improvement is shifting
Over the past few years, improvements in large models have mostly come from scale: more parameters, more data, and longer training. A recent signal is worth noting. Publicly disclosed information suggests that a trillion-parameter open-source flagship model in a given series only added substantial post-training compute on the same base model, including reinforcement learning fine-tuning, RLHF, and other agent-oriented post-training methods. According to public reports (as of this article’s writing, the publisher has not fully disclosed the evaluation methodology, number of tool calls, or statistical scope; this point is for trend reference only), the model narrowed the gap with leading closed-source models in some terminal-operation agent evaluations to roughly one percentage point.
The technical meaning behind this is clear: pretraining determines what the model knows, while post-training determines what it can do. Once pretraining reaches a certain scale, the key constraint on real-world performance shifts from knowledge capacity to execution ability, including multi-step planning, tool use, error recovery, and result verification. For developers and enterprises, post-training creates new model-selection logic, new cost structures, and new requirements for evaluation.
Analysis framework
This article does not review a single model. Instead, it uses four technical pillars to explain how post-training improves execution ability and how teams should adopt it:
- Architecture pillar: Sparse mixture-of-experts (MoE), which expands capacity while controlling cost.
- Training pillar: Reinforcement learning and post-training centered on outcome rewards, which turn knowledge into behavior.
- Capability pillar: Long context and native multimodality, which expand task boundaries.
- Quality pillar: Hallucination governance, which determines whether the model can be trusted in business settings.
1. Architecture pillar: Sparse activation expands capacity and controls cost
1.1 Decoupling total parameters from activated parameters
MoE splits the feed-forward layers into multiple parallel “experts” and uses a gating network to select a small number of experts for each token. This separates “knowledge capacity” from “compute per token”: total parameters grow with the number of experts, while compute per token depends roughly on the number of activated experts and the size of each expert.
In a conventional dense model, parameter count and inference compute are roughly proportional, so capability growth usually comes with higher inference cost. Sparse architecture decouples total capacity from compute, providing a structural basis for “large capacity, lower cost.” Post-training depends on large-scale sampling to generate trajectories. The size of activated parameters determines sampling compute cost, making large-scale post-training more cost-feasible.
1.2 Engineering costs of sparse architecture
Sparsity is not free; it moves costs to engineering:
- Memory: During inference, expert weights must generally remain in GPU memory or system memory so they can be accessed when needed. A trillion-parameter total size can require hundreds of GB of memory and usually needs multi-device parallelism. Tensor parallelism handles matrix splitting inside a single expert, while expert parallelism distributes different experts across devices.
- Communication: Every MoE layer requires exchanging tokens across devices (All-to-All communication). High-speed interconnects or high-bandwidth networks are prerequisites for throughput.
- Load balancing: If a small number of experts are selected too often, devices may become unevenly loaded. During training, auxiliary load-balancing losses are commonly used to constrain routing distributions. During inference, expert cache hit rates also matter.
- Quantization: Quantization is a common way to reduce memory usage. Because each token passes through only a few experts, expert-weight utilization is relatively low, so group-wise quantization and similar methods may be needed to control accuracy loss.
1.3 Example: one MoE forward pass
The following code first groups tokens by expert and then aggregates the tokens and gating weights for each expert. It illustrates the calculation logic only and is not a production implementation:
def moe_forward(x, experts, router, k=2):# router outputs the probability of selecting each expert, shape [num_tokens, num_experts]probs = router(x)topk_prob, topk_id = probs.topk(k, dim=-1) # select k experts for each tokentopk_prob = topk_prob / topk_prob.sum(-1, keepdim=True) # normalize gating weightsy = torch.zeros_like(x)for eid, expert in enumerate(experts): # iterate over experts, not tokenssel = (topk_id == eid) # shape [num_tokens, k]: which slots select this expertidx = sel.any(dim=-1).nonzero()[:, 0] # token indices that need this expertif idx.numel() == 0:continuew = (topk_prob * sel).sum(dim=-1)[idx] # gating weight of these tokens for this experty[idx] += w.unsqueeze(-1) * expert(x[idx]) # compute only selected tokens and accumulatereturn y
The truly complex parts in engineering happen outside this loop: how tokens are grouped by expert and sent to the right device, how uneven load is handled, and how expert caches are managed. When selecting a framework, teams should pay attention to its support for expert parallelism and communication, not only to the parameter scale.
2. Training pillar: Reinforcement learning and post-training turn knowledge into execution
2.1 Division of labor between supervised fine-tuning and reinforcement learning
Supervised fine-tuning (SFT) learns from human demonstrations: “what to do” and “how to do it.” It is suitable for quickly aligning output format and basic behavior. Reinforcement learning optimizes policies with outcome rewards and lets the model explore “what must be done to succeed.” For multi-step tasks, successful paths are often not unique, and many valuable intermediate steps do not appear in demonstration data. Reinforcement learning can extract signals from both successful and failed trajectories. This is one of the main reasons post-training performs well on agent tasks.
2.2 Core paradigm: reinforcement learning based on verifiable outcomes
Reinforcement learning works well only when rewards are reliable. In terminal-operation, code-generation, and mathematical-reasoning tasks, outcomes can be judged automatically by programs. For example, whether a command executed successfully, whether unit tests pass, or whether the final answer is correct. These verifiable rewards reduce dependence on a reward model and limit the model’s ability to exploit the reward signal.
One simplified method used in recent years samples multiple answers for the same problem and uses the average reward within the group as a baseline to compute relative advantage. This removes the need for a separate value network:
# Illustrative code: group-relative advantage estimation, matching common GRPO-style methodsdef group_relative_advantage(rewards, eps=1e-4):# rewards: shape [G], where G is the number of samples for the same problemmean = rewards.mean()std = rewards.std(unbiased=False)return (rewards - mean) / (std + eps) # samples above the group mean get positive advantage
2.3 From single-turn Q&A to multi-step agents
Terminal-operation tasks usually involve a multi-step loop: observe the environment, make a plan, execute commands, read results, and correct errors. Reinforcement learning has evolved accordingly:
- Early approaches calculated rewards per single-turn answer, making it hard to handle failures in intermediate steps.
- Trajectory-level rewards, computed from complete task trajectories and assigned according to whether the final task succeeds, are better suited to long-horizon tasks.
- Splitting “task success,” “format compliance,” and “no dangerous operations” into multiple reward terms and weighting them allows finer control of model behavior.
# Illustrative code: weighted composite rewarddef composite_reward(task_result, fmt_ok, unsafe_steps):return (1.0 * task_result.success # outcome: whether the task is completed+ 0.2 * fmt_ok # format: whether the output follows the required structure- 0.5 * unsafe_steps # safety: penalty for high-risk operations)
Three issues deserve attention during training:
- Reward design: Reward terms must cover the dimensions the business truly cares about; otherwise, the model will optimize the reward rather than the business objective.
- Stability: It is necessary to constrain how far the policy drifts from the reference model to avoid training collapse or loss of language ability.
- Exploration efficiency: Sampling is expensive, and task difficulty is unevenly distributed. Methods such as curriculum learning and difficulty-stratified sampling can improve the return per unit of compute.
2.4 Reallocation of training and inference costs
Post-training requires repeated “generate, evaluate, update” cycles. The sampling stage itself is large-scale inference. Inference-side techniques directly affect post-training efficiency, including efficient attention, KV cache reuse, continuous batching, and reasonable control of activated parameter size.
This leads to an important implication for engineering teams: while post-training drives capability improvements, it also raises compute investment per unit of effectiveness. A rough estimate can use the following dimensions:
cost per task ≈ average steps × average tokens per step × number of samples × inference price per token
“Inference price per token” is directly related to activated parameter size, parallelism strategy (tensor parallelism or expert parallelism), and batching efficiency. When selecting models, teams should not only look at whether performance approaches that of leading models. They should also evaluate cost per task, inference latency, and throughput to avoid a situation where capability catches up but the bill doubles.
3. Capability pillar: Long context and native multimodality
3.1 Long context: from “how long can it read” to “how accurately can it read”
A context window of millions of tokens allows entire code repositories, long documents, and multi-turn history to enter the model at once, reducing dependence on retrieval pipelines. In practice, usability depends on three factors:
- Effective context: A claimed context window is not the same as an effective one. Information in the middle of a long context is easily overlooked. A practical test is to insert key information at different depths and then ask questions about it.
- Inference overhead: Attention overhead grows rapidly with sequence length, and KV cache usage grows linearly with length. Very long inputs can significantly increase time to first token and memory usage.
- Engineering methods: Hierarchical retrieval, placing key information first, reusing prefix caches, and summarizing conversation history are common ways to control cost.
3.2 Native multimodality: unified representation instead of a bolt-on approach
Native multimodality means that text and images are jointly modeled under the same parameters and training objectives, rather than using a “visual encoder + projection layer + language model” patchwork. The advantage of the former is more natural cross-modal reasoning. Examples include reading reports with charts, understanding UI screenshots, and recognizing layouts and tables in scanned documents. For terminal agents, screenshot understanding is a foundational capability for “seeing the screen and clicking buttons.”
In deployment, image resolution and tiling strategy directly affect detail recognition and token consumption. Consider a tradeoff example. If a screenshot is split into a 2×2 grid of 512×512 tiles, token consumption grows roughly linearly with the number of tiles, while recognition of small text, icons, and table lines improves noticeably. Using only one image saves compute but is more likely to miss details. It is advisable to run canary tests by scenario and use the tradeoff between “seeing clearly” and “saving compute” to find the optimal point.
4. Quality pillar: Hallucination governance determines whether models can be trusted
4.1 Why lower hallucination rates matter more than leaderboard scores
Terminal-operation tasks are highly sensitive to errors. One wrong command or one wrong file path can cause real damage. According to public materials from the relevant publisher (as of this article’s writing, complete evaluation details were not provided, and this is for trend reference only), the series reduced the hallucination rate from about 20% to around 8%. This is an important result of post-training: the model no longer forcibly gives an answer but learns to distinguish between “certain” and “uncertain.”
4.2 How post-training reduces hallucinations
- Format and refusal alignment: Supervised data can explicitly include examples such as “say you do not know when you do not know.”
- Outcome rewards: When answers are verifiable, incorrect answers receive zero reward, encouraging the model to converge toward reliable paths.
- Confidence calibration: Making the model’s confidence better aligned with actual accuracy helps upper-level systems decide whether to execute directly or ask a human for confirmation. Calibration can be implemented with temperature scaling or reliability diagrams.
4.3 Engineering defenses
Improved model capability does not mean the model can be left unsupervised. Four layers of defense are recommended:
- Retrieval augmentation: Require answers to include sources. Conclusions without sources should be marked as needing verification.
- Structured validation: Apply schema and syntax checks to strongly formatted outputs such as JSON, commands, and file paths.
- Sandboxed execution: Run code and commands first in an isolated environment, and fall back if they fail.
- Human fallback: Require confirmation and operation logs for high-risk actions.
# Illustrative code: confidence-based response dispatch (thresholds should be determined through business backtesting)def dispatch(answer, confidence, verified):if verified:return {"action": "execute"} # verified: execute directlyif confidence >= 0.85:return {"action": "execute_with_log"} # high confidence: execute with loggingif confidence >= 0.6:return {"action": "ask_user"} # medium confidence: ask the user for confirmationreturn {"action": "abstain", "msg": "Insufficient information. Please provide more material."} # low confidence: refuse
The thresholds 0.85 and 0.6 are only examples. In practice, confidence should first be calibrated, and thresholds should be backtested with canary or production data. Different businesses have different risk tolerances.
5. Evaluation: do not let leaderboards dictate decisions
5.1 Three traps of leaderboards
- Leaderboard overfitting: When training objectives highly overlap with an evaluation set, higher scores may not reflect generalization.
- Differences in evaluation methodology: Different leaderboards may define tool use, sampling counts, and prompt engineering differently. Cross-comparison requires aligned methodology.
- Missing cost dimension: Comparing only effectiveness while ignoring cost per task and latency can lead to wrong conclusions.
5.2 Building an internal evaluation set
A practical structure is “three layers plus one set,” with a minimal starter checklist:
- L1 Basic layer: Format compliance, instruction following, and refusal ability, about 100–200 cases.
- L2 Task layer: Sample and construct tasks from real business logs. Layer them by failure mode, with about 30–50 cases per layer.
- L3 End-to-end layer: Run complete workflows in a sandbox. Use 20–30 representative scenarios to measure task success rate and human intervention rate.
- Regression set: Trigger a full regression whenever the model version or prompts change, preventing capability regressions.
For labeling, it is recommended to consistently record “task goal / expected result / allowed action boundaries.” Recommended metrics include task success rate, average steps, cost per task, P95 latency, hallucination rate, and human takeover rate. The business side can start with a small-scale canary rollout and then decide whether to increase traffic.
6. Adoption recommendations and common pitfalls
6.1 Adoption path
- Start with a baseline: Use the existing model and prompt engineering to run the business workflow end to end. Determine whether the bottleneck is capability or engineering.
- Then choose the form: If tasks mainly involve long-document understanding, prioritize long-context capability. If stable structured output is needed, prioritize format following and tool-use capability.
- Then optimize: If necessary, use business data for lightweight fine-tuning or retrieval augmentation. Avoid retraining from scratch at the start.
6.2 Common pitfalls
- Pitfall 1: Bigger parameter count is always better. In sparse models, total parameters and activated parameters differ greatly. Actual compute and memory needs should be estimated based on activated parameter size and parallelism strategy.
- Pitfall 2: Post-training is just about adding more data. The core of post-training is reward design and training stability. Data quality matters more than quantity.
- Pitfall 3: A lower hallucination rate means the model is trustworthy. Hallucination is a probability issue, not an on/off switch. Engineering validation cannot be omitted.
- Pitfall 4: Long context can completely replace retrieval. Very long inputs bring latency and cost. Retrieval still has advantages in precise sourcing and cost control.
6.3 General pieces of cloud deployment
The following is a vendor-neutral capability checklist (these are general deployment elements, not any vendor’s evaluation result or commercial recommendation). Actual selection should consider compliance and cost requirements:
- Use container orchestration to run inference services and support elastic scaling for traffic fluctuations.
- Use object storage for model weights, evaluation datasets, and business documents.
- Use message queues to decouple long-running tasks and support asynchronous agent jobs.
- Use logging services and monitoring alerts to track token consumption, latency, task success rate, and human takeover rate.
Conclusion
The divide between answering questions and performing work is not in the parameter table; it is in post-training. Sparse architecture provides a way to decouple capacity and cost. Reinforcement learning turns knowledge into executable behavior. Long context and native multimodality expand task boundaries. Hallucination governance determines the boundary of trust. For engineering teams, real competitiveness lies in building a closed loop of evaluation, model selection, canary rollout, and monitoring, so that improvements in model capability can be converted into stable business value.
