Post-Training and Deployment Practices for Ultra-Large Mixture of Experts (MoE) Models
This article explores the practical aspects of post-training and deploying ultra-large Mixture of Experts (MoE) models. It highlights why post-training is critical for releasing model capabilities, analyzes the benefits and costs of sparse architectures with long-context and multimodal features, provides insights on interpreting public benchmarks versus private evaluations, and outlines best practices for deployment, cost management, data governance, and continuous iteration.
1. The Key to Capability Leap: Shifting from “Base Model Replacement” to “Post-Training”
In the competition of ultra-large models, the base architecture determines the capability ceiling, while post-training determines whether these capabilities can be reliably released. Public materials from such flagship models provide a typical example: new versions do not replace the base model but instead invest significant post-training compute on the same base. By reshaping the model’s behavioral distribution through supervised fine-tuning, preference optimization, reinforcement learning with verifiable rewards, tool-use trajectory training, long-context curriculum learning, multimodal alignment, and hallucination suppression, developers realize that model iteration does not necessarily require structural parameter changes. Version management should not only record the “model name” but also track the base version, post-training recipe, data version, evaluation set version, and inference configuration. Otherwise, attributing performance differences for the same model across different service environments becomes difficult.
The core value of post-training lies in transforming “knowledge capacity” into “task completion rate.” For instance, terminal agent tasks require the model to understand commands, plan steps, call tools, read execution feedback, and retry upon failure. Deep browsing and retrieval tasks require the model to decompose problems, conduct multi-round retrieval, cross-verify, and provide traceable citations. These capabilities are hard to obtain solely through base pre-training; they require substantial trajectory data with execution feedback and verifiable rewards. Engineering-wise, a closed loop of “data—training—private evaluation—gray release” should be established, avoiding decisions based solely on public leaderboards.
2. 2.8 Trillion Total Parameters and 104B Activations: Benefits and Costs of Sparse Architecture
These models adopt a Mixture of Experts (MoE) architecture, with a total parameter scale reaching 2.8 trillion and approximately 104B parameters activated per inference. The significance of sparse activation is using larger parameter capacity to carry knowledge and multi-task capabilities while keeping the per-token compute close to that of a 100B-level dense model. However, the costs are equally apparent. Expert parallelism introduces cross-node communication, and uneven routing loads can cause tail latency. Batch size, expert capacity factor, tensor parallelism, and pipeline parallelism strategies all affect throughput. During deployment, one must not only look at average latency but also observe P95/P99 latency, expert hit distribution, memory fragmentation, and communication wait times.
A 1 million token context is another key capability. It enables repository-level code understanding, long-log troubleshooting, multi-round tool trajectory replay, and long-document review. However, long context is not “free.” KV cache grows with sequence length and batch size, significantly increasing attention computation and memory pressure. Production systems typically require a combination of prefix caching, chunked summarization, retrieval augmentation, hierarchical memory, and timeout control. Native text-plus-image multimodality further expands scenarios like screenshot understanding, chart Q&A, and UI automation, but visual token budgets, image resolution strategies, and cross-modal alignment quality directly impact cost and accuracy.
3. Reading Public Benchmarks Correctly: Small Gaps Do Not Mean No Difference in Production
Public data shows that these models achieve about 65.7% on terminal agent benchmarks, only 0.7 percentage points behind a contemporaneous first-tier closed-source counterpart at 66.4%, while another closed-source counterpart scored 58.2%. On deep browsing retrieval benchmarks, they report 92.1%, ranking first in their public comparison table. These numbers indicate the gap between the open-weight route and closed-source flagships is narrowing. However, reading leaderboards must be done with caution: terminal tasks are affected by tool permissions, sandbox environments, timeout settings, retry counts, scoring scripts, and task snapshots. Browsing retrieval scores are affected by retrieval sources, web page timeliness, and citation verification criteria. Self-reported leaderboards should especially be reviewed for task slicing and failure cases.
In contrast, the hallucination rate dropping from 23.4% to 8.3% is more engineering-relevant. A lower hallucination rate means fewer factual errors, unsupported assertions, and incorrect citations, but evaluations should distinguish between closed-book Q&A hallucinations, post-tool-call hallucinations, unfaithful citations, and lost-in-the-middle information in long contexts. Enterprises are advised to build private evaluation sets, fix prompts, random seeds, tool versions, and timeouts, report confidence intervals, and slice by task type. For example:
eval_config:suites: [terminal_agent, deep_browse, long_context_qa, multimodal_ui]tool_permissions: sandbox_limitedretries: 1timeout_s: 600seeds: [1, 2, 3]metrics: [success_rate, citation_faithfulness, hallucination_rate, p95_latency]report_by_slice: true
Only by converting public leaderboards into private benchmarks consistent with business distributions can one determine whether the model is truly usable.
4. Deployment and Cost: MoE Serving Requires Systematic Design
Deploying ultra-large MoE models is a systems engineering effort. On the inference side, container platforms can handle orchestration, object storage manages weights and evaluation sets, logging services and monitoring alerts build an observability loop, and general model inference service capabilities enable elastic scaling. Capacity planning should simultaneously consider weight memory, KV cache, activation memory, and fragmentation. Long-context requests should have tiered queues, timeouts, and degradation strategies. Multimodal requests should control image size and concurrency during preprocessing. The cost model should not only calculate token unit price but also include cache hit rates, tool call counts, retry rates, and manual review costs. It is recommended to use “task success rate / unit cost” rather than pure throughput as the core metric.
5. Best Practices for Post-Training Data and Governance
Post-training data should cover general instructions, domain tasks, tool trajectories, long context, multimodality, and safety rejections. Synthetic data must be deduplicated, quality-filtered, and retain verifiable rewards. Reinforcement learning prioritizes verifiable tasks like code execution, command execution, and retrieval Q&A. Hallucination prevention should combine citation constraints, tool-first strategies, rejection policies, and factual consistency rewards. In terms of governance, one should adhere to least privilege, sandbox execution, audit logs, manual review, and gray releases. Ultra-large models are highly capable, but only by making evaluation, deployment, observation, and governance a closed loop can the “leaderboard gap” be transformed into sustainable productivity.
