Open-Source Trillion-Parameter MoE Flagship: Post-Training, Benchmark Scores, and Deployment Costs
This article examines a newly released open-source Mixture-of-Experts (MoE) flagship model with roughly 2.8 trillion total parameters and about 104B active parameters per token, released as a post-trained version of the previous generation's base model. It explains why sparse MoE decouples compute from capacity while still requiring all weights to be resident in memory, covers the model's 1M-token context window and native multimodal input, and breaks down how its post-training improved agentic task performance and reduced hallucination from 23.4% to 8.3%. You will also find guidance on reading benchmark scores (and their statistical noise), a cost model for its $2/$10-per-million-token API pricing, the memory math behind its 1.56TB weight footprint, and five engineering recommendations for putting a flagship model into production.
1. Introduction: In the Trillion-Parameter Era, the Question Has Changed
The open-source LLM community recently got a flagship release: a team published an open-weight model with roughly 2.8 trillion total parameters on October 23, and uploaded the weights to a public model hosting platform on November 4, so developers can download, deploy, and fine-tune it themselves. By parameter count it has joined the “trillion club,” but the more interesting fact is not the size — it is how the model was produced. It is a post-trained version of the previous generation: the base architecture was not replaced, and the team simply poured large-scale post-training compute into the same base and got a clear performance jump.
That is the real signal. As pretraining data approaches the limits of high-quality corpora, simply adding parameters and tokens yields diminishing returns, and post-training has become the variable that actually separates models. It also puts a hard engineering reality in front of developers: storage, GPU memory, and deployment complexity at trillion-parameter scale are far beyond what an ordinary development environment can handle.
Using this model as a reference point, this article covers five dimensions — architecture, post-training, evaluation, cost, and deployment — to answer three questions: where does the capability come from? How should it be evaluated objectively? And how do you actually put it into a production system?
2. Architecture: Two Ledgers for a Sparse MoE Model
2.1 Total parameters and active parameters must be read separately
The model uses a Mixture of Experts (MoE) architecture: about 2.8 trillion parameters in total, but only about 104B are activated per token, a ratio of roughly 27:1. In other words, each forward pass touches only about 3.7% of the parameters.
That resolves an apparent contradiction. The model has a “knowledge capacity” several tens of times larger than a dense model of comparable size, yet its inference compute cost is closer to that of a hundreds-of-billions-scale dense model. The core idea of MoE is conditional computation, which decouples capacity from compute. The routing step looks roughly like this:
def moe_forward(x, router, experts, top_k):# router: gating network, outputs each token's affinity for every expertlogits = router(x)scores, expert_ids = topk(softmax(logits, dim=-1), top_k)# Normalize the selected experts' weights to keep the output magnitude stablescores = scores / scores.sum(dim=-1, keepdim=True)out = zeros_like(x)for expert_id in torch.unique(expert_ids):mask = (expert_ids == expert_id)# Only routed tokens are computed by the corresponding expertout[mask] += experts[expert_id](x[mask]) * scores[mask]return out# Sparsity check: active vs. total params correspond to two very different cost curvestotal_params, active_params = 2.8e12, 104e9print(f"Active parameter share: {active_params / total_params:.1%}") # ~3.7%print(f"Sparsity ratio: {total_params / active_params:.1f}:1") # ~26.9:1
Two conclusions follow:
- Compute scales with active parameters; memory scales with total parameters. The router must scan all experts on every pass to decide which ones to use, so the compute cost of a single inference is roughly the active parameter count plus a routing overhead — not all 2.8 trillion parameters.
- All weights must be resident. Routing decides “who gets used,” not “who gets loaded.” Any expert may be selected by the next token, so the full weight set must stay in GPU memory or fast host memory at all times. Sparsity saves compute, not memory.
2.2 A 1M-token context and native text/image input
Beyond the sparse architecture, the model extends in two more directions.
- The context window has been expanded to 1 million tokens. That is enough to hold hundreds of pages of technical documentation, an entire code repository, or a full day of conversation and operation logs in one pass — a foundation for long-document understanding, whole-repository code analysis, and agentic tasks.
- It supports native multimodality over text and images. “Native” means the visual capability comes from unified modeling in the base model rather than a bolt-on vision module. The benefit is that image-text reasoning happens inside a single model, keeping the architecture simpler.
One caveat: the cost of native multimodality is that images are also converted into large numbers of visual tokens, and a single high-resolution image often accounts for thousands of tokens. Packing text plus several images into a million-token context pushes up both the compute and the time-to-first-token during prefill.
2.3 Key takeaway: the model has three ledgers
| Resource dimension | Determined by | Main impact |
|---|---|---|
| Compute (FLOPs) | Active parameters (~104B) | Per-token inference speed, GPU utilization, concurrent throughput |
| GPU memory (weights) | Total parameters and weight precision (~1.56TB) | Minimum deployment size, cluster cost |
| GPU memory (dynamic) | Context length, number of concurrent requests | Maximum supported context and concurrency |
Many teams fixate on the parameter count when evaluating candidates, and end up either underestimating compute requirements or getting blocked by memory during deployment. Getting these three ledgers right is the prerequisite for every cost decision that follows.
3. Post-Training: Same Base, Different Capability
3.1 Pretraining builds the base, post-training raises it
LLM training generally splits into two stages.
- Pretraining: self-supervised learning over massive corpora, which teaches the model language patterns and world knowledge and forms the “base.” This stage sets the ceiling of the model’s capability and its baseline style.
- Post-training: on top of the base, techniques such as instruction fine-tuning, reinforcement learning from human feedback, and reinforcement learning on verifiable tasks teach the model to “do what it is told, and do it the right way.” This stage determines whether the model is usable and reliable.
Plenty of industry practice shows that a given base plus more thorough post-training produces substantial gains on coding, math, and tool-calling tasks, without paying the steep pretraining cost again. This open-source flagship is a textbook case of that route.
3.2 Where the post-training effort went: agentic capability
Based on publicly disclosed information, the new version’s post-training focus is clearly on “doing things” rather than “talking”:
- In a specialized test measuring an agent’s ability to complete multi-step tasks in a command-line environment, it scored about 65.7%;
- In a test targeting browsing-based retrieval and deep information gathering, it scored about 92.1%, first in the comparison table the team published;
- In a comprehensive third-party evaluation leaderboard, its intelligence index was about 55, ranking fourth globally.
All of these dimensions point to the same capability: understanding a goal in a real environment, calling tools, executing multiple steps, and self-correcting based on feedback. Gains here come mainly from reinforcement learning with verifiable rewards, where the training signal comes from real execution outcomes — whether a command succeeded, whether tests passed, whether information was found, whether the answer was correct. The more objective the signal, the less likely the model is to learn superficial patterns that merely “look right.”
3.3 Hallucination down ~65%: a more meaningful metric than rankings
More practically important is the improvement in reliability. According to the vendor’s published data, the model’s hallucination rate dropped from 23.4% in the previous generation to 8.3%, a relative reduction of about 65%.
This matters more than a ranking position because hallucination directly determines whether a model can be used in production:
- In user-facing scenarios (customer support, knowledge Q&A), a single factual error can trigger complaints or even a trust crisis;
- In developer-facing scenarios (coding assistants, agents), fabricated APIs and non-existent configuration options cause cascading debugging costs;
- In document-analysis scenarios (contracts, financial reports), a wrong conclusion can directly affect decisions.
When capabilities are close, reliability is what separates product experiences.
That said, vendor-published numbers come from specific test sets and methods. Before adopting the model, enterprises should re-test independently on their own corpora and real questions, and separate three error types — factual errors, stale information, and logical errors — each with its own acceptable threshold.
3.4 Verifying reliability in code: a minimal test
Below is an illustrative reliability acceptance test that checks whether the model knows what it should answer and what it should decline:
def evaluate_reliability(client, samples, evidence_store):total = supported = abstained = 0for s in samples:evidence = evidence_store.search(s.question) # authoritative evidence from the enterprise knowledge basectx = [{"role": "system","content": "Answer strictly from the provided material; if the material does not support an answer, state that it cannot be determined. Do not fabricate."}, {"role": "user","content": f"Material: {evidence}\nQuestion: {s.question}"}]answer = client.chat(ctx).texttotal += 1if judge_answer_supported(answer, evidence): # is the answer backed by the material?supported += 1if "cannot be determined" in answer and not s.answerable: # does it abstain when it should?abstained += 1print(f"Evidence-supported rate: {supported / total:.1%}")print(f"Correct abstention rate: {abstained / total:.1%}")
A reasonable acceptance bar: the evidence-supported rate should clear a target threshold (say 95%), while the model maintains a high abstention rate on questions that should not be answered. Together, these two numbers guard against both fabrication and under-answering.
4. Reading Benchmark Scores: Signal, Noise, and Your Own Evaluation
4.1 The capability gap has entered the “noise zone”
Using public data as an example: this open-source model scored about 65.7% on the command-line agent test, while the leading closed-source flagship scored about 66.4% — a gap of less than one percentage point. A newer-generation closed model scored about 58.2%.
On the comprehensive public leaderboard, it ranks behind the top closed-source flagship, a mid-tier closed model, and another flagship. This means the open-source flagship has joined the first tier in general intelligence, and its remaining gap to the most advanced closed models shows up in a handful of individual capabilities and in product maturity rather than a generational difference.
4.2 What benchmarks do and do not tell you
Benchmarks provide useful horizontal reference points. Specialized tests in particular come closer to concrete work scenarios than a vague “overall capability” score. A command-line task test reflects how well the model actually acts in a real environment; a browsing-retrieval test reflects its ability to gather and verify information.
Benchmarks carry inherent noise. Keep the following in mind:
- Statistical variation: a gap under one percentage point usually sits inside the noise band. Sampling randomness, small differences in grading rules, and even updated question versions can all produce swings of the same magnitude. Repeated runs at different times can flip the ranking, so there is no reason to agonize over 0.7 points.
- Scenario transfer differences: benchmark questions and a company’s real workload are rarely distributed the same way, so evaluation needs to be redone for your own scenario.
- Test contamination: public test sets may already be in the training data, which inflates scores.
- Limits of vendor leaderboards: comparisons published by the vendor are worth reading, but should not be the sole basis for a decision.
The right way to read them is per scenario. For long-document Q&A, look at long-context and information-extraction tests; for coding assistants, look at code and command-line task tests; for customer support, look at hallucination rate and instruction following; for information gathering, look at browsing-retrieval tests.
4.3 Build your own evaluation set: replace the leaderboard with business metrics
What enterprises actually need is a small evaluation set that covers their own business:
- Sampling: draw 300–1,000 real questions from production logs, support tickets, and engineering documentation;
- Labeling: have business experts supply reference answers and grading dimensions such as correctness, completeness, and format compliance;
- Testing: have each candidate model answer in turn, and track task success rate, evidence-supported rate, average latency, and cost per task;
- Regression: rerun the suite after every upgrade or model swap to build a traceable comparison record.
Here is a simplified business evaluation flow:
def run_biz_eval(candidates, biz_cases):report = {}for name, model in candidates.items():passed, cost, latency = 0, 0.0, 0.0for case in biz_cases:resp = model.invoke(case["prompt"])usage = resp.usage # input/output token counts, cache hits, and other metering infocost += usage.input_tokens * model.in_price \+ usage.cached_tokens * model.cache_price \+ usage.output_tokens * model.out_pricelatency += resp.latency_mspassed += int(grade(resp.text, case["expected"])) # business grading functionn = len(biz_cases)report[name] = {"task_success_rate": passed / n,"avg_latency_ms": latency / n,"cost_per_task": cost / n,}return report
The leaderboard decides the candidate list; the business evaluation decides the final choice.
5. Cost: The Inference Economics Behind API Pricing
5.1 Price signal: entering the first tier on price-performance
The model’s API pricing is $2 per million input tokens and $10 per million output tokens, roughly half that of the leading closed-source flagship. The vendor also offers a Highspeed version that is 2–3x faster than the standard one at twice the price — about $4 input / $20 output per million tokens on published rates. This “standard plus highspeed” dual tier splits latency-sensitive and cost-sensitive workloads into two products, so enterprises can pick by business need.
Output tokens usually cost several times more than input tokens because the two kinds of computation have different cost structures:
- Prefill: input tokens are processed in parallel in one pass, with high GPU compute utilization;
- Decode: output tokens are generated one at a time, constrained by memory bandwidth and unable to saturate the compute units.
The highspeed version typically buys its speedup with more compute resources, more aggressive batching, or a larger memory budget — all of which cost money. If the workload tolerates it, the standard version is usually enough in most scenarios.
5.2 Running the numbers: cost per task and monthly bill
Assume a typical interaction for a coding assistant:
- about 2,000 input tokens per turn, of which roughly 1,500 (system prompt plus code context) can be served from cache;
- about 500 output tokens per turn;
- an average of 5 turns to complete one task.
| Version | Cost per turn ($) | Cost per task ($) | 10k tasks/day, monthly (30 days) |
|---|---|---|---|
| Standard ($2/$10) | 0.0110 | 0.055 | 16,500 |
| Highspeed ($4/$20) | 0.0230 | 0.115 | 34,500 |
| Peer closed flagship ($4/$20) | 0.0230 | 0.115 | 34,500 |
| Next-gen closed flagship ($8/$24) | 0.0430 | 0.215 | 64,500 |
At the same per-million-token pricing, every $2 increase in the output price adds about $0.005 to the cost per task; at 10,000 tasks per day, that accumulates to roughly $547,500 over a year. For high-volume workloads, price differences quickly become material cost differences.
Here is runnable sample code for a quick estimate:
def task_cost_musd(in_tok, out_tok, cache_hit_ratio,rounds, tasks_per_day, days, p_in, p_out):"""Estimate monthly cost in USD.in_tok / out_tok: input/output tokens per turn; cache_hit_ratio: cache hit ratio;rounds: average turns per task; tasks_per_day: average daily tasks; days: number of days;p_in / p_out: input/output price per million tokens (USD); cached tokens should bepriced according to the actual billing rules."""cached_in = in_tok * cache_hit_ratiofresh_in = in_tok - cached_incost_per_round = (fresh_in * p_in + cached_in * p_in * 0.1 + out_tok * p_out) / 1e6return cost_per_round * rounds * tasks_per_day * daysif __name__ == "__main__":base = dict(in_tok=2000, out_tok=500, cache_hit_ratio=0.75,rounds=5, tasks_per_day=10000, days=30)std = task_cost_musd(p_in=2, p_out=10, **base)fast = task_cost_musd(p_in=4, p_out=20, **base)print(f"Standard monthly cost: ${std:,.0f} Highspeed monthly cost: ${fast:,.0f}")# Output: Standard monthly cost: $16,500 Highspeed monthly cost: $34,500
Note that the “peer closed flagship” row is priced at the same per-million-token rates purely to compare cost structures; it does not represent that vendor’s actual official pricing. Real billing also depends on cache-hit and batch-inference discounts, and on how input and output tokens are metered. The method does not change, and the core conclusion holds: output length and the number of turns usually move total cost more than input length does.
6. Deployment: What 1.56TB of Weights Actually Means
6.1 The memory ledger: weights, KV cache, and activations
The published weight size is about 1.56TB. As a rule of thumb, generating one token requires reading the weights that participate in the computation once, and 1.56TB is the combined memory of dozens of high-end data-center GPUs. Actual memory requirements include at least three parts:
- Weights: about 1.56TB, all of which must stay resident because every expert is in play;
- KV cache: grows linearly with context length and concurrency;
- Activations and communication buffers: vary with batch size and parallelism strategy.
KV cache can be estimated with a simple formula. The code below assumes typical layer and attention-head configurations; these are not the model’s actual parameters:
def kv_cache_gb(tokens, layers=80, kv_heads=8, head_dim=128, dtype_bytes=2):# Each token caches the Key and Value of every layer; standard attention assumed hereper_token = 2 * layers * kv_heads * head_dim * dtype_bytesreturn per_token * tokens / 1e9 # ~0.33MB per tokenprint(f"100k tokens: {kv_cache_gb(100_000):.1f} GB") # 100k tokens: 32.8 GBprint(f"1M tokens: {kv_cache_gb(1_000_000):.1f} GB") # 1M tokens: 327.7 GB
In other words, a single request approaching 1 million tokens can need roughly 330GB of GPU memory for KV cache alone — the equivalent of several more high-end GPUs. Hence:
- 1 million tokens is a capability ceiling, not a sensible default configuration. The usable length depends on hardware scale, concurrency, and optimization strategy;
- Common ways to cut dynamic overhead: quantize and compress the KV cache; reuse identical prefixes so system prompts, knowledge bases, and code context are not recomputed; retrieve over long context in segments and feed the model only relevant snippets; schedule dynamically based on request length.
6.2 Stacking optimizations still leaves a high bar
Even with the following engineering optimizations, the practical deployment footprint remains substantial:
- low-precision weight quantization on expert layers to shrink weight size;
- expert parallelism, distributing different experts across different GPUs;
- disaggregated prefill and decode, so each workload type gets suitable resources;
- continuous batching to improve concurrency utilization.
Rough estimate: spread 1.56TB of weights across 80GB GPUs and the theoretical minimum is about 20 cards; with 141GB high-end GPUs, a single 8-GPU server already provides about 1.1TB of memory.
Practical recommendation: once KV cache and communication overhead are added, expect to start with two to three 8-GPU servers, and to scale further for stable concurrency. That excludes high-speed interconnect, datacenter power and cooling, and operational monitoring.
So ordinary developers realistically cannot run this model locally, and for the vast majority of teams the rational and practical choice is to call it through a cloud API or a managed inference service.
7. Production Practice: Five Engineering Recommendations
7.1 Prefer managed cloud inference and choose the version per workload
In most cases, enterprises do not need to build their own trillion-parameter cluster. Baidu AI Cloud, for example, offers general-purpose large-model capabilities covering model invocation, model development, and inference deployment; check the official documentation for regional availability. Engineering teams can use object storage to manage corpus and model assets, container platforms to host inference services, and logging plus monitoring and alerting to track call quality.
Choose the highspeed version for latency-sensitive scenarios and the standard version for cost-sensitive ones, and put a unified model access layer in front so versions can be switched and rolled out gradually without binding business code to a specific model.
7.2 Let business metrics drive selection, not leaderboard rank
As discussed earlier, a sub-one-percentage-point benchmark gap has almost no bearing on business outcomes. What determines the experience is task success rate, evidence-supported rate, and cost per task. Build a regression evaluation loop that runs continuously and rerun it automatically after every model upgrade.
7.3 Build a reliability architecture of “model + retrieval + verification”
Cutting hallucination from 23.4% to 8.3% is real progress, but the remainder still needs engineering safeguards:
- feed enterprise knowledge-base retrieval results in as context so the model has grounds to answer from;
- add rule-based validation, unit tests, or human review at critical output points;
- use the system prompt to require the model to abstain when evidence is insufficient.
7.4 Long context is a capability and a cost center
A million-token context suits long-document analysis and whole-repository code understanding, but it brings two problems: cost rises linearly with input length, and so does latency. Stick to “send only what is needed” — narrow the scope with retrieval first, then let the model process the content that actually matters.
7.5 Budget compute for multimodal scenarios
Native multimodality simplifies the architecture of image-text applications, but image token counts can far exceed expectations. Before launch, measure token consumption and end-to-end latency for images at different resolutions, and set resolution caps and concurrency limits accordingly.
8. Conclusion: From Benchmark Competition to Cost Engineering
This open-source flagship release sends three clear signals:
- Capability keeps closing in on the frontier. On several key benchmarks, the gap between open models and the leading closed-source flagship is already under one percentage point;
- Reliability is the new competitive focus. A roughly 65% drop in hallucination shows the industry shifting from “who is smarter” to “who is more trustworthy”;
- Cost is the decisive variable for adoption. Half-price API access lowers the barrier to use, but a 1.56TB weight footprint keeps private deployment on the heavy-asset path.
For developers and technical decision-makers, the next focus should not be benchmark comparison but systematic cost engineering:
- replace leaderboard rank with business evaluation;
- build trustworthy output with “model + retrieval + verification”;
- replace speculative self-hosting with managed cloud inference.
Teams that can run a flagship model affordably, reliably, and accurately are the ones that will turn model progress into business value.
