2.8T-Parameter Open-Source MoE: Benchmarks, Deployment, and Cost Decision Guide

    This article examines a 2.8-trillion-parameter open-source MoE model and explains how to interpret its benchmark results, estimate deployment requirements for its 1.56TB weights, and decide between self-hosting and managed API usage based on cost, latency, compliance, and operational constraints.

    1. What changed: an iteration that mostly added post-training compute

    In late October 2025, a large-model team released an open-source flagship model with 2.8 trillion parameters. About ten days later, the weights were published to mainstream model hosting platforms, allowing developers to download, quantize, and deploy the model themselves.

    This was not a newly pretrained base model. It was an enhanced version of the previous open-source generation, with substantially more post-training compute. That distinction shapes its capability profile: the base model defines the ceiling, while post-training determines how well the model uses that ceiling.

    For readability, this article refers to this class of 2.8-trillion-parameter open-source flagships as the candidate model.

    Dimension Parameter / Notes
    Architecture Sparse mixture-of-experts (MoE); 2.8 trillion total parameters; about 104 billion active parameters per inference
    Context length About 1 million tokens
    Modality Native text + image multimodal
    Open-weight size About 1.56TB
    API pricing $2 input / $10 output per million tokens; roughly half the price of top international closed-source flagships
    Value-added version High-speed version: 2–3x throughput, with double the price
    Key derivative result Previous generation hallucination rate: 23.4%; current generation: 8.3%

    Why post-training compute matters

    The base model determines what the model can do. Post-training determines how well it does it. The main post-training approaches are:

    • Supervised fine-tuning: teaches the model to imitate high-quality instructions, tool calls, and code execution trajectories.
    • Preference alignment: makes model outputs more aligned with human preferences and safety requirements.
    • Reinforcement learning: uses automatically verifiable tasks, such as unit tests, terminal command results, and math answers, to iteratively optimize behavior based on outcome correctness.

    For terminal-agent tasks that involve multiple steps and verifiable results, reinforcement learning is especially effective. The model can learn from success and failure trajectories in realistic environments.

    Public results support this. On the same base model, post-training reduced the hallucination rate on general question-answering tasks from 23.4% to 8.3%, a reduction of about 65%.

    However, three caveats matter:

    1. The base model still defines the ceiling. Post-training brings the model closer to that ceiling, but hard limits in reasoning, planning, and complex mathematics still depend on the base model.
    2. Returns may diminish. Doubling post-training compute does not necessarily double capability.
    3. Alignment can introduce capability loss. Both unintended “alignment tax” and deliberate removal of safety-sensitive data can reduce some capabilities.

    Therefore, when upgrading to a newer post-trained version, keep the old version as a gray-release baseline and validate it with your own business evaluation set. Do not assume the new version is automatically better.

    2. Understanding 2.8 trillion parameters: total parameters vs. active parameters

    Sparse MoE separates two different concepts: knowledge capacity and per-inference compute. This is the key to understanding the model.

    • Knowledge capacity is determined by total parameters. All expert weights must reside in GPU memory, which directly determines memory usage and the minimum hardware requirement.
    • Per-inference compute is determined by active parameters. Each token is routed through only a small subset of experts, so compute is much lower than that of a dense model of the same total size.

    In this case, only about 3.7% of the parameters participate in a single forward pass. Some people therefore conclude that a 2.8-trillion-parameter model requires only small-model-level compute. This confuses memory capacity with compute capacity.

    • Prefill is compute-intensive. It benefits from batch processing.
    • Decode generates tokens one by one and is memory-bandwidth-intensive. The bottleneck is reading weights and the KV cache. All experts must remain resident in memory whether or not they are selected.

    Engineering implication: hardware planning must cover four dimensions at the same time: memory capacity, memory bandwidth, inter-GPU interconnect bandwidth, and compute. Judging the requirement by only one dimension will lead to a wrong conclusion.

    3. How to read benchmark results: agent capability, retrieval capability, and hallucination rate

    Benchmarks are the most shareable part of a model launch, but they are also the easiest to misread. The table below summarizes publicly available data and explains how to interpret each result.

    Evaluation dimension Candidate model International closed-source flagship A International closed-source flagship B Interpretation
    Terminal Agent benchmark 65.7% 66.4% 58.2% Only 0.7 percentage points behind the strongest closed-source flagship
    Web retrieval benchmark 92.1% Not listed Not listed Ranked first in the comparison table published by the model provider
    Third-party composite intelligence index 55 — — Ranked fourth globally
    Hallucination rate vs. previous generation 8.3% — — Down from 23.4% in the previous generation, a reduction of about 65%

    Note: the data above comes from the model provider or third-party disclosures. Evaluation configurations may differ across sources. When citing these numbers, go back to the original evaluation report.

    3.1 Terminal Agent benchmark: the score is a function of model + execution framework

    A terminal agent must perform multi-step operations in real environments such as command lines and file systems: reading files, modifying code, running tests, and retrying based on errors. This type of benchmark measures end-to-end ability to complete tasks, not single-turn answer quality.

    • Meaning of the gap: 0.7 percentage points is roughly the scale of fewer than 10 tasks. Pricing, availability, and compliance all sit between the candidate model and the strongest closed-source flagship. The engineering meaning of “only 0.7 points behind” is more important than the marketing meaning of “tied for first.”
    • Scaffolding effect: the same model can score differently under different execution frameworks. Toolsets, context management, retry policies, and timeout settings can all change the result. When comparing models, keep the scaffolding fixed.
    • Sampling settings: pass@1 and pass@k are not interchangeable. Parameters such as temperature and top-p also affect results.
    • Scoring method: does the benchmark score per step or per completed task? Does it allow partial completion? These choices change the final percentage.

    Recommendation: treat public leaderboards as a reference coordinate for the capability ceiling, and treat end-to-end success rate in your own business scenarios as the admission threshold.

    3.2 Web retrieval benchmark: a high score does not mean RAG is ready out of the box

    Web retrieval tasks require the model to search, locate, filter, and summarize information in an open information environment. A score of 92.1% suggests that the “retrieve + summarize” chain is relatively complete. This is supported by long context, tool calling, and multimodal understanding.

    Inside an enterprise, however, data usually lives in private documents, databases, and ticketing systems. There is no public web page to retrieve directly. What transfers is not the score itself, but the capability pattern:

    decompose the question → retrieve → cross-validate → provide an answer with sources

    Practical points:

    • Citations and traceability: require every conclusion to include evidence sources so that humans can verify it.
    • Refusal mechanism: when evidence is insufficient, the model should explicitly say it is uncertain rather than forcing an answer.
    • Multimodal input: native image understanding can help parse screenshots, scanned documents, and reports. But image resolution and slicing strategy directly affect token consumption.

    3.3 Lower hallucination rate: a more practical gain than benchmark points

    Reducing the hallucination rate from 23.4% to 8.3% was achieved through post-training on an almost unchanged base model. For developers, this is usually more valuable than gaining one or two points on a single benchmark.

    Hallucination is a major obstacle for Q&A systems, document summarization, and customer-service bots. Lower hallucination directly reduces human review costs. Such a large drop also suggests that factual issues in the previous generation were quite significant.

    But note: 8.3% still means roughly one in 12 answers may contain a factual error. For high-risk scenarios involving money, contract terms, or medical/legal advice, human review is still required.

    Practical suggestions:

    1. Build your own factual evaluation set. Cover enterprise knowledge, time-sensitive information, and numerical calculation. Run regular regression tests.
    2. Combine retrieval augmentation. For fact-sensitive questions, retrieve first and answer with sources.
    3. Use tiered fallbacks. When confidence is low, ask for clarification or escalate to a human.

    3.4 Third-party composite intelligence index: look at the trend, not the rank

    A score of 55 and a fourth-place global ranking indicate that the model is in the top tier of comprehensive evaluations covering reasoning, coding, math, and knowledge. This type of index aggregates multiple dimensions into a single weighted score, which can hide individual strengths and weaknesses.

    Correct use: focus on whether the model remains consistently ahead across model generations and how much it improves over the previous generation. Rankings can fluctuate because evaluation sets change and question difficulty changes.

    For model selection, run a small blind test in your own scenarios and compare human preference win rates.

    4. Deployment threshold: what 1.56TB of weights means

    4.1 Minimum GPU memory estimate

    Public information indicates that the open-weight size is about 1.56TB, or about 0.56 bytes per parameter, which roughly corresponds to FP8-level storage precision. You can estimate the number of GPUs required with the following formula:

    1. Minimum GPU count = weight size × (1 + KV cache and runtime overhead ratio) ÷ per-GPU memory capacity

    Assume 80GB of memory per GPU and reserve an additional 25% for KV cache, activations, and communication buffers.

    • Weights alone: 1560GB ÷ 80GB ≈ 19.5, so about 20 GPUs.
    • With the 25% overhead: about 24 GPUs are needed. In production, this means at least two 8-GPU servers.

    If the model is stored in BF16, at 2 bytes per parameter, the weights alone would be about 5.6TB, requiring more than 70 80GB GPUs. This is why weight files are usually distributed in low-precision formats such as FP8.

    4.2 KV cache: the hidden cost of long context

    The KV cache grows linearly with context length. For a single request, it can be approximated as:

    1. KV cache bytes = 2 × number of layers × number of KV heads × head dimension × bytes per value × sequence length

    For example, with 100 layers, 8 KV heads, a head dimension of 128, and FP16, the KV cache for a single 1-million-token request is about 390GB.

    The decode stage is more critical: generating each token requires reading the entire KV cache again.

    • Assuming about 3TB/s of memory bandwidth per GPU, generating one token would take more than 0.1 seconds in theory.
    • Multi-GPU tensor parallelism can share this cost, but it introduces communication latency.
    • Common mitigations include grouped-query attention (GQA), KV quantization, paged KV-cache management, and speculative decoding, where a small number of draft tokens are decoded first and then verified in batches.

    Conclusion: a 1-million-token context is “supported,” not “free.” Whether a single request really needs one million tokens should be decided by business requirements, not by the parameter limit.

    4.3 Parallelism strategies and bottlenecks

    Parallelism dimension Role Main constraint
    Tensor parallelism (TP) Splits a single layer across GPUs Frequent communication; best kept within a single node with high-bandwidth interconnect
    Expert parallelism (EP) Distributes different experts across GPUs Relies on All-to-All communication; sensitive to interconnect bandwidth
    Pipeline parallelism (PP) Splits layers across GPUs or nodes Pipeline bubbles; suitable for cross-node scaling
    Data parallelism / replicas Uses multiple replicas to handle concurrency Replicas share memory-capacity pressure

    4.4 Two main deployment routes

    Route 1: Build your own cluster

    • Hardware: inference GPU memory capacity and bandwidth determine single-request experience. Inter-GPU interconnect determines multi-GPU scaling efficiency.
    • Software: use a mainstream high-performance inference engine and enable continuous batching, paged KV cache, prefix caching, and speculative decoding.
    • Topology: align the split boundaries of expert parallelism and pipeline parallelism with the physical network topology to reduce cross-node communication.
    • Observability: monitor TTFT, TPOT, throughput, GPU memory usage, and request error rates.

    Route 2: Use a managed API

    • Eliminates cluster operations, capacity planning, and version-upgrade work.
    • Suitable for scenarios with volatile traffic, highly variable context lengths, or teams without large-scale inference operations experience.
    • Pay attention to rate limits, regional availability, data residency, and compliance requirements.

    There is also a middle route: quantize or distill the 1.56TB-class weights into a mid-sized version that can run on fewer GPUs. This trades some capability for a major reduction in deployment complexity. It is suitable for edge scenarios or cost-sensitive workloads.

    5. Security trade-offs after open weights: reduced capability is intentional

    One important but often overlooked part of the public explanation is that, to release the weights, the provider proactively removed cybersecurity-related training data. Internal tests showed that after removing this data, the model’s exploit capability dropped from 90.5% to 81.0%.

    The logic is straightforward. Once weights are public, they cannot be recalled. High-risk capabilities must be reduced before release. This is very different from closed-source models, which can rely on API-side controls.

    Implications for enterprise users:

    1. Reduced capability does not mean zero risk. 81.0% is still a high proportion. A “safety-tuned” version is not a safety guarantee.
    2. Attack and defense are asymmetric. A reduction in exploit capability does not give defenders an advantage. Security teams should not rely on the model being “unable” to find vulnerabilities.
    3. Your own processes are critical. Code security audits, dependency vulnerability scanning, and penetration testing should have independent toolchains and processes.
    4. Agent permission design matters. Follow least privilege. Write operations, network access, and sensitive data access should be tiered and logged.

    This is also a typical example of post-training strategy interacting with compliance constraints. Technical capability and legal compliance are not separate issues.

    6. Licensing and compliance: open source does not mean unconditional

    The public license includes a threshold: model-as-a-service businesses with more than 100 million monthly active users or more than $20 million in monthly revenue must sign a separate agreement.

    • For individuals and small teams: there are basically no constraints. They can use, fine-tune, and deploy the model freely.
    • For large companies and commercial MaaS platforms: if user or revenue thresholds are met, a commercial licensing process is required.
    • For startups: the scale threshold should be included in compliance-cost planning. Otherwise, growth beyond the threshold can create unexpected legal and operational issues.

    Two additional points deserve attention:

    1. License terms are part of technical selection. Confirm whether commercial use is allowed, whether distillation is allowed, and whether derivative models must also be open-sourced under copyleft-like terms.
    2. Regulatory thresholds create uncertainty. At 2.8 trillion parameters, the model has crossed the 2-trillion-parameter threshold line in a draft export-control proposal. The sustainability of the open ecosystem is not only a technical issue but also a policy issue. Architecturally, design for model replaceability. For example, put model calls behind a unified interface so that the system is not deeply bound to a single model.

    7. Self-host or use an API: decision table and cost estimate

    Dimension Self-hosted deployment Managed API
    Initial investment High: multi-GPU cluster, inference engine, operations team Low: pay per token
    Marginal cost Decreases as utilization rises; clear economies of scale Grows linearly with usage
    Elasticity Slow to scale; requires capacity planning Second-level elasticity for burst traffic
    Data control Data stays inside the private network; suitable for sensitive scenarios Requires data compliance and cross-border review
    Version control Fully controllable; supports fine-tuning Follows the vendor’s iteration
    Operational complexity High: monitoring, failure handling, version upgrades Low: focus on rate limits and retries

    Cost estimate example

    Assumptions:

    • The workload requires 2.8-trillion-parameter-class model capability.
    • Self-hosting requires 24 80GB inference GPUs.
    • Cloud rental is $3 per GPU-hour.
    • The cluster runs at full capacity continuously.
    • Prefix caching is enabled, and input is effectively discounted by 50%.

    Calculation:

    • Monthly cost: 24 × 3 × 24 × 30 = $51,840.
    • If the workload uses 200 billion input tokens per month, a 50% cache discount leaves 100 billion effective input tokens, or 100,000 million tokens.
    • Input-only equivalent: $51,840 ÷ 100,000 ≈ $0.518 per million tokens, about 26% of the $2 API input price.
    • If output tokens are included, the effective cost depends on the input/output mix. In the example’s simplified calculation, the output-side estimate is about $0.346 per million tokens, and a bundled estimate is about $0.047 per million tokens.

    Conclusions and cautions

    1. Utilization determines everything. The table above assumes full load. If actual utilization is only 20%, the effective input cost rises to about $2.6 per million tokens, which is higher than the API price.
    2. Long requests reduce economies of scale. Longer contexts consume more KV cache, reducing the concurrency that the same number of GPUs can support and lowering effective token output.
    3. The bottleneck determines economics. Decode is memory-bandwidth-bound. Actual throughput and cost cannot be calculated simply from compute capacity.
    4. Hidden costs matter. Cluster operations, failure redundancy, and version-upgrade labor costs are often not lower than hardware costs.
    5. Caching is leverage. A high prefix-cache hit rate can significantly reduce input-side cost. It is worth building at the gateway layer.

    Decision recommendations

    • Low monthly volume, high volatility, and no inference-ops experience → prioritize APIs.
    • Highly sensitive data, stable large-scale usage, and fine-tuning needs → self-host.
    • Both → use a hybrid architecture: sensitive requests go to a private cluster, while burst traffic uses an API fallback.

    8. Engineering deployment checklist

    1. Model abstraction layer: unify the inference interface and hide differences between models and versions. This makes gray releases and A/B testing easier.
    2. Request routing: send simple tasks to small models and complex tasks, such as long context, multi-step agents, and multimodal workloads, to the flagship model.
    3. Context management: organize context by relevance rather than stuffing everything in. For long documents, retrieve first and summarize later.
    4. Tool calling and agent loop: set maximum steps, per-step timeout, and retry limits to prevent infinite loops.
    5. Caching and batching: prefix caching plus continuous batching are key levers for throughput and cost.
    6. Observability: track TTFT, TPOT, throughput, GPU memory, error rate, and per-request token cost across the full stack.
    7. Safety guardrails: add input/output content moderation, sensitive-data masking, tiered agent permissions, and audit logs.
    8. Evaluation regression: run your own evaluation set after every model or prompt change to prevent capability regression.

    9. Summary

    The significance of this open-source flagship is not the number “2.8 trillion.” It lies in three signals:

    1. Post-training compute has become the main battlefield for capability improvement. On the same base model, the hallucination rate dropped from 23.4% to 8.3%, and terminal-agent capability approached the strongest international closed-source flagship.
    2. The gap between open and closed models has been significantly compressed in agent scenarios. A 0.7-percentage-point gap turns “self-host or call an API” into a real cost question rather than only a capability question.
    3. The boundary of “open source” is being redefined. Pre-release removal of security-sensitive data, scale-threshold licensing, and draft export-control thresholds show that open weights are now a product of technology, commerce, and policy.

    Recommendations for developers:

    • Evaluate before selecting. Validate with your own business evaluation set instead of being persuaded by leaderboards.
    • Calculate before deploying. Do the math on GPU memory, throughput, and cost before deciding whether to self-host.
    • Isolate before opening up. Design model permissions, data boundaries, and audit trails before going live.

    Optional next step: connect the GPU-memory estimate and cost comparison in code, and put them into a team capacity-planning tool so that every model-selection decision is backed by data.