2.8T-Parameter MoE Model Released with Open Weights: Architecture Deep Dive and Deployment Guide

As large language models continue to advance, trillion-parameter mixture-of-experts (MoE) models are moving from theoretical research into real engineering practice. A research team has recently fulfilled its open-weights commitment by pushing a complete MoE model with 2.8 trillion total parameters to a public model hosting platform. The model ranks first among open-weight models in evaluations based on real user Agent tasks. Setting aside product-specific details, this article analyzes the technical characteristics and engineering deployment guidance for this class of ultra-large models across four dimensions: underlying architecture, quantization and storage, local deployment requirements, and the Agent evaluation mechanism.

1. Core Network Architecture and Multimodal Support

To understand and deploy a 2.8T-parameter model, you first need to break down its network structure. The model consists of 93 layers in total, with a deeply optimized combination of attention mechanisms and expert layers.

1. From Full Attention to Sparse Attention

The model contains 69 KDA attention layers and 24 Gated DSA (Gated DeepSeek Sparse Attention) layers. Compared with the full attention mechanisms widely used in previous-generation architectures (such as Gated MLA), this generation introduces gated sparse attention layers. With full attention, computational complexity and memory usage grow exponentially when processing million-token contexts; sparse attention, by contrast, substantially reduces the compute cost of long-context inference while preserving long-text modeling accuracy, making a 1-million-token context window practically feasible in real deployments.

2. Mixture-of-Experts Routing Strategy

Most of the model’s parameters are concentrated in its MoE layers. The architecture includes 896 expert networks. For each input token, the routing mechanism dynamically selects 16 experts for activation, complemented by 2 shared experts. The shared experts ensure stable extraction of general knowledge, while the dynamically selected experts capture fine-grained features of specific tokens. This design—2.8T total parameters with only 104B activated per token—is the core mechanism for maintaining extremely high capacity while keeping inference latency under control.

3. Vision-Language Multimodal Integration

Beyond text-only processing, the model natively supports image-and-text input. Its vision component integrates a visual encoder of roughly 401M parameters (the V2 generation). This vision module converts images into high-dimensional feature sequences aligned with the text representation space, enabling the model to seamlessly handle complex instructions that mix images and text.

2. Quantization-Aware Training and Storage Engineering

Another major challenge in open-sourcing an ultra-large model is weight storage and distribution. Stored in conventional FP16 or BF16 format, 2.8T parameters would be enormous, and most development teams could not absorb the download and memory overhead.

1. Mixed-Precision Quantization Strategy

The model adopts an advanced mixed-precision quantization scheme: weights are stored in MXFP4 format, while activations are computed in MXFP8. Unlike conventional pure post-training quantization with INT4 or INT8, the team explicitly confirmed the use of quantization-aware training (QAT). Quantization noise is introduced during pretraining or fine-tuning, so the model actively adapts to low-precision environments during backpropagation. This strategy effectively compensates for the precision loss introduced by FP4 and ensures inference quality at low bit widths.

2. Weight Size and Distribution

With this quantization, the repository on the hosting platform occupies about 1.56TB, and the underlying parameter count based on the safetensors files is approximately 2.78 trillion. For distribution, a 1.56TB download requires high-bandwidth network conditions on the consumer side and places extreme demands on the platform’s object storage and CDN delivery capabilities.

3. Local Deployment Hardware Requirements and Inference Framework Support

Given a 1.56TB model repository, the hardware threshold for local deployment is very high. Developers who want to self-host, fine-tune, or audit the model need rigorous hardware cost planning.

1. Hardware Configuration Estimates

Third-party engineering experts have produced estimates of feasible deployment configurations. Because weights are stored in MXFP4 with MXFP8 activations, plus the memory required for a 1-million-token KV cache, a workable server configuration needs roughly 2.3TB of total accelerator memory. That means at least 8 flagship AI accelerator cards (the estimates reference B300-series cards with around 320GB of memory each). At current hardware market prices, a single such server costs roughly $550,000 to $650,000. This is clearly enterprise-grade compute territory, well beyond what ordinary developers can host on a single machine.

2. Inference Frameworks and Deployment Recipes

Major open-source inference frameworks have already adapted to this architecture and published concrete deployment recipes:

• Mainstream inference engines: Widely used high-performance frameworks such as vLLM, SGLang, and TokenSpeed all provide loading and optimization solutions for this MoE architecture, supporting memory-optimization techniques like PagedAttention.
• API-based deployment: Developers who prefer not to bear high local hardware costs can use the official API service. The API is highly compatible with mainstream communication interface standards and is offered in both a standard version and a high-speed version (2–3x faster).
• Reasoning depth control: The API supports a reasoning_effort parameter that can be set to low, high, or max (the default), allowing developers to balance reasoning depth against real-time requirements.

3. State Management for Multi-Turn Tool Calling

For Agent developers, multi-turn tool calling is one of the model’s highlights, but it also raises the bar for context management. During multi-turn function calling, developers must pass the previous turn’s reasoning_content (the reasoning process) and tool_calls (tool invocation instructions) back to the model verbatim. This ensures that the model retains a complete memory of the task state and its own chain of thought across multiple conversation turns.

4. Inside the Real-Session Agent Evaluation System

The model ranks first among open models and thirteenth overall on the leaderboard. This result comes not from traditional static benchmarks (such as MMLU) but from a real-world Agent Arena evaluation system. Evaluation based on real human feedback reflects practical usability in actual business scenarios far better than static tests.

1. Signal Extraction Mechanism

Agent Arena data is drawn from real user Agent sessions. The system assesses actual capability through multiple signals:

• Task success confirmation: whether the user explicitly confirms at the end of a session that the task was resolved.
• Sentiment feedback analysis: extracting praise and complaint text from the interaction process.
• Error-correction response: the quality and efficiency of the model’s response when a user points out a mistake and requests a fix.
• Shell execution reliability: in code-execution tasks, whether the Bash or other shell commands generated by the model run successfully in a real environment without destructive operations.

2. The Quantitative Metric: Net Improvement

Based on these signals, the platform computes a composite metric called “Net Improvement,” which reflects the model’s positive contribution rate on real tasks relative to a baseline. According to an analysis of more than 17,500 real long-horizon sessions, the model (with Max reasoning mode enabled) achieved a +5.1% net improvement, ranking first among open models in all three core categories: Work, Code, and Chat. The previous-generation model achieved only +0.91%. Notably, in the coding and automated operations subcategories, its Bash reliability metric reached +7.9% (versus -4.5% for the previous generation)—a sign that ultra-large MoE models have crossed the practicality threshold in complex logical reasoning and code generation.

Conclusion

The open release of this 2.8T-parameter MoE model demonstrates the engineering maturity of quantization-aware training and sparse attention mechanisms, while further raising the hardware bar for local deployment of ultra-large models. For enterprise users, combining mature cloud-native compute platforms with APIs compatible with mainstream protocols is the fastest path to accessing such cutting-edge model capabilities. Guided by Agent evaluation systems, developers should focus on reliability in real multi-turn interactions and tool calling, and use it to build intelligent applications that genuinely solve complex business problems.