2.8T-Parameter MoE Model Open-Sourced: Architecture, Quantization, and Local Deployment Explained

As large language models push parameter counts toward the trillion scale, the Mixture-of-Experts (MoE) architecture has become the industry consensus for balancing compute costs against model performance. Recently, a leading model team made good on its open-source promise by releasing the full weights of its newest-generation MoE model, which totals 2.8 trillion parameters. The model not only supports million-token contexts and multimodal input, but also tops the open-model rankings in real-world agent conversation evaluations. This article takes a deep look at the model’s technical core from four angles: architecture design, quantization strategy, local deployment, and agent evaluation.

Ultra-Large MoE Architecture and the Shift to Sparse Attention

At 2.8T total parameters, the compute cost of every forward pass is enormous. The model adopts a typical sparse activation strategy: each token activates only about 104B parameters, effectively controlling inference compute. According to the model card, the network is 93 layers deep, comprising 69 KDA layers and 24 Gated DSA layers.

Compared with the previous generation, the model makes a major change in long-context modeling: it replaces the full-attention Gated MLA layers with an industry-leading sparse attention mechanism. This evolution targets the core pain point of ultra-long contexts—memory usage growing quadratically with sequence length. With sparse attention, the model significantly reduces KV cache memory pressure when processing million-token sequences, making long-text inference and long-horizon task planning far more feasible in engineering terms.

The model also integrates a vision encoder of roughly 401M parameters, supporting joint image-text input and extending its applicability in multimodal scenarios. Its expert layers contain 896 expert networks; each token is dynamically routed to 16 activated experts, supplemented by 2 shared experts, striking a balance between general knowledge capture and domain-specific specialization.

Quantization-Aware Training and Storage Optimization

For a 2.8T-parameter model, weight storage and loading are the first barrier to local deployment. The open-sourced weights do not use traditional FP16 or BF16 formats. Instead, they adopt an aggressively compressed mixed-precision quantization scheme: weights are stored in MXFP4 format, while activations use MXFP8.

Notably, the team emphasizes that the model was trained with quantization-aware training (QAT) rather than post-training quantization (PTQ). This means the model simulated low-precision information loss during training itself, with network weights actively adapting to quantization noise throughout backpropagation. Compared with PTQ, QAT better preserves performance on complex reasoning tasks and effectively suppresses quantization-induced degradation.

Even with 4-bit storage, the sheer parameter count means the Hugging Face repository totals roughly 1.56TB, with Safetensors reporting about 2.78 trillion actual parameters. This storage strategy presents developers with new challenges—and opportunities—in memory allocation and bandwidth utilization.

Local Deployment: Hardware Requirements and Inference Framework Support

With weights at the 1.5TB level, the hardware bar for local deployment is extremely high. According to third-party developer estimates, one viable configuration requires 8 flagship AI accelerator cards and at least 2.3TB of high-speed memory. At current hardware prices, a single such server costs roughly $550,000 to $650,000. This sends a clear signal: local inference of ultra-large MoE models remains the preserve of a small number of compute-rich enterprises.

On the inference framework side, mainstream high-performance inference engines have all published targeted deployment plans. To load and schedule MoE models efficiently, developers typically rely on distributed inference and tensor parallelism:

# Pseudocode: distributed MoE loading logic based on a mainstream inference engine
from inference_engine import AsyncLLMEngine, MoEConfig
# Configure quantization and parallelism parameters
model_config = MoEConfig(
    model_path="./models/moe-2.8t-mxfp4",
    quantization="mxfp4",
    activation_quantization="mxfp8",
    tensor_parallel_size=8,  # 8-way tensor parallelism
    max_model_len=1000000,   # million-token context
    trust_remote_code=True
)
engine = AsyncLLMEngine.from_model_config(model_config)

In practice, enterprises typically rely on container platforms for environment isolation and resource scheduling, combine object storage services for fast distribution and version control of large weight files, and use monitoring and alerting systems to track GPU memory and compute utilization in real time.

API Integration and Multi-Turn Tool Calling

For developers without the means to deploy locally, the cloud API is the more practical option. The model offers a standard API interface that is fully compatible with mainstream LLM API protocols, allowing developers to switch over seamlessly.

Several technical details in the API design deserve close attention. First is control over reasoning depth: via the reasoning_effort parameter (low, high, or max; default max), developers can trade off response speed against logical depth based on their use case. Second, an accelerated endpoint is available, officially claimed to be 2 to 3 times faster than the standard version, suited to latency-sensitive, high-concurrency scenarios.

In agent scenarios, multi-turn tool calling is a core capability, and the API enforces strict conventions around it. In multi-turn interactions, developers must pass the model’s previous-turn reasoning_content (the reasoning process) and tool_calls (tool invocation instructions) back as context. This design ensures the model maintains a complete chain of thought and state memory when executing chained tasks, avoiding tool mis-calls caused by fragmented context.

Agent Evaluation: From Static Benchmarks to Real-World Signals

High scores on static benchmarks no longer fully reflect real-world business performance. The model ranks first among open models and 13th overall on the Agent Arena leaderboard—an evaluation system with considerable engineering reference value.

Agent Arena’s evaluation data comes entirely from real user agent conversations. Its scoring goes beyond traditional text-similarity comparison, computing a “net improvement” from multiple real-world signals:

1. Task success confirmation: whether the user explicitly confirms the task was completed;
2. Sentiment feedback: capturing user praise and complaints about the model;
3. Error-correction response: response quality after the user corrects an instruction;
4. Execution reliability: whether low-level operations such as shell commands actually succeed.

Based on statistics from 17,500 real long-horizon conversations, the model achieved a net improvement of +5.1%. Within subcategories, it ranks first among open models in code execution, work tasks, and daily conversation. Notably, it posted a +7.9% net improvement on Bash reliability. This shows that, combined with QAT and sparse attention, ultra-large MoE models are not only strong on theoretical metrics but also highly reliable in engineering terms for code execution and agent tasks that demand rigorous logic and state tracking. For teams building enterprise-grade agents, evaluation data based on real conversations is far more useful for decision-making than parameter counts alone.