2.8T-Parameter MoE Model Goes Open Source: Quantization-Aware Training and Agent Deployment Practices

Open-source large language models have hit another milestone. A research team has delivered on its promise to release open weights, publishing its latest generation of ultra-large-scale Mixture-of-Experts (MoE) model to the open-source community. The model carries 2.78 trillion parameters in total (approximately 2.8T) and activates about 104 billion parameters per token. Notably, on a leading agent arena—evaluated across 17,500 real long-horizon conversations—the model posted a +5.1% net improvement, ranking first among open-weight models and thirteenth overall, while leading the open-source field in all three core categories: Work, Code, and Chat.

Architecture Evolution: Deep Fusion of Sparse Attention and Mixture-of-Experts

In ultra-large model design, the central architectural challenge is extending parameter count and context length while keeping computational complexity and memory footprint under control. The newly released model was deeply re-engineered at the foundational level.

The network spans 93 layers: 69 use the KDA mechanism and 24 use a gated sparse attention mechanism. Whereas the previous generation relied on full attention, the new model replaces it with gated sparse attention, using sparse mapping to substantially reduce the compute and memory cost of long-context modeling. The model supports context inputs of up to 1 million tokens and offers image-text multimodal understanding, with a vision module of roughly 401M parameters capable of handling complex interleaved image-text inputs.

At the expert layer, the model contains 896 expert networks. During inference, each token is dynamically routed to activate 16 experts, complemented by 2 shared experts. This fine-grained expert partitioning, combined with a shared-expert mechanism, delivers extreme total capacity (2.8T parameters) while keeping actual per-inference compute at the hundred-billion level (104B activated parameters)—a deliberate balance between compute cost and model performance.

Extreme Compression: Quantization-Aware Training and Mixed-Precision Storage

Storing and loading open-weight models has long been an engineering pain point. Stored in conventional FP16 or BF16, 2.8T parameters would mean extremely long downloads and peak memory demands at load time that push deployment out of reach for most teams.

The model adopts a mixed-precision quantization strategy: weights are stored in MXFP4 format, and activations use MXFP8. MXFP4 represents weights in 4 bits, paired with 8-bit activations and block-wise quantization, sharply reducing size while preserving numerical precision. More importantly, the team did not simply apply post-training quantization (PTQ) after training. Instead, it adopted quantization-aware training (QAT): the model simulated low-precision quantization error during training, allowing the weights to adapt in advance to 4-bit and 8-bit representation ranges.

The results speak for themselves. The model repository occupies about 1.56TB on Hugging Face, with Safetensors reporting approximately 2.78 trillion parameters. QAT ensures the model maintains excellent inference quality and generation stability at extremely low storage precision, laying the groundwork for production deployment.

Hardware Requirements and Inference Framework Support for Local Deployment

For teams that want to deploy, fine-tune, or security-audit the model themselves, compute assessment comes first. Although QAT and MXFP4 compress the model considerably, 1.56TB of weights still imposes steep memory requirements.

1. Hardware estimation

Beyond the weights themselves, inference requires accounting for the KV cache, activation states, and runtime overhead. With 1-million-token context support, KV cache memory growth is impossible to ignore. Based on a leading GPU server configuration, one feasible deployment plan calls for 8 high-end accelerators with roughly 288GB of memory per card (about 2.3TB in total). According to third-party hardware cost estimates, the server hardware alone runs roughly $550,000 to $650,000. This barrier means local deployment at this scale remains realistic mainly for large enterprises and research institutions with substantial compute infrastructure.

2. Inference framework support

To lower the deployment barrier, mainstream open-source inference frameworks quickly provided support. For this model, widely used engines including vLLM, SGLang, and TokenSpeed have all published detailed deployment recipes. Through optimizations such as PagedAttention and continuous batching, these frameworks further squeeze memory utilization and boost throughput under high concurrency.

A sketch of the launch configuration logic (pseudocode):

python -m inference_server.main \
    --model-path /path/to/downloaded_moe_model \
    --quantization mxfp4 \
    --kv-cache-dtype mxfp8 \
    --max-model-len 1000000 \
    --tensor-parallel-size 8 \
    --enable-long-context-support

API Integration and Multi-Turn Tool Calling in Practice

For developers without local compute, the cloud API is the better route. The API is highly compatible with mainstream OpenAI and Anthropic interface standards, keeping migration costs low. The official offering provides a standard mode and a high-speed mode, with the high-speed mode responding 2 to 3 times faster than the standard one.

For agent scenarios, the model exposes a reasoning_effort parameter with three levels—low, high, and max—defaulting to max to get the most out of its deep reasoning capability. In multi-turn tool calling, context state management is critical. Once the model decides to call an external tool in the first turn, the developer must pass the reasoning_content (reasoning process content) and tool_calls (tool invocation instructions) generated in that turn back as history messages in the next request.

Example of context assembly for multi-turn tool calling:

{
  "messages": [
    {"role": "user", "content": "What's the weather in Beijing today?"},
    {
      "role": "assistant",
      "reasoning_content": "The user wants real-time weather. I need to call the weather API to get current data for Beijing.",
      "tool_calls": [{"id": "call_01", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Beijing\"}"}}]
    },
    {
      "role": "tool",
      "tool_call_id": "call_01",
      "content": "{\"temp\": 22, \"condition\": \"Sunny\"}"
    }
  ],
  "reasoning_effort": "max"
}

This mechanism lets the model “remember” its previous reasoning chains and intended actions across turns, preventing tool-calling loops or hallucinations caused by broken context.

Real-World Agent Evaluation and Technical Takeaways

Traditional benchmarks can no longer fully measure how large models perform in real business scenarios. The agent arena draws on real user conversations to provide a far more meaningful evaluation dimension. It goes beyond generation quality, reading deeper signals of task execution success: whether users explicitly confirm the task succeeded, praise or complaints about the model, how well it responds after user corrections, and—most critically—whether shell commands execute successfully. The “net improvement” metric computed from these signals directly reflects a model’s reliability across complex tool chains.

The latest data shows the model achieved a remarkable +7.9% improvement in Bash command execution reliability, taking first place among open models (its predecessor scored -4.5% on this metric and ranked 13th). Behind that number lies a leap in code generation accuracy, format compliance, and logical rigor. For enterprise users, it translates into stronger “first-pass success” and error-correction capability when wiring large models into automated operations, CI/CD pipelines, or complex data-processing workflows.

Engineering Recommendations for Enterprise Adoption

For enterprises adopting an open model of this scale with strong agent capabilities, a rigorous strategy is essential:

1. Compute assessment and cloud-native deployment. Teams without sufficient compute can start by relying on cloud providers’ heterogeneous computing platforms and container services (e.g., Baidu AI Cloud) for elastic deployment. Elastic GPU instances let enterprises scale compute up and down with business peaks and valleys, avoiding heavy fixed hardware investment. Object storage services can host the 1.5TB+ of model weights, ensuring efficient and stable distribution across multiple nodes.

2. Build an evaluation system. Although the model performs well in long-horizon conversations, every enterprise’s business scenarios are unique. After integrating the API or deploying locally, build evaluation sets based on your own business data. In particular, stress-test tool-calling success rates, JSON output format compliance, and multi-turn context retention.

3. Security and compliance auditing. For sectors with strict data security requirements, such as finance and government, local deployment with open weights is the first choice. Observability tools and logging services can audit the model’s inputs, outputs, and tool-calling chains end to end, keeping agent behavior controllable and traceable.

Conclusion

The release of a 2.8T-parameter MoE model demonstrates not only the engineering maturity of quantization-aware training and sparse attention, but also a new high-water mark for open-model reliability in agent workloads. Developers and enterprises embracing these advanced models need a full-chain strategy—from evaluation to deployment to application—grounded in their own compute infrastructure, business requirements, and engineering capabilities to truly unlock the productivity potential of large models.