Open-Weight Release and Local Deployment Engineering for a 2.8-Trillion-Parameter MoE Model

Open-Weight Release and Local Deployment Engineering for a 2.8-Trillion-Parameter MoE Model

A leading AI team has followed through on its earlier open-source commitment by publishing the weights of its latest mixture-of-experts model, which has a total parameter count of 2.8 trillion. Beyond its massive scale, the model has also topped the open-weight leaderboard in real-user agent capability evaluations, achieving a notable net improvement rate. This release marks a key step in moving very large MoE models from closed development to open availability, while giving enterprise AI developers deeper opportunities for customization and auditing.

1. Model Architecture: Merging Sparse Attention with Mixture of Experts

The publicly released model card reveals substantial engineering complexity and architectural innovation. The base architecture is a 93-layer Transformer variant designed to balance long-context understanding with computational efficiency.

1.1 Differentiated Attention Layers
Across the 93 layers, the model uses 69 KDA layers and 24 Gated DSA layers. Unlike the previous generation, which relied entirely on a full attention mechanism, this version introduces Gated DeepSeek Sparse Attention. The primary goal is to greatly reduce computational complexity and memory usage during long-context modeling. A gating mechanism dynamically determines which tokens need to participate in global attention, allowing the model to support million-token contexts and multimodal image-text inputs while maintaining efficient inference.

1.2 Expert Routing and Sharing
The expert layer employs 896 experts. At inference time, each token dynamically activates 16 experts, with an additional 2 shared experts always engaged. This design of sparse activation plus shared knowledge keeps the total parameter count at 2.8T while limiting activated parameters per token to approximately 104B, achieving a strong balance between model capacity and inference cost.

1.3 Multimodal Vision Fusion
To support image-text inputs, the model includes a vision encoder module with roughly 401M parameters. It transforms image features into vector sequences with the same dimensionality as text tokens, enabling early fusion of visual and language features in the underlying representation space.

2. Mixed-Precision Quantization and Storage Strategy

Distribution and deployment of extremely large models have long been constrained by storage and bandwidth bottlenecks. The released weights demonstrate a forward-looking approach to data types and quantization.

2.1 Quantization-Aware Training
The weights are not post-training quantized. Instead, they are stored directly in MXFP4 format, with activations using MXFP8 precision. The official documentation states that the model underwent quantization-aware training. This means simulated quantization was introduced during the forward pass of training, allowing the model to adapt to low-precision information loss during convergence. As a result, inference precision is largely preserved after deployment.

2.2 Storage and Parameter Count
According to the public repository metadata, the model occupies approximately 1.56TB of storage. The actual parameter count calculated from safetensors files is about 2.78 trillion. A single file of this size places extreme demands on network stability and filesystem support during download.

3. Engineering Requirements for Local Deployment

Downloading and deploying a model at this scale imposes serious demands on enterprise compute infrastructure.

3.1 Hardware Resource Estimation
According to third-party estimates from hardware specialists, building an inference server capable of running this model smoothly requires at least 8 top-tier GPUs with very high memory bandwidth, totaling roughly 2.3TB of VRAM. Overall hardware procurement cost is estimated between $550,000 and $650,000. This indicates that local deployment of very large MoE models remains feasible mainly for a small number of well-resourced enterprises.

3.2 Inference Framework Support
To ease deployment, mainstream open-source inference engines provide their own deployment recipes. The following is an illustrative configuration for setting up an inference service with common frameworks such as vLLM or SGLang:

# Example launch command; actual parameters depend on the inference framework version
python -m inference_server.entrypoints.openai.api_server \
    --model-path /path/to/downloaded_model \
    --quantization moe_mxfp4 \
    --moe-router-top-k 16 \
    --max-model-len 1000000 \
    --tensor-parallel-size 8 \
    --trust-remote-code

In practice, enterprises typically rely on container platforms to manage complex dependencies and use object storage services to host the 1.56TB of weight files for efficient multi-node distribution.

4. API Integration and Multi-Turn Tool Calling

For teams without local deployment capacity, the provider offers an API service that is highly compatible with standard interfaces. Model names are available in both standard and high-speed versions, with the high-speed variant offering 2-3x faster responses.

4.1 Reasoning Effort Control
The API supports a reasoning_effort parameter with values low, high, and max, defaulting to max. Developers can adjust this based on task complexity: lower settings reduce token consumption and latency for simple Q&A, while maximum settings are retained for complex logical reasoning.

4.2 Multi-Turn Agent Calls with Reasoning Chains
As a model that ranked first in agent capability testing, its multi-turn tool-calling mechanism is carefully designed. In multi-turn tool calls, developers should not pass back only the tool execution result from the previous round. They must also include the reasoning_content generated by the model in the previous turn along with the tool_calls instructions as part of the context.

The following example shows a standard context structure for multi-turn tool calling:

{
  "messages": [
    {"role": "user", "content": "Check today's weather in Beijing and analyze whether it is suitable for outdoor running"},
    {
      "role": "assistant",
      "reasoning_content": "The user needs two steps: 1. Get weather data; 2. Give a running recommendation based on the data. First call the weather API.",
      "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Beijing\"}"}}]
    },
    {
      "role": "tool",
      "tool_call_id": "call_1",
      "content": "{\"temperature\": 15, \"aqi\": 45, \"weather\": \"Clear\"}"
    }
  ],
  "reasoning_effort": "max"
}

This design preserves the reasoning chain, allowing the model to continue from its earlier thought trajectory in the next generation step and greatly improving logical coherence in long-running agent tasks.

5. Agent Evaluation Framework and Measured Performance

The model has attracted considerable attention because of its strong performance in real-session agent evaluations. Unlike traditional static benchmarks, the evaluation system was built on a large volume of real long-session user conversations, with more than 17,500 sessions.

5.1 Evaluation Signals
The system does not rely only on direct user feedback such as likes or complaints. It also analyzes deeper signals during interaction: whether the user explicitly confirms task success at the end of the session, how much the model improves after the user corrects an instruction, and whether shell commands generated by the model actually execute successfully in a sandbox environment. The net improvement score calculated from these multidimensional signals reflects the model’s agent capability in real production scenarios more accurately.

5.2 Key Results
With maximum reasoning effort enabled, the model achieved an overall net improvement of +5.1%, ranking first among open-weight models and thirteenth across all models including closed-source ones. It ranked first among open-weight models in the code, work, and chat subcategories. Particularly notable is the Bash reliability metric, where the model reached +7.9%, completely reversing the negative performance of the previous generation (-4.5%). This demonstrates strong reliability in complex code generation and terminal command execution scenarios.

Conclusion

The full open-weight release of a very large MoE model provides the industry with a valuable window into top-tier AI architectures. From quantization-aware training to gated sparse attention and multi-turn tool calling with deep reasoning chains, these engineering practices offer a useful reference for developers building enterprise-grade agent applications. Given the high local deployment threshold, enterprises can choose between local inference and cloud API access based on their available compute resources, and can use platforms such as Baidu AI Cloud for container orchestration, object storage, and monitoring to build stable and efficient AI inference infrastructure.