2.8T Parameter MoE Model Open Sourced: Architecture Evolution and Local Deployment Guide

Architectural Analysis: Design and Evolution of the 2.8T MoE Model

Recently, an AI research team fulfilled its open-source commitment by releasing the full weights of a 2.8 trillion parameter Mixture-of-Experts (MoE) model to the community. Beyond its industry-leading parameter scale, the model ranks first among open-weight models on an Agent capability leaderboard based on real user interactions, showing a significant net improvement rate. This release provides a valuable infrastructure option for enterprise engineering teams that require autonomous control over model weights for deep fine-tuning or security auditing.

According to the model card, the model utilizes a highly optimized MoE architecture. The network depth reaches 93 layers, consisting of a mixture of 69 KDA layers and 24 Gated DSA layers. The most critical architectural evolution from its predecessor is the replacement of the fully attentive Gated MLA layers with Gated DeepSeek Sparse Attention. This improvement aims to reduce VRAM and compute consumption during ultra-long context modeling, enabling native support for a 1 million Token context window while seamlessly processing multimodal inputs (image and text). For expert routing, the model integrates 896 expert networks, activating only 16 per Token alongside 2 shared experts, effectively balancing parameter scale with inference efficiency. The vision component integrates a lightweight vision encoder with approximately 401M parameters.

Quantization and Storage: Extreme Compression for Local Deployment

Open-sourcing ultra-large models typically imposes high storage and loading barriers. To mitigate this, the model’s weight release employs an aggressive hybrid quantization scheme: weights are stored in MXFP4 format, while activations utilize MXFP8. Officially, this is not standard Post-Training Quantization (PTQ) but incorporates Quantization-Aware Training (QAT).

The core logic of QAT allows the model to “perceive” precision losses during the training phase, adjusting weight distributions during backpropagation to maximally preserve high-frequency features and inference accuracy. According to physical metrics from the open-source repository, this compression strategy reduces the storage footprint to approximately 1.56TB, with an actual measurable parameter count of about 2.78 trillion. This design significantly lowers VRAM usage and loading latency while maintaining robust inference capabilities.

Local Deployment Guide: Hardware Configuration and Inference Framework Adaptation

For teams planning to deploy this model locally, hardware thresholds represent a significant challenge. Based on third-party developer estimates, a viable hardware configuration to run an MoE model of this scale smoothly requires 8 high-performance computing accelerator cards (e.g., B300-level), totaling approximately 2.3TB of VRAM. The overall hardware investment ranges between $550,000 and $650,000. This configuration targets large enterprises or specialized computing centers rather than individual developers.

Regarding the software ecosystem, mainstream open-source inference engines have quickly provided corresponding deployment recipes. Developers can adapt the model using the following general workflow:

1. Environment Preparation and Weight Retrieval: Ensure local high-speed storage with sufficient capacity. Since the repository exceeds 1.5TB, an SSD array is recommended to accelerate model loading. Download the weight files via Hugging Face compatible interfaces.
2. Inference Engine Configuration: Current mainstream efficient inference frameworks (such as vLLM, SGLang, etc.) support MoE architecture routing and VRAM partitioning. For example, deployment scripts must explicitly specify Tensor Parallelism (TP) and Expert Parallelism (EP):

   # Example: General command-line arguments for starting the inference service
   python -m inference_server.serve \
    --model-path /path/to/2.8t-moe-model \
    --tensor-parallel-size 8 \
    --enable-expert-parallel \
    --max-model-len 1000000 \
    --quantization mxfp4

3. API Integration and Multi-Turn Tool Calling: To avoid heavy local deployment costs, developers can directly call the official API, which is compatible with mainstream OpenAI and Anthropic interface standards. In Agent scenarios, multi-turn tool calling is a core requirement. Developers must note that when passing the execution results of the previous tool back to the model, the reasoning_content (reasoning process) and tool_calls (tool call records) generated in the previous turn must be included to ensure the continuity of the model’s context state. Additionally, the API supports the reasoning_effort parameter, which can be set to low, high, or max, defaulting to max for maximum reasoning capability. A high-speed version offering approximately 2x to 3x faster inference speeds is also available for latency-sensitive scenarios.

Agent Evaluation: Real-World Session Signals and Reliability Analysis

In the Agent Arena real-user session evaluation, the model achieved first place among open-source models and thirteenth overall. Its net improvement metrics are highly valuable for analysis. Unlike traditional static benchmarks, Agent capability evaluation is entirely based on real user Agent interaction sessions.

The evaluation system primarily extracts the following signals: whether the user explicitly confirms task success, user praise or complaints, model responses after user corrections, and whether the finally generated Shell command executed successfully. The “net improvement” calculated from these signals objectively reflects the model’s usability in actual engineering. Based on statistical data from 17,500 real long-context sessions, the model achieved an overall net improvement of +5.1%, with a notable +7.9% reliability in Bash command execution (compared to -4.5% in the previous generation). This indicates a qualitative leap in code generation, logical reasoning, and tool-calling loops. The model no longer just produces code-like text but possesses the engineering reliability to execute on servers and solve practical problems.

In conclusion, the open-sourcing of this 2.8T parameter MoE model marks a new stage for open-source large models in terms of Agent capabilities and ultra-long context processing. Enterprise teams with sufficient computing resources can leverage container platforms and object storage services, such as those provided by Baidu AI Cloud, to build a secure and efficient private model inference foundation. Meanwhile, smaller teams can quickly integrate these cutting-edge capabilities via standardized API interfaces and multi-turn tool-calling specifications to drive business innovation.