2.8T-Parameter MoE Model Open-Weight Release: Quantization-Aware Training, Local Deployment, and Agent Evaluation

The open-source community has just received a major release. A leading large model research team has fulfilled its earlier open-source commitment by officially opening up part of the complete weights of its latest-generation ultra-large Mixture-of-Experts (MoE) model. The model has approximately 2.8 trillion (2.8T) total parameters, activates roughly 104 billion parameters per token, and supports a context window of up to 1 million tokens along with image-text multimodal input. Beyond giving developers and enterprises the opportunity to directly download, fine-tune, or audit the model weights, the release has pushed the engineering realities of ultra-large MoE models—quantization strategy, local deployment, and agent capability evaluation—to the center of technical discussion.

1. Core Architecture: Sparse Attention and Fine-Grained Expert Routing

The architecture reflects the field’s current pursuit of “long context with high compute efficiency.” The model consists of 93 layers, and the most notable evolution is in its attention mechanism: the new generation replaces the traditional global multi-head latent attention with a gated sparse attention mechanism, applied across 69 layers of KDA (Key-Value Drive Attention) and 24 layers of gated DSA. The goal is to use sparser KV cache computation to substantially reduce VRAM usage and compute overhead in long-context modeling, breaking through the engineering bottleneck of long-text processing without sacrificing accuracy.

At the expert layers, the model adopts a highly fine-grained routing strategy. The expert layer contains 896 independent experts; for each input token, only 16 experts are activated, together with 2 shared experts. This extremely low activation ratio (about 1.7%) is the key to controlling compute consumption during inference. In addition, the model integrates a vision encoder of roughly 400 million parameters, giving it image-text dual-modality understanding and expanding its applicability to complex enterprise scenarios such as document parsing and chart reasoning.

2. Quantization-Aware Training (QAT) and Memory Footprint

To reduce storage and memory pressure while preserving model performance, the team adopted an aggressive quantization strategy for the weight release: weights are stored in MXFP4 format, and activations use MXFP8. The team emphasizes that this is not simple post-training quantization (PTQ) but quantization-aware training. Quantization noise is introduced during the training stage so that the model adapts to low-precision computation as it converges, resulting in almost no performance loss at inference.

Judging by the physical size of the open-source repository, the full parameter files occupy about 1.56TB of storage, and file-level parsing puts the actual parameter count at roughly 2.78 trillion. This ultra-low-precision storage format creates new engineering challenges for model loading and VRAM allocation, and it points the way for adaptation work in mainstream inference engines.

3. Local Deployment: Hardware Barriers and Inference Framework Support

For teams pursuing private, on-premises deployment, hardware configuration is the first barrier. Mainstream high-performance inference frameworks—such as open-source engines supporting distributed tensor parallelism—have quickly followed up with deployment plans and configuration guides.

Even with MXFP4’s extreme compression, however, the compute infrastructure required to load nearly 2.8 trillion parameters and maintain a KV cache for 1 million-token contexts remains extremely expensive. According to independent estimates from open-source community developers, one viable configuration is a cluster server equipped with multiple latest-generation flagship GPUs offering 288GB of VRAM per card. As an example, a single server with 8 top-tier compute cards and roughly 2.3TB of total VRAM costs between $550,000 and $650,000 in hardware alone. This means that local deployment of ultra-large MoE models remains an “exclusive game” for a small number of compute-rich leading enterprises.

Developers who want to avoid these hardware costs can call the model through the official API. The API is compatible with mainstream standard APIs, and an additional high-speed variant delivers 2 to 3 times the inference speed of the standard version. For parameter configuration, the reasoning depth can be adjusted dynamically (with low, high, and max reasoning effort options; max is enabled by default). In complex agent scenarios involving multiple rounds of tool calling, the system requires developers to pass back the previous round’s reasoning content along with the tool call records, ensuring logical continuity of the context.

4. Objective Agent Evaluation and Challenges

Benchmarks alone are no longer sufficient to assess how ultra-large models perform in real business settings; agent arena evaluations based on real user interactions have become a more meaningful reference. Such evaluation systems typically rely on tens of thousands of real long-horizon sessions and compute a “net uplift” score from multi-dimensional feedback signals, including: whether users explicitly confirm task success, how frequently users praise or complain about the model, the quality of the model’s responses after users correct its instructions, and the final execution success rate of underlying shell commands.

On the latest arena leaderboard, the open-source model ranked near the top of the overall standings and first among open-source models, thanks to its strong logical planning and orchestration capabilities. Notably, it took first place among open-source models in three core categories: workflows, coding, and everyday conversation.

Take the Bash reliability metric as an example: the model ranks first among open-source models with a net uplift of +7.9%, while its predecessor scored a negative value (-4.5%) on the same metric. This leap indicates a substantial breakthrough in the closed loop of “understanding tool intent → generating precise instructions → parsing execution results.”

5. Engineering Recommendations for Enterprise Adoption

Based on the architecture features and evaluation results above, enterprises introducing ultra-large MoE models should focus on the following engineering practices:

1. Compute and cloud infrastructure planning: Given the model’s sheer size, deploying and maintaining it on physical machines is extremely costly. Enterprises are advised to use cloud-native container platforms with elastic compute, combined with large-capacity object storage for unified weight file management. High-speed internal networks and distributed file systems on cloud platforms can accelerate weight loading synchronization across multiple compute nodes.
2. Fine-tuning and customization strategy: Full-parameter fine-tuning is practically impossible, but parameter-efficient methods such as LoRA can be used to train lightweight adapters on vertical business data while keeping the base model’s MXFP4 storage unchanged, then expose the customized model through standardized model serving interfaces.
3. Agent workflow orchestration: When building agent applications on this model, make full use of its multi-turn tool call memory. Introduce robust logging services and monitoring/alerting mechanisms at the orchestration layer, and execute model-generated tool call code (such as shell scripts) in a sandbox with compliance auditing to prevent irreversible system operations.

In summary, the release of these ultra-large MoE model weights is both an important technical contribution to the open-source community and a full-scale test of the current engineering maturity of low-precision quantization, sparse attention, and inference framework adaptation. Given the high deployment barrier, choosing the right combination of cloud compute resources, optimizing quantized deployment strategies, and building a rigorous agent evaluation system will be the key for enterprises to turn frontier large models into real productivity.