2.8T MoE Model Weights Open-Sourced: Local Deployment, Architecture Analysis, and Inference Practices

1. Background: Open-Sourcing a 2.8T Parameter Model

Recently, the complete weights of a Mixture-of-Experts (MoE) model with approximately 2.8 trillion total parameters and around 104 billion activated parameters per token were hosted in a public model repository. For developers and enterprise engineering teams, this means the model structure, parameters, configurations, and inference code can be directly accessed for local deployment, fine-tuning, auditing, or secondary development.

This release was anticipated. During the model’s initial launch, the official team committed to releasing the weights by a specific date. As scheduled, the repository went live on November 4th. Alongside the weights, an Agent capability evaluation based on real-world, long-horizon user tasks was published. The results ranked this model 1st among open-weight models and 13th overall.

From a technical perspective, the key takeaways of this release go beyond simply downloading a model:

• Weights are stored in a low-bit format, resulting in a large file size.
• The architecture features a hybrid design of multiple attention mechanisms.
• Initial adaptations for deployment frameworks are available.
• API users have options for different speed tiers and reasoning intensities.
• Agent scenarios show quantifiable net improvement metrics.

Let’s explore these dimensions in detail.

---

2. Weight Scale and Storage Format

Looking solely at the “2.8T parameters” metric, many teams might underestimate the engineering implications. During actual download and deployment, the primary bottleneck is disk usage.

The repository currently occupies about 1.56TB, with weight files stored in MXFP4 format and activation computations utilizing MXFP8. This indicates that weights were quantized to low bits for storage, while activations use higher precision during inference to closely match the original training precision.

Official documentation mentions that Quantization-Aware Training (QAT) was used during the model’s training phase. This means the quantization process wasn’t a simple post-training compression; instead, quantization errors were introduced during training to adapt the parameters to the subsequent low-bit inference environment. This approach typically mitigates capability loss from quantization but adds complexity to the training pipeline.

For developers, the critical points to understand are not just “4-bit weights,” but rather:

1. The 1.56TB repository size demands substantial storage and bandwidth.
2. VRAM requirements far exceed the capacity of a single GPU.
3. The inference framework must correctly support MXFP4 de-quantization and MXFP8 activation computation.
4. Multi-GPU parallelism is a fundamental prerequisite, not an optional optimization.

Statistics show that the safetensors files contain approximately 2.78 trillion parameters, aligning closely with the official 2.8T figure. Minor discrepancies primarily arise from naming conventions, embedding layers, or the calculation scope of vision modules.

---

3. Model Architecture: MoE, Hybrid Attention, and Vision Modules

Structurally, this is not a traditional dense Transformer. It consists of 93 layers:

• 69 layers utilize the KDA architecture.
• 24 layers utilize the Gated DSA architecture.

In the previous generation, full attention mechanisms were concentrated in Gated MLA layers. In this version, these have been replaced with Gated DeepSeek Sparse Attention to reduce modeling costs in long-context scenarios. The model supports a context length of up to 1 million tokens and features multimodal (image-text) input capabilities.

The expert layer contains 896 experts, activating 16 experts per token, alongside 2 shared experts. This is a standard MoE design approach:

• A large number of experts increases model capacity.
• Activating only a subset of experts per token controls inference costs.
• Shared experts capture common features, reducing instability caused by routing.

The vision module is MoonViT-V2, containing 401M parameters. Although the vision parameter count is small compared to the entire model, in image-text tasks, the tokens output by the vision encoder are concatenated with text tokens before entering subsequent layers. Therefore, the engineering implementation must handle modality concatenation, positional encoding, and sequence length management.

From an architectural perspective, the core challenges are not just the layer count, but:

• Switching between hybrid attention strategies across different layers.
• Expert routing and load balancing.
• VRAM and computation planning for sparse attention under long sequences.
• Gradient flow and fine-tuning strategies between the vision encoder and the text backbone.

For enterprise teams looking to fine-tune, it is crucial to determine before experimentation whether the vision modality will be frozen, which layers are trainable, and whether expert routing can be adjusted.

---

4. Deployment Frameworks and Hardware Barriers

Currently, several mainstream inference frameworks offer deployment solutions, such as vLLM, SGLang, and TokenSpeed. Their adaptations typically include:

• Weight loading and model structure registration.
• MXFP4/MXFP8 computation support.
• Tensor parallelism and pipeline parallelism configurations.
• Long-context KV cache management.
• Dynamic batching and preemptive scheduling.

Even with framework support, the hardware barrier remains exceptionally high. Third-party community estimates suggest a viable configuration: a server equipped with 8 B300 accelerator cards totaling approximately 2.3TB of VRAM. The hardware cost is estimated between $550,000 and $650,000. It is important to note that this is a community estimate, not an official recommendation, as no official hardware procurement standards have been released.

The rough estimation logic is as follows:

• Weights require about 1.56TB; even compressed as MXFP4, some structures typically need to be decompressed to higher precision after loading.
• Activation computations require additional VRAM.
• KV cache for a 1-million-token context grows significantly with batch size.
• Multi-GPU communication requires high-bandwidth interconnects.

Therefore, real-time local inference is impractical for average small-to-medium teams. More common choices include:

1. Using online APIs for complex tasks.
2. Utilizing local deployment strictly for offline batch processing.
3. Fine-tuning or distilling business data for domain-specific tasks.
4. Running only with limited context lengths to reduce KV cache pressure.

Scenarios genuinely suited for local deployment are typically restricted to enterprises with strict data security requirements, highly stable task volumes, and existing cluster resources.

---

5. API Integration and Reasoning Intensity Control

For teams looking to avoid maintaining massive clusters, API integration is the more pragmatic path. Currently, two primary model variants are offered:

• Standard version.
• High-speed version, officially stated to be 2 to 3 times faster than the standard version.

The API is compatible with mainstream chat completion formats, requiring minimal changes to existing call logic. A key control parameter is:

reasoning_effort: low | high | max

The default value is max. This parameter adjusts the reasoning intensity, affecting the volume of generated content and the time taken before providing a final answer. Different tasks can be configured based on specific needs:

• Simple classification, information extraction: low.
• Standard Q&A, code generation: high.
• Complex multi-step planning, long document analysis: max.

In multi-turn tool-calling scenarios, developers must pay close attention to context passing rules. After the model generates reasoning content, subsequent requests must pass back both the previous reasoning content and the tool execution results. Otherwise, the model may lose intermediate reasoning states, degrading the quality of subsequent decisions.

A typical request can be abstracted as follows:

Input messages:
  - User question
  - Previous round assistant reasoning content
  - Previous round tool call results
Output parameters:
  - reasoning_effort = max
  - Tool definition list

While this design is familiar to developers, it is easy to omit fields within an Agent chain. The crucial step is verifying that every request retains the complete history, rather than passing only the tool return values.

---

6. Agent Capability Evaluation: From “Correct Answers” to “Task Completion”

Traditional model evaluations primarily measure answer accuracy, but the core of Agent tasks is “whether the user’s goal was achieved.” Alongside the weights, an evaluation based on approximately 17,500 real-world long-horizon user sessions was released.

Evaluation signals included:

• Explicit user confirmation of task success.
• User expressions of praise or complaint.
• Effective model adjustment after user corrections.
• Successful execution of shell commands.

These signals were consolidated into a “net improvement” metric. The current version achieved a net improvement of +5.1%. For comparison, the previous version was +0.91%, and its ranking among open-weight models improved from 6th to 1st.

In terms of specific sub-categories:

• Ranked 1st among open-weight models in Work, Code, and Chat categories.
• Bash reliability reached +7.9%, up from -4.5% in the previous version, improving its rank from 13th to 1st.

This shift indicates a significant reliability improvement in executable tasks. However, this should be viewed objectively:

1. These results are based on specific evaluation criteria.
2. Real production environments may have different task distributions.
3. Agent capabilities are also influenced by toolchains, execution environments, and prompt design.
4. The net improvement metric is better suited for horizontal comparison rather than judging absolute success rates.

For developers, a more valuable approach is to use the same prompts and tool environments on their own task sets to compare task completion rates and costs under different reasoning intensity settings.

---

7. Practical Recommendations: Evaluating Adoption

Based on the information above, teams can make decisions across four dimensions:

1. Task Complexity

If the core business involves long-document analysis, multi-step tool calling, or code generation and debugging, high reasoning intensity generally yields more stable results. However, for simple Q&A or information extraction, the standard-speed version may offer better cost efficiency.

2. Data Compliance Requirements

When data cannot leave the corporate intranet, local deployment becomes a necessity. In these cases, prioritize evaluating:

• Availability of approximately 2TB of storage and multi-GPU VRAM.
• Tolerance for lower concurrency.
• Internal capability to maintain inference frameworks and manage weight upgrades.

3. Cost Structure

Local deployment requires high upfront hardware investment, making it suitable for long-term, continuous, large-scale invocations. API integration offers more flexibility for scenarios with fluctuating demands. A viable strategy is to establish baseline performance using APIs while validating local deployment feasibility in a medium-scale shadow environment.

4. Tech Stack Adaptation

If the team already uses mainstream inference frameworks, the deployment barrier primarily involves resource allocation and configuration tuning. If the team’s tech stack is highly customized, a longer adaptation period should be anticipated.

---

8. Conclusion

The release of a 2.8T parameter MoE model’s weights marks a significant milestone in recent model publications. It encompasses high-capacity parameters, low-bit storage, long context windows, multimodal inputs, adjustable reasoning intensity, and quantifiable Agent task evaluations.

For developers, the focus should not be on “how the model ranks,” but rather on understanding its engineering implementation and usage conditions:

• Inference frameworks have initial support, but resource barriers remain extremely high.
• API integration lowers the barrier to entry, but requires correct multi-turn state passing.
• Agent capabilities are backed by quantitative data, but should be revalidated on custom tasks.
• Fine-tuning and secondary development require special attention to hybrid structures and routing strategies.

During technical selection, teams should first define task boundaries and cost ceilings before deciding between online inference, offline deployment, or a hybrid architecture. The model itself is just the starting point; what truly determines the outcome is the tooling workflow, data feedback, and operations system built around the business objectives.