Technical Deep Dive and Deployment Practices for a 2.8T Open-Weight MoE LLM

1. Background: A New Milestone for Open-Weight Models

A recent AI research institution has officially released the complete weights of its latest flagship model for download, fulfilling its commitment to open-weights within two weeks of release. This move has garnered significant attention in the developer community, not just for the model’s scale—2.8 trillion total parameters with a 104B activated parameters per token in a MoE architecture—but because it is the first open-weight model to position its Agent capabilities in direct competition with closed-source models.

For teams looking to self-host, fine-tune, or audit model behavior, the availability of complete weights marks a shift from “API-only access” to “full control over the inference pipeline.” This article analyzes the model across five dimensions: architecture, quantization strategy, deployment barriers, API design, and Agent evaluation, helping developers determine if this technical path suits their use cases.

2. Model Architecture: Hybrid Attention and Sparse Experts

2.1 Overall Architecture Overview

According to the model card, the architecture consists of 93 layers: 69 layers utilize the KDA attention mechanism, while 24 layers use Gated DSA (sparse attention). Compared to its predecessor, the most notable change is replacing full-attention Gated MLA layers with Gated DSA, aiming to reduce computational overhead in ultra-long context scenarios.

This addresses a core contradiction: for million-token context windows, standard full attention causes quadratic growth in both KV Cache memory consumption and computational complexity. Sparse attention mitigates this by having each token attend only to a subset of historical tokens. The Gated mechanism builds on this by introducing learnable gates that dynamically determine information flow paths.

2.2 Expert Layer Design

The MoE component is configured with 896 experts. For each token, 16 experts are selected for computation, alongside 2 shared experts that are always activated. This combination of “fine-grained experts + shared experts” represents a mainstream MoE design philosophy:

• Fine-grained experts: A large number of experts with smaller individual parameter sizes facilitates load balancing and specialized division of labor.
• Shared experts: Always active, responsible for storing and transferring general knowledge, preventing individual experts from redundantly learning foundational capabilities.

Looking at the activation ratio, 104B activated / 2.8T total parameters ≈ 3.7%, which is a highly sparse MoE configuration. This means the actual computational volume during inference is far smaller than the total parameter scale suggests, though the VRAM footprint still needs to accommodate all weights.

2.3 Vision Modality

The vision component utilizes MoonViT-V2, containing 401M parameters, to support image-text inputs. This module is a critical capability for scenarios requiring document understanding, chart analysis, and multimodal Agent tasks.

3. Quantization Strategy: MXFP4 Storage and MXFP8 Activations

3.1 Quantization-Aware Training

Official specifications indicate the model employs Quantization-Aware Training (QAT). Weights are stored in MXFP4 format, while activations use MXFP8. It is important to distinguish between these two formats:

• MXFP4: A 4-bit floating-point with microscaling format. Compared to INT4, it better preserves dynamic range, making it suitable for weight storage.
• MXFP8: An 8-bit floating-point format used for activations, maintaining higher numerical precision during inference.

The significance of QAT lies in the fact that the model “knows” during training that it will eventually be quantized, allowing it to learn a parameter distribution that is more robust to quantization errors. This typically yields better accuracy retention than post-training quantization.

3.2 Storage Footprint Testing

The weights repository occupies approximately 1.56TB, with safetensors reporting around 2.78 trillion parameters. Compared to the theoretical value of FP16 storage (2.78T × 2 bytes ≈ 5.56TB), MXFP4 compresses storage requirements by about 3.5x. While this is a substantial benefit for downloading and local storage, 1.56TB remains a significant footprint that requires proactive storage planning.

4. Deployment Practices: Framework Support and Hardware Barriers

4.1 Inference Framework Adaptation

Mainstream inference frameworks have provided deployment solutions, including vLLM, SGLang, and TokenSpeed. Using vLLM as an example, a typical deployment workflow looks like this:

# Example: Starting an inference service using vLLM (illustrative)
vllm serve <model-path> \
  --tensor-parallel-size 8 \
  --quantization mxfp4 \
  --max-model-len 1000000 \
  --trust-remote-code

Key parameters to note include:

• --tensor-parallel-size: Sets the tensor parallelism degree based on available GPUs.
• --quantization: Specifies the quantization format to correctly load MXFP4 weights.
• --max-model-len: Sets the maximum context length; a million tokens requires substantial KV Cache space.

4.2 Hardware Barrier Estimation

Third-party estimates suggest a viable configuration requires 8 high-end datacenter GPUs with approximately 2.3TB of VRAM, costing around $550,000 to $650,000. This figure should be treated with caution—it is a third-party estimate, not an official recommendation, and actual requirements vary based on quantization methods, batch sizes, and context lengths.

For most teams, a more pragmatic approach includes:

1. API Access: The model name is compatible with OpenAI and Anthropic interfaces. A high-speed version is also available, which the official claims is 2 to 3 times faster than the standard version.
2. Cloud On-Demand Instances: Deploying on GPU instances from major cloud providers avoids upfront hardware investments.
3. Small-Scale Fine-Tuning: Performing parameter-efficient fine-tuning like LoRA based on the open weights, which can be completed on smaller clusters.

4.3 API Design Highlights

Several API design features are noteworthy:

• The reasoning_effort parameter accepts low, high, or max (default is max), allowing developers to adjust reasoning depth based on task complexity.
• During multi-turn tool calling, the previous round’s reasoning_content and tool_calls must be passed back together. This is crucial for building Agent loops.
• Compatibility with both OpenAI and Anthropic API specifications reduces migration costs.

5. Agent Capability Evaluation: Extracting Signals from Real Sessions

5.1 Evaluation Methodology

Data from Agent Arena is derived from real user Agent sessions rather than synthetic benchmarks. It extracts several types of signals:

• Whether the user explicitly confirms task success.
• Praises and complaints regarding the model.
• The response quality after user corrections.
• Whether shell commands execute successfully.

A “net improvement” metric is calculated based on these signals, essentially measuring the positive value the model brings to real-world workflows.

5.2 Evaluation Results

Based on over 17,500 real long-context sessions, this model (Max tier) achieved a net improvement of +5.1%, ranking 1st among open-weight models and 13th overall. The previous model recorded +0.91%, ranking 6th among open weights. In sub-categories, Bash reliability improved by +7.9%, ranking 1st for open weights (compared to -4.5% and 13th for the predecessor).

The significance of this data lies in demonstrating that open-weight models now possess the capability to compete with closed-source models in Agent scenarios, particularly in command-line interactions and tool calling reliability.

6. Conclusion and Selection Recommendations

The release of this open-weight model provides a new option for teams requiring self-controllable Agent capabilities. Key takeaways:

• Architecture: 93-layer hybrid attention (69 layers KDA + 24 layers Gated DSA), 896-expert MoE, 104B activated parameters.
• Quantization: MXFP4 storage + MXFP8 activations, quantization-aware training, repository size ~1.56TB.
• Deployment: Mainstream inference frameworks are adapted, but local deployment hardware barriers are high; API access is a more practical starting point.
• Agent Capability: Outstanding performance in real session evaluations, with Bash reliability being particularly impressive.

When selecting a model, teams should weigh their actual needs: if the core requirements are data privacy and inference pipeline control, and the team has the necessary hardware resources, local deployment is a sound choice. If rapid validation and cost control are prioritized, API access combined with reasoning_effort tuning offers excellent cost-performance across most scenarios.