Open Weights for Massive Sparse Models: Deploying 2.8T Parameters and Analyzing Long-Horizon Agent Evaluation

I. What Open Weights Mean for Developers

Recently, a major event in the AI industry occurred: a Mixture-of-Experts (MoE) model with approximately 2.8 trillion total parameters and 104 billion activated parameters per token officially released its complete weights ahead of schedule. For teams looking to self-host, perform deep fine-tuning, conduct security audits, or deploy offline, this is fundamentally different from merely accessing the model via an API.

With open weights, users can:

• Perform inference deployment on their own hardware or in controlled environments;
• Conduct static audits on model structure, parameter distribution, and safety behaviors;
• Perform continued pre-training, instruction fine-tuning, or reinforcement learning based on internal data;
• Optimize for specific domains through distillation, pruning, and quantization;
• Build compliant data pipelines without relying on external inference endpoints.

This release is not just a handful of configuration files. It provides directly loadable complete weight shards, model structure definitions, and deployment guidelines for inference frameworks. According to repository metadata, the total weight file size is approximately 1.56TB, with the parameter count in the safetensors format reaching about 2.78 trillion.

II. Model Architecture and Computational Characteristics

2.1 Overall Structure: MoE and Layered Attention

The model consists of 93 layers. 69 of these use a structure known as KDA, while the remaining 24 use gated dynamic sparse attention layers. Compared to the previous generation, the primary change is replacing some full-attention layers with sparse attention mechanisms designed for long contexts, reducing the computational and memory pressure during long-sequence inference.

The expert layer contains 896 experts, of which 16 are activated per token, alongside 2 constantly active shared experts. This design allows the model to maintain a massive capacity for knowledge storage while keeping the actual computation per token at the ~104B parameter level.

2.2 Vision Module

The multimodal component utilizes a vision encoder with approximately 401 million parameters for image embedding and mixed image-text inputs. The vision encoder connects to the main large language model backbone via an adapter layer, enabling the model to process multimodal inputs such as text with images, UI screenshots, and charts.

2.3 Quantization Storage and Activation Format

Weights are stored in MXFP4 format, and activations use MXFP8. Official documentation states that Quantization-Aware Training (QAT) was used rather than post-training quantization. This means the model accounted for low-precision numerical distributions during the training phase, theoretically preserving accuracy better than simply converting FP16 weights to 4-bit.

From a deployment perspective, this choice has the following impacts:

1. Significant VRAM reduction: FP4 storage reduces the weight footprint to about a quarter of FP16. Even so, the complete weights reach 1.56TB, indicating that the original FP16 weights far exceed single-machine VRAM capacities.
2. Dequantization or mixed-precision computation required: The inference engine must convert FP4 weights to a computable format after loading to perform matrix multiplication with FP8 activations.
3. Inference framework support required: Not all inference engines natively support MXFP4/MXFP8 formats; specific adaptations are needed.

III. Long Context and Inference Capabilities

The model supports a 1-million-token context window, which is highly relevant for scenarios like long-document analysis, code repository comprehension, and multi-turn agent tasks. The core challenge of ultra-long contexts lies in the computational complexity and memory consumption of attention mechanisms.

The layered attention design offers a compromise:

• Most layers use standard local or sliding window attention to ensure basic semantic modeling;
• Some layers use gated sparse attention, computing only key positions within long sequences;
• The gating mechanism dynamically decides the attention scope based on the current token’s representation, avoiding information loss caused by fixed sparse patterns.

This structure allows the model to process long code files, lengthy log outputs, and large document collections without subjecting all tokens to full attention computation, thereby reducing the resource pressure caused by O(n²) complexity.

IV. Deployment Strategies and Hardware Requirements

Following the weight release, the community quickly provided deployment solutions for various inference engines. Users can choose a framework that fits their tech stack and launch inference services using official configuration templates.

4.1 Basic Deployment Workflow

The general deployment workflow is as follows:

1. Download weight shard files and verify integrity
2. Install an inference engine that supports this model structure
3. Configure distributed inference parameters, including tensor parallelism, pipeline parallelism, and expert parallelism
4. Load weights and convert them to an internal format usable by the engine
5. Start the service and verify text generation and multimodal inputs
6. Adjust concurrency, batch size, and context length based on load

4.2 Hardware Resource Estimation

Due to the massive weight size, single-GPU inference is impractical. According to community estimates, a viable configuration is a single-machine server using 8 high-memory accelerators, totaling approximately 2.3TB of VRAM. The hardware cost is estimated to be between $550,000 and $650,000. This estimate originates from third parties and is not an official recommendation.

If using accelerators with less VRAM, more nodes are required for tensor and pipeline parallelism, introducing additional communication overhead. For teams with limited budgets, more practical approaches include:

• Using inference endpoints with pay-as-you-go billing;
• Opting for smaller distilled or pruned versions;
• Building temporary self-hosted inference clusters only during peak periods for specific tasks.

4.3 Recommended Inference Configuration Example

When self-hosting, you can reference the following configuration approach:

tensor_parallel_size: 8
pipeline_parallel_size: 1
expert_parallel_size: 1
dtype: "auto"
quantization: "mxfp4"
max_model_len: 131072
enable_prefix_caching: true

Note that this is only a general configuration example. Actual parameters should be adjusted according to the inference engine documentation and hardware conditions, and should not be mapped to any specific cloud provider’s proprietary parameters or interface names.

V. API Usage and Tool Calling

For users who prefer not to bear the hardware costs, the model can also be accessed via API.

5.1 Model Names and Inference Intensity

The API provides two types of endpoints: Standard and Fast. The Standard endpoint focuses on inference depth, while the Fast endpoint claims 2 to 3 times the throughput of the Standard. Inference intensity supports three levels:

• low: Suitable for simple instructions and short response scenarios;
• high: Suitable for complex problems requiring multi-step reasoning;
• max: The default level, maximizing inference depth, suitable for math, coding, and planning tasks.

5.2 Multi-Turn Tool Calling Considerations

In agent scenarios, the model frequently needs to call external tools. During multi-turn tool calls, the reasoning content and tool call results from the previous turn must be passed back; otherwise, the model cannot maintain a complete reasoning chain.

A general message structure example is as follows:

{
  "role": "assistant",
  "content": "I will first check the current system status.",
  "reasoning_content": "The user wants to know if the service is running normally. I need to execute a status query command first.",
  "tool_calls": [
    {
      "id": "call_1",
      "type": "function",
      "function": {
        "name": "exec_shell",
        "arguments": "{\"command\": \"systemctl status app\"}"
      }
    }
  ]
}

In the next request, the above reasoning_content and tool_calls must be passed back, along with the appended tool execution results.

VI. Agent Capability Evaluation: Net Improvement Metrics

6.1 Evaluation Methodology

Traditional evaluations typically rely on static benchmark datasets. The recently published data comes from real user-agent sessions, comprising over 17,500 long-horizon conversations. The evaluators extracted the following types of signals from these sessions:

• Whether the user explicitly confirmed task success;
• User feedback (positive or negative) on the model’s output;
• The model’s correction behavior after user adjustments;
• Whether shell commands were actually executed successfully.

These signals were combined to calculate a “net improvement” value, measuring whether the model provided perceivable benefits in real agent tasks compared to a baseline.

6.2 Key Results

The model achieved a net improvement of +5.1% in the agent arena, ranking 13th overall and 1st among open-weight models. Across task categories, it topped the open-weight model rankings in Work, Coding, and Conversation.

Compared to the previous generation model, the progress is significant: the previous generation had a net improvement of +0.91%, ranking 6th among open-weight models. In the shell command reliability sub-category, the new generation reached +7.9% (up from -4.5% for the previous generation), with its ranking jumping from 13th to 1st.

This change indicates that adjustments to the sparse attention structure and training strategy have yielded substantial gains for command execution tasks. Command execution reliability is critical in agent scenarios, as a single erroneous command can lead to environment corruption or task chain interruption.

VII. Practical Recommendations

For teams planning to integrate this model, evaluation from the following perspectives is recommended:

1. Task Type: If the primary scenarios involve long-horizon agents, code repository comprehension, or long-document analysis, this model is worth testing. For short text classification or simple Q&A, smaller models may be more cost-effective.
2. Deployment Model: Consider self-hosting if there are compliance and data residency requirements; otherwise, pay-as-you-go API calls offer more flexibility.
3. Context Length: In practice, you do not always need to use the 1-million-token limit. A shorter context window can significantly reduce time-to-first-token latency and cost.
4. Tool Call Design: Always retain the reasoning content field; otherwise, multi-turn tool calls will experience context breakage.
5. Monitoring and Fallback: When self-hosting, integrate logging and alerting systems to record request latency, queue depth, and error rates, and establish a fallback chain with backup inference endpoints.

Overall, open weights lower the barrier for technical audits and deep customization. However, the true value of deployment still depends on a team’s comprehensive assessment of deployment resources, inference costs, and task alignment.