Open-Weight Deployment of an Ultra-Large Sparse Model: Architecture, Hardware, and Engineering Practices

1. Model Scale and Release Format

The open weights released for this model have two notable characteristics: the total parameter count is approximately 2.8 trillion, while the model uses a sparse activation mechanism that activates only about 104 billion parameters per inference step. This design allows the model to retain a large capacity and high knowledge density while substantially reducing the computational cost of each forward pass.

From a storage perspective, the weight files are saved in MXFP4 format, and activation computation uses MXFP8. According to the official description, the model was trained with quantization-aware training, meaning quantization error was incorporated into the optimization objective during training rather than handled as a post-training step. For models with very large parameter counts, this approach typically preserves model quality better while reducing storage footprint.

The repository occupies approximately 1.56 TB of disk space, and the parameter count derived from the safetensor files is about 2.78 trillion. The model supports a context length of 1 million tokens and accepts both text and image inputs, which is practically relevant for long-document analysis, codebase understanding, and multimodal tasks.

2. Architecture Design and Sparse Attention

The model consists of 93 layers. Among them, 69 layers use a KDA structure, while 24 layers use a gated DSA structure. Compared with the previous generation, the main change is that the original gated MLA layers, which used full attention, have been replaced with a gated deep sparse attention mechanism. The goal is to reduce computational complexity in long-context modeling scenarios.

For the expert layers, the model contains 896 experts. Each token activates 16 of these experts, and 2 shared experts also participate in every forward computation. This configuration allows the model to maintain expressive diversity while controlling inference cost through sparse routing.

The vision component uses a vision encoder with approximately 401 million parameters to process image inputs. For mixed image-text tasks, the vision encoder output is fused with the text representation before entering the subsequent sparse expert network.

3. Deployment Requirements and Inference Framework Support

Because of the model’s large total size, local deployment places high demands on hardware resources. One viable configuration is a server with 8 B300-class accelerator cards and approximately 2.3 TB of total GPU memory. Based on current hardware market prices, the estimated hardware cost for this configuration is roughly $550,000 to $650,000. This is a third-party estimate, not an official recommended configuration.

Several mainstream inference frameworks already provide deployment configuration templates, including vLLM, SGLang, and TokenSpeed. When supporting large-scale sparse models, these frameworks typically need to address the following key issues:

1. Weight sharding and loading: The approximately 1.56 TB of weight files cannot fit on a single accelerator, so they must be sharded according to tensor parallelism, pipeline parallelism, or expert parallelism strategies.
2. Sparse expert routing: Only 16 experts are activated per token, so the framework needs to efficiently perform expert selection and computation scheduling during inference, avoiding the memory and bandwidth pressure caused by loading all experts.
3. Long-context attention: Attention computation over a 1 million token context requires sparse strategies. Otherwise, the computational load and memory consumption would be difficult to sustain.
4. Mixed-precision execution: MXFP4 storage and MXFP8 activation computation require dequantization and precision conversion during inference, and the framework must provide efficient implementations.

An illustrative deployment configuration can be expressed as follows:

model_config:
  total_params: 2.8T
  active_params_per_token: 104B
  expert_total: 896
  expert_activated: 16
  shared_experts: 2
  weight_dtype: mxfp4
  activation_dtype: mxfp8
  context_length: 1000000
hardware_requirement:
  gpu_count: 8
  total_memory: 2.3TB
  interconnect: high_bandwidth

The configuration above illustrates the key parameters to consider when deploying the model. Actual deployment parameters should be adjusted according to the specific hardware environment and inference framework version.

4. API Access and Tool-Calling Mechanism

For developers who do not want to maintain their own inference cluster, the model can be accessed through an API. The API interface is compatible with both OpenAI and Anthropic call formats, with standard and high-speed tiers available. According to the official description, the high-speed tier is 2 to 3 times faster than the standard tier.

For inference intensity, the API provides three options: low, high, and max. The default is max. Inference intensity directly affects how much reasoning the model performs before answering. Higher intensity generally improves accuracy on complex tasks, but it also brings higher latency and cost.

In multi-turn tool-calling scenarios, one key detail is that the caller needs to return the reasoning content from the previous response together with the tool call results. This means the model’s reasoning process is not necessarily completed in a single pass; it may progress across multiple tool interactions. If the client returns only the final text and discards the reasoning state, the model may lose full awareness of the task context.

A typical multi-turn tool-calling flow is as follows:

User question
  -> Model returns reasoning_content + tool_calls
  -> Client executes the tools and obtains results
  -> Client returns reasoning_content + tool_calls + tool_result together
  -> Model continues reasoning and provides the final answer

This interaction pattern places additional requirements on client-side session state management. Developers need to properly preserve the intermediate state returned in each turn.

5. Long-Horizon Task Performance and Evaluation Dimensions

The evaluation data comes from real user long-horizon sessions rather than artificially constructed benchmark tests. The evaluation system extracts task completion signals from multiple sources, including whether the user explicitly confirms task success, positive or negative user feedback, the model’s response quality after user correction, and the actual execution results of shell commands.

The core evaluation metric is net improvement. This metric reflects whether the model delivers real efficiency or quality gains relative to the baseline in long-horizon tasks, rather than simple single-turn accuracy.

In an evaluation of more than 17,500 real long-horizon sessions, the model achieved a net improvement of +5.1% at maximum inference intensity, ranking first among open-weight models. By task category, it also ranked first among open-weight models on work-related, coding-related, and conversation-related tasks.

On the more granular metric of Bash command reliability, the model achieved a net improvement of +7.9%, while the previous generation scored -4.5% on the same metric. This comparison indicates that in complex automation scenarios requiring continuous shell command execution, the new model can complete tasks more reliably. For developers, this means the model has stronger practical value for building automation script generation, environment configuration, log analysis, and similar long-horizon task flows.

6. Suitable Scenarios and Engineering Recommendations

Based on the model’s characteristics and evaluation results, it is well suited to the following scenarios:

Coding and automation tasks: Its long-context capability enables it to understand large codebase structures, the sparse attention mechanism is relatively efficient for processing long file sequences, and its tool-calling capability can be used to execute commands, verify results, and iteratively correct errors.

Complex workflow orchestration: The model’s performance in multi-turn tool interactions indicates that it can maintain goal consistency across longer task chains, making it suitable for building intelligent workflows that require multiple execution steps.

Long-document processing: The 1 million token context window allows large document collections to be processed in a single pass, reducing the information fragmentation caused by chunked processing.

From an engineering practice perspective, several points are worth noting:

• If your team has GPU resources and strict data compliance requirements, local deployment is an option, but you should reserve enough GPU memory and engineering adaptation time.
• If you prioritize rapid validation and elastic scaling, the API approach is more pragmatic, especially for early-stage prototyping.
• In multi-turn tool calls, always preserve and return the complete reasoning state. Otherwise, model performance may degrade noticeably.
• The inference intensity parameter should be adjusted according to task complexity. Using the highest intensity for simple tasks is not economical.

Overall, ultra-large sparse models are transitioning from research weight releases to engineering deployment. For developers, understanding the weight storage format, sparse activation mechanism, deployment resource requirements, and API interaction details is a prerequisite for effectively leveraging the capabilities of this class of models.