Massive Sparse Model Weights Released: MXFP4 Quantization, Long-Context Deployment, and Agent Evaluation Explained

What Open Weights Actually Mean

A sparse Mixture-of-Experts model with roughly 2.8 trillion total parameters and about 104 billion parameters activated per token has released its complete weights, following an earlier commitment to do so before early November. For research and engineering teams that need private deployment, fine-tuning, security auditing, or deep customization, this is significantly more valuable than API-only access. The weight files are directly available, the repository takes up approximately 1.56TB, and the model card includes structural details, quantization settings, and deployment recipes for supported inference frameworks.

Open weights do not mean turnkey deployment. This model imposes substantial requirements around storage format, memory footprint, context length, multimodal input, and agentic tool calling. Understanding these constraints is a prerequisite for downstream deployment and further development.

Model Architecture and Long-Context Design

The model consists of 93 layers, with 69 layers using a KDA structure and 24 layers using a Gated DSA structure. Compared with the previous generation, the main change is concentrated in long-context modeling: the earlier all-attention Gated MLA layers have been replaced by a gated sparse attention mechanism. The goal is to preserve long-range dependency modeling while reducing compute and memory overhead in long-sequence scenarios.

The expert layer contains 896 experts in total. Each token selects 16 of them, and there are also 2 shared experts. Shared experts typically capture cross-domain general features, while routed experts handle more fine-grained knowledge specialization. On the vision side, a visual encoder with approximately 401 million parameters enables image-text input. The encoder is not particularly large, but it is practical enough for document understanding, UI screenshot analysis, chart question answering, and similar tasks.

One clarification is necessary: “sparse” here does not mean randomly zeroing out weight matrices. It means each token activates only a small fraction of the total parameters. With 2.8T total parameters but only 104B activated per token, inference latency is primarily tied to activated parameters, expert routing, and KV cache, rather than scaling linearly with total parameter count. Even so, 104B activated parameters is still very large and places high demands on memory bandwidth and compute capacity.

MXFP4 Weights and MXFP8 Activations: The Role of Quantization-Aware Training

The model stores weights in MXFP4 format and uses MXFP8 for activations. The team behind it has stated that quantization-aware training was used, rather than simple post-training quantization. This distinction matters.

MX formats differ from traditional INT4/INT8 or standard FP8. Integer quantization has limited dynamic range and can produce large errors on extreme values. Standard FP8 typically uses a uniform exponent bit allocation, which also struggles to accommodate the numerical distributions of different tensors. MX formats share an exponent across a block of data while preserving independent mantissas for each element, providing more flexible numerical representation at low bit widths. Using MXFP4 for weights means each weight occupies an average of 4 bits. Using MXFP8 for activations means each activation occupies 8 bits. Combined with quantization-aware training, the model learns to adapt to quantization error during the training phase instead of passively accepting precision loss after training is complete.

From a deployment perspective, the approximately 1.56TB weight storage corresponds exactly to the storage requirement of 2.8T parameters at around 4 bits. At BF16 precision, weights alone would require about 5.6TB, exceeding the memory capacity of typical single-node setups. MXFP4 makes it feasible to load the full model on a single high-end server, provided the inference framework correctly supports dequantization and matrix operation paths for this format.

MXFP8 activations are used to reduce the memory footprint of intermediate results during inference. In long-context scenarios, KV cache and intermediate activations expand rapidly with sequence length. 8-bit activations significantly alleviate this pressure. However, not all inference engines natively support MXFP8 activations, so deployment requires choosing validated versions and kernel combinations.

Deployment Conditions and Hardware Budget

The released repository includes deployment recipes for three inference frameworks, covering the mainstream open-source inference stacks. Their configuration approaches are similar: load weights and parse formats first, then configure expert parallelism or tensor parallelism according to the model structure, and finally enable long-context optimizations.

The hardware threshold is not low. One third-party estimate suggests a viable configuration of eight latest-generation high-performance accelerators with approximately 2.3TB total memory, at a hardware cost of roughly $550,000 to $650,000. This estimate comes from the community rather than official guidance, but it still serves as a useful budget reference. Why is so much memory needed? Three reasons:

• The weights themselves take about 1.56TB and must be distributed across multiple cards.
• During long-context inference, KV cache can occupy hundreds of gigabytes or more.
• Activations, even at 8 bits, still consume substantial space in batched scenarios.

If self-hosting is not immediately feasible, using the API remains the more practical choice. Both standard and high-speed endpoint options are available. The high-speed endpoint claims to be 2 to 3 times faster than the standard one, making it suitable for interactive agents or online assistant scenarios. The reasoning intensity parameter offers three levels: low, high, and maximum, with maximum as the default. Higher reasoning intensity generally means the model performs a longer internal reasoning chain before answering, which is friendlier for complex tasks but also increases token consumption and latency.

Multi-turn tool calling has an additional API requirement: the reasoning content from the previous turn and the tool call information must both be passed back. This differs from standard chat APIs. In normal chat, only the message list needs to be returned. Here, the internal reasoning state generated by the model must be preserved and returned; otherwise the model may fail to correctly continue subsequent actions. Developers wrapping the client should ensure that both fields are fully saved in each turn and sent back unchanged in the next request.

How Agent Capabilities Are Measured

On the same day the weights were released, the model appeared in an agent arena leaderboard: first among open-weight models and 13th overall. This result comes from approximately 17,500 real long-horizon sessions, not a fixed pre-designed test set. The evaluation system reads multiple signals from user-agent interactions, including whether users explicitly confirm task success, praise or complaints about the model, whether the model’s response improves after user correction, and whether shell commands actually execute successfully. These signals are then used to compute a “net uplift” metric.

Net uplift can be understood as the net value the model delivers to users across all interaction turns, calculated as positive impact minus negative impact. Compared with single-turn question-answering scores from traditional benchmarks, this approach is closer to real workflows. A model may be strong at single-turn responses but fail to maintain state in multi-step tasks, make frequent mistakes, or struggle to understand user corrections, resulting in a lower net uplift.

Looking at breakdown metrics, the model shows notable improvement in Bash reliability, jumping from negative values in the previous generation to first among open models. This indicates stronger stability in real operational scenarios such as command-line execution, file operations, and script debugging. It also ranks first among open models in the work, coding, and chat categories, suggesting a relatively balanced capability distribution. The overall 13th-place ranking indicates there is still room for improvement across the full model landscape, especially compared with leading closed-source frontier models.

It is worth noting that this evaluation relies on voluntarily submitted user sessions, so it cannot fully exclude biases from user preferences, uneven task type distribution, or interface experience differences. Teams looking to build a similar internal evaluation system should consider maintaining three types of signals simultaneously: automated benchmarks, human annotation, and real user sessions. A combined judgment is more reliable for deciding whether a model is ready for production.

Practical Recommendations

For teams considering API integration or private deployment, the following suggestions may be useful:

1. Validate task fit through the API first. Abstract real business tasks such as long-document analysis, multi-step tool calling, and code repository operations into a test set. Evaluate whether output quality and latency meet requirements before committing hardware investment.
2. Prefer validated inference framework and hardware combinations. Do not assume that open weights mean any framework can load them seamlessly. Beyond the base architecture, KV cache management, batching strategy, and quantization kernels all require targeted configuration.
3. Long context is not an unconditional advantage. Million-token context is well suited for document analysis and codebase reading, but it may add unnecessary cache overhead on short tasks. Set a reasonable context window limit based on the use case to balance memory and effectiveness.
4. Tool calling requires strict state management. For multi-turn agent workflows, passing back reasoning state and tool call results is a critical link in the chain. The wrapper layer should ensure field completeness and a sound timeout-and-retry mechanism to avoid task interruption from state loss.
5. Evaluation should not focus solely on leaderboard scores. Public arena rankings provide a useful cross-model reference, but before production rollout, teams should build evaluation sets that more closely match their own business needs, especially regarding error propagation and recovery in long workflows.

Open weights lower the observability barrier for model research, but the real engineering challenges begin the moment the download completes. Understanding quantization formats, memory layout, inference framework compatibility, and agent state management is more practically valuable than fixating on parameter counts alone.