2.8T-Parameter Open-Weight Model Tops Agent Leaderboard: Weights, Deployment, and Real-World Analysis

From Open Weights to Proven Capability: A Key Move for the Open-Source Ecosystem

A mixture-of-experts (MoE) large language model with a total of 2.8T (trillion) parameters has officially released its complete weights to the public. Published by a leading AI lab, the release fulfills an earlier commitment to open the weights. The significance of this move goes beyond the model’s own benchmark numbers: it gives the open-source community an auditable, modifiable high-performance foundation model. This article presents a complete technical overview across four dimensions: model architecture, quantization and storage, real-world evaluation results, and deployment paths.

1. Architecture: Smarter Sparsity Behind 2.8T Parameters

This model is not brute-force parameter stacking; its core design principle is efficient sparsity. Specifically, it is a MoE model with roughly 2.78 trillion total parameters, of which only 104B are activated per token. This “massive total, small active footprint” approach gives the model enormous knowledge capacity while keeping inference compute within a manageable range.

The model stacks 93 Transformer layers, and its attention mechanism uses a hybrid design:

• 69 layers adopt the traditional KDA (Key-Value Decomposed Attention) structure, handling standard context modeling;
• 24 layers are upgraded to Gated DSA (DeepSeek Sparse Attention), a gated sparse attention mechanism that uses gating units to dynamically determine the sparsity pattern of attention heads, aiming to substantially reduce the computational complexity and memory footprint of long-sequence processing.

At the expert layer, the model ships with 896 routed experts, activating only 16 of them per inference, plus 2 shared experts that stabilize baseline capability output. This fine-grained expert partitioning — many small experts rather than a few large ones — improves the precision of expert specialization and represents an important direction in current MoE evolution.

The model also natively supports image-and-text multimodal input. Its vision encoder is a Vision Transformer variant of roughly 401M parameters, deeply aligned with the text backbone.

2. Storage and Quantization: MXFP4 Weights with MXFP8 Activations

For a model at the 2.8T-parameter scale, whether developers can actually “download and run it” is the crux of engineering adoption. The released weights take a bold approach to storage format:

• Weight storage uses MXFP4 (Microscaling Floating Point 4-bit), a microscaling floating-point format optimized for AI inference. By introducing per-tensor scaling factors on top of FP4, it effectively mitigates the precision loss typical of low-bit quantization.
• Activation computation uses MXFP8, preserving higher precision during intermediate matrix multiplications.

This “store at low bit, compute at mid bit” strategy is a direct product of quantization-aware training (QAT). The team has confirmed that the model was adapted to this quantization scheme during training — not quantized after the fact — so the weights are better fitted to the information bottleneck introduced by 4-bit storage.

In terms of actual footprint, the Hugging Face repository lists the weight set at approximately 1.56TB. That is more than 70% smaller than uncompressed FP16 (theoretically about 5.5TB), dramatically lowering the physical barriers of downloading and disk storage.

3. Agent Capability, Verified: Net Improvement Rate from Real Sessions

Whether a model can genuinely “get the job done” — especially on multi-step Agent reasoning tasks — has become the industry’s accepted litmus test. On release day, the model claimed the top spot in the “open-weight models” category of a well-known model Arena. What makes this leaderboard distinctive is that its data comes entirely from real users’ long-horizon interactive sessions rather than a preset question bank.

The core of its scoring mechanism is the “net improvement rate,” which weighs the following behavioral signals:

1. Task success rate: whether the user explicitly confirms that the task is complete.
2. User sentiment feedback: praise or complaints expressed during the session.
3. Correction response quality: after a user makes a correction, whether the model can quickly adjust its strategy and execute accurately.
4. Tool execution reliability: for example, whether Shell commands succeed on the first run without a second attempt.

Under this system, the model (with reasoning effort set to Max) achieved a +5.1% net improvement rate, ranking first among open-weight models and 13th overall on a leaderboard that includes closed-source commercial models. Its predecessor scored only +0.91%. Among the sub-dimensions, the Bash (command-line interaction) reliability score of +7.9% is particularly telling for tool-call-heavy Agent scenarios: it indicates more stable command comprehension and argument generation when executing commands on real operating systems.

4. Deployment in Practice: From Hardware Thresholds to Framework Support

4.1 The hardware baseline for local, private deployment

Given the total parameter count, local deployment is clearly beyond what consumer-grade GPUs can handle. According to third-party community estimates, one feasible minimum configuration looks like this:

• Accelerators: 8x NVIDIA B300 (or equivalent data center GPUs);
• Total VRAM: about 2.3TB (288GB per card);
• Server hardware cost: roughly $550,000–$650,000.

Note that this estimate assumes full loading across 8 cards (based on the ~1.56TB MXFP4 weight footprint). Actual training or fine-tuning scenarios will require more. The team has not published an official recommended configuration list; procurement teams should size their purchase against their own concurrency and throughput requirements.

4.2 Progress on mainstream inference framework support

To help the community get services running quickly, three major open-source inference engines have completed adaptation and published deployment recipes:

• vLLM: supports PagedAttention and continuous batching, well suited to high-concurrency online serving.
• SGLang: deeply optimized for structured generation and multi-turn dialogue; its RadixAttention reuses historical KV caches, delivering clear advantages in long-context scenarios.
• TokenSpeed: an emerging inference framework focused on extreme low latency, suited to applications sensitive to first-token latency.

On quantization support, all three frameworks explicitly support direct loading of MXFP4 weights with no additional dequantization at service startup, effectively shortening cold-start time.

4.3 A note on API access

For teams that do not need self-hosted compute, an official hosted API is also available. It is compatible with both the OpenAI and Anthropic interface protocols, so existing codebases can migrate with minimal friction. The API offers a standard tier and a high-speed tier (throughput roughly 2–3x that of the standard tier). For multi-turn Agent calls, there is one mandatory detail: in subsequent turns, the previous turn’s reasoning_content (chain-of-thought content) and tool_calls (tool call records) must be passed back in the request; otherwise the multi-turn state breaks.

5. Industry Takeaways and Selection Advice

This weight release carries at least two core messages for the AI engineering community:

• Quantization-aware training is now a prerequisite for open-sourcing ultra-large models. Without the MXFP4 storage scheme, a 1.56TB download would put the model out of reach for most small and mid-sized companies in terms of bandwidth cost alone, let alone deployment. This sets a new “size benchmark” for similar open-source releases going forward.
• Real-scenario evaluation is more persuasive than static benchmarks. The Agent Arena’s real-session methodology effectively sidesteps leaderboard gaming and offers a more practical reference frame for assessing model capability. When selecting a model, enterprises should focus on the net improvement rate in their target scenarios — such as code generation, complex task decomposition, and command-line interaction — rather than a single aggregate score.

In summary: this is a heavyweight model aimed at advanced developers and large enterprise users. Its open, ultra-large parameter scale and advanced quantized deployment approach can support complex business agent applications out of the gate; but its steep hardware cost also means it is not a mass-market product. For teams planning to embrace open-source LLMs in depth, the practical path is to validate business impact through the API mode first, and only decide whether to invest in costly local deployment after the ROI is clearly demonstrated.