Open-Weight Release of a 2.8T-Parameter MoE Model: Architecture, Deployment, and Evaluation

1. Why Open Weights Matter

A key dividing line in the large model landscape is the difference between API-only access and releasing full model weights. The former treats the model as a black-box service. The latter hands the model to developers, enabling self-hosting, fine-tuning, quantization, auditing, and even redistribution. For teams that require data compliance, private deployment, or deep customization, open weights are the prerequisite for any real usability.

Recently, a research team fulfilled a prior commitment by releasing the complete weights of its flagship Mixture-of-Experts (MoE) model to a public model community for global download. The technical significance is this: a sparse model with approximately 2.8 trillion total parameters is, for the first time, available to the public at a relatively manageable storage and deployment cost. On the same day, a third-party evaluation platform published ranking data based on real user sessions, placing this model first in the open-weight category. This article examines five aspects—model architecture, weight format, deployment options, hardware requirements, and evaluation signals—to help developers assess whether self-hosting is worth the investment.

2. Model Architecture: Sparse Experts and Sparse Attention

Understanding deployment cost starts with understanding the model’s structural design. According to the public model documentation, the core architecture parameters are as follows:

• Total layers: 93 layers, of which 69 use a KDA structure and 24 use a Gated DSA structure.
• Attention mechanism evolution: Compared with the previous generation, this release replaces the full-attention Gated MLA layers with Gated DeepSeek Sparse Attention. The goal is to significantly reduce attention computation overhead in long-context scenarios. By computing attention weights for only a subset of token pairs, sparse attention reduces the cost from quadratic growth with context length to a more manageable level—a key enabler for million-token contexts.
• Expert layer design: 896 experts in total, with 16 activated per token, plus 2 shared experts. This “many experts, few activated” configuration is a classic high-capacity sparse MoE approach—total parameter count is very high, but the active parameters per forward pass are far smaller than the total.
• Active parameters: Approximately 104B parameters are activated per token. In other words, although the repository holds 2.8 trillion parameters, only about 104 billion actually participate in computation during inference. This puts per-token compute requirements in the same range as a dense model with hundreds of billions of parameters, while model capacity is nearly an order of magnitude higher.
• Vision encoder: The visual component is MoonViT-V2 with approximately 401M parameters, supporting mixed image-text input.

The direct consequence of this design is that storage cost is determined by total parameters, while compute cost is determined by active parameters. This large gap is the core tension in MoE deployment and the starting point for the hardware analysis below.

3. Weight Format: MXFP4 Storage and MXFP8 Activation

The weight file format directly determines download size and GPU memory usage during deployment. The model uses a mixed-precision quantization scheme:

• Storage format: Weights are stored in MXFP4 (4-bit floating point with a shared exponent). The MXFP family is a widely adopted microscaling format that preserves dynamic range at very low bit widths by assigning a shared exponent scaling factor to groups of elements.
• Activation format: Activations at inference time use MXFP8. Activations require higher precision than weights because their distribution varies dynamically with input, and overly low precision can cause numerical instability.
• Training method: The official documentation states that quantization-aware training (QAT) was used—simulating low-precision numerical behavior during the training stage so that model weights adapt to quantization error, rather than applying post-training quantization (PTQ) after training. QAT typically significantly reduces accuracy loss from quantization, which is a key reason why 4-bit storage remains viable.

According to public repository information, the weight repository occupies approximately 1.56TB, and the total parameter count across weight files is about 2.78 trillion. The 1.56TB figure is worth noting: storing 2.8 trillion parameters at 16-bit precision would require roughly 5.6TB; 4-bit storage theoretically comes to about 1.4TB, and with scaling factors, metadata, and the uncompressed vision encoder, 1.56TB is a reasonable figure. This means the download and storage barrier has dropped from multi-terabyte territory to the 1.5TB range, making it far more accessible for organizations with moderate storage resources.

4. Deployment Paths: Mainstream Inference Frameworks Already Support the Model

The value of open weights depends heavily on how quickly inference frameworks adapt. The good news is that several mainstream high-performance inference frameworks have already provided deployment recipes covering everything from single-node multi-GPU to multi-node distributed scenarios. A typical deployment flow looks like this:

# 1. Download weights (illustrative; refer to framework documentation)
# Repository size is about 1.56TB; use a download tool
# with resumable transfer support
# 2. Choose an inference framework and load the model
# Option A: high-throughput serving framework for online API services
# Option B: lightweight high-performance framework for low-latency single-node inference
# Option C: hardware-specific optimized runtime
# 3. Start serving (pseudocode)
serve --model ./weights \
      --tensor-parallel-size 8 \
      --max-model-len 1000000 \
      --quantization mxfp4

Key engineering considerations:

1. Tensor parallelism degree: A 2.8T-parameter model requires roughly 1.4TB of GPU memory for weights even at 4-bit precision. A single GPU cannot hold this, so tensor parallelism (TP) or expert parallelism (EP) is mandatory. The expert layers of an MoE model are naturally suited to expert parallelism—distributing the 896 experts across devices so each token only accesses a subset through routing—which can significantly reduce per-GPU memory pressure.
2. Memory budget for long contexts: KV Cache overhead for million-token contexts cannot be ignored. While sparse attention reduces compute, KV Cache storage must still be provisioned based on context length. Set max-model-len according to your actual maximum context length to avoid over-reserving GPU memory.
3. Quantization kernel support: MXFP4/MXFP8 requires inference frameworks to have corresponding dequantization and matrix multiplication kernels. Verify that your chosen framework version supports this quantization format before deployment; otherwise you may encounter load failures or fallback to inefficient execution paths.
4. Load the vision module separately: The vision encoder has only 401M parameters and can typically be loaded independently. If your workload does not require image-text input, consider omitting it to save GPU memory.

5. Hardware Requirements: Third-Party Estimates and Practical Considerations

The official release does not include hardware recommendations, but community estimates are available. Based on the weight size and active parameter count, one developer estimate suggests a server with 8 high-end data-center GPUs and roughly 2.3TB of total GPU memory, with an estimated hardware cost of about $550K to $650K. This is a third-party estimate—different parallelism strategies, batch sizes, and whether KV Cache quantization is enabled can significantly change actual requirements.

From an engineering perspective, hardware requirements can be broken down into three dimensions:

• GPU memory capacity: Weights must either reside fully in GPU memory or use tiered loading. If memory is insufficient, CPU offload is possible but introduces significant latency, making it more suitable for offline batch processing than online serving.
• GPU memory bandwidth: MoE inference bottlenecks are often in bandwidth rather than compute—each token activates 16 experts, which means substantial weight reads. High memory bandwidth may have a more direct impact on throughput than peak compute.
• Interconnect bandwidth: In multi-node multi-GPU deployments, expert parallelism involves frequent routing communication. Insufficient inter-node interconnect bandwidth can become the throughput bottleneck. This is why such models tend to favor single-node deployments with high-bandwidth interconnects.

For most teams, the pragmatic path is to validate business value through an API first, then evaluate self-hosting once the return is confirmed. The API side offers two model variants—standard and high-speed—with compatibility for mainstream interface protocols and three inference intensity levels (low, high, and maximum), defaulting to maximum. In multi-turn tool-calling scenarios, the previous turn’s reasoning content and tool call results must be passed back together to maintain context consistency.

6. Evaluation Signals: Capability Read from Real Sessions

The other important piece of news on the same day was a third-party ranking based on real user agent sessions. The value of this type of evaluation lies in the fact that the signal source is not a constructed test set but behavioral traces from real long-running tasks. The platform collects signals including whether users explicitly confirm task success, positive or negative feedback, model response quality after user corrections, and whether command-line operations execute successfully. These signals are combined into a “net uplift” metric.

The published data shows that the model achieved a net uplift of +5.1% across more than 17,500 real long-running sessions, ranking first among open-weight models and 13th overall on the full leaderboard. It ranked first among open-weight models in the work, coding, and conversation categories. The previous-generation model had a net uplift of +0.91% and ranked 6th among open-weight models, indicating significant improvement. On sub-metrics, command-line reliability improved to +7.9%, first among open-weight models, compared with −4.5% and 13th place for the previous generation. This jump is particularly notable because command-line reliability directly reflects model stability in multi-step tool calling—a core capability for agent scenarios.

It is worth noting that while evaluations based on real sessions are closer to actual usage, they still have limitations such as sample distribution bias and user population differences. Readers should validate against their own business scenarios rather than using rankings as the sole selection criterion.

7. Should You Self-Host?

Taking a comprehensive view, this open-weight release sends several clear signals: storage costs for ultra-large MoE models have been brought into a manageable range through 4-bit quantization; mainstream inference frameworks have adapted quickly enough to support rapid validation; and real-session evaluations show the model is competitive on agent-style tasks. Self-hosting is realistically viable for teams with the following needs:

• Data cannot leave the internal network and requires private deployment;
• Continued pre-training or fine-tuning in a specific domain is required;
• Model behavior needs to be audited and analyzed for interpretability;
• Long-term high-frequency calls make API costs higher than amortized self-hosting costs.

For most small and medium-sized teams, the more prudent path remains: first validate the business case through an API, then decide whether to self-host based on call volume and compliance requirements. The significance of open weights is not that everyone needs to download a 1.56TB file—it is that the choice is back in the hands of developers. That is itself a sign of ecosystem maturity.