Trillion-Parameter MoE Weights Open for Download: Deployment Practices and Engineering Considerations

1. Background: Trillion-Scale MoE Weights Become Downloadable

Competition among large models is shifting from benchmark scores to accessibility. When a MoE model with trillions of total parameters announces full weight downloads, commits to a clear release date, and delivers on schedule, its significance to the developer ecosystem goes far beyond a routine version update: teams can actually pull the weights locally for deployment validation, domain fine-tuning, and security auditing, rather than relying solely on hosted APIs.

The subject of this article is a mixture-of-experts model with roughly 2.8 trillion total parameters and about 104B activated per token, supporting million-token context and image-text input. Its weights are now available for download as promised, alongside published results on real-session Agent evaluations. The following sections cover five dimensions: architecture, storage, deployment, invocation, and evaluation.

2. Model Architecture: Hybrid Attention and Sparse Experts

The first step to understanding deployment cost is reading the architecture. According to the model card, the model has 93 layers, and the attention layers are not a single type: 69 layers use KDA and 24 layers use Gated DSA. Compared with the previous generation, the key change is replacing the fully attentive gated MLA layers with Gated DeepSeek Sparse Attention. The goal is clear: reduce the compute and memory overhead of long-context modeling.

It is worth unpacking why sparse attention works for long context. Full attention scales with L² in sequence length L. When context reaches hundreds of thousands or even a million tokens, the KV cache and attention matrix become the main bottlenecks. Sparse attention keeps only a subset of key-value pairs per query, pushing effective compute toward linear or sub-quadratic levels. The cost is selective information loss, which is why a gating mechanism is needed to learn which positions deserve attention — that is the engineering meaning of the “Gated” prefix.

On the expert side, there are 896 experts, 16 selected per token, plus 2 shared experts. This is a design with many experts and a low per-token activation ratio: total parameters are spread across a large number of experts, but each forward pass uses only a tiny fraction, so activated parameters (~104B) are far smaller than total parameters (~2.8T). The shared experts carry general knowledge so that every token does not depend on routed experts alone, improving training stability and inference consistency.

The vision component is a ~401M-parameter vision encoder handling image-text input. Relative to the language backbone, the vision tower is small, but in multimodal scenarios its resolution strategy and patch splitting directly affect peak memory, so it needs separate evaluation during deployment.

3. Quantization: What MXFP4 Storage + MXFP8 Activation Means

Weights are stored in MXFP4 and activations use MXFP8, with quantization-aware training according to the official description. These three points should be examined separately.

MXFP4 is a block-level floating-point quantization format, typically sharing one exponent across a small block (e.g., 32 elements), with each element represented in 4 bits. Compared with naive INT4, micro-scaling formats preserve dynamic range better, which is friendlier to expert-layer weights with large distribution variance. Storing at 4 bits directly halves weight volume or more — the repository occupies about 1.56 TB, while safetensors counts roughly 2.78 trillion parameters, and comparing the two numbers gives a rough verification of the compression ratio.

MXFP8 activations are a different matter: activations are generated dynamically at inference time, and their distribution varies sharply with input. 8-bit block floating point balances precision and throughput. The truly critical phrase is “quantization-aware training”: it means the model simulated quantization error during training rather than applying post-training quantization (PTQ) afterward. For a trillion-scale MoE, PTQ easily introduces uncontrollable bias in routing decisions and sparse attention gating, so quantization-aware training is a necessary means of preserving a capability floor.

Note that quantization is not “free cost reduction.” 4-bit weights still need to be dequantized or participate in matrix multiplication at mixed precision during inference. Actual memory usage depends on the inference framework’s kernel implementation and KV cache strategy — you cannot estimate GPU count simply by dividing weight volume by per-GPU capacity.

4. Deployment Paths: Inference Framework Support and Hardware Thresholds

From an engineering standpoint, deployment can follow three paths.

Path 1: Mainstream inference frameworks. Several open-source inference frameworks already provide deployment recipes. The value of such recipes is fixing parallelism strategy, quantization kernels, KV cache management, and batching parameters into reproducible configurations. A typical launch configuration looks roughly like this (illustrative, not real parameters):

# Illustrative: tensor parallelism + expert parallelism + quantized weight loading
python -m serving.launch \
  --model-path /models/weights \
  --tensor-parallel-size 8 \
  --expert-parallel-size 8 \
  --quantization mxfp4 \
  --max-model-len 1000000 \
  --enable-prefix-caching

The key parameters center on parallelism dimensions: a trillion-scale MoE must use both tensor parallelism and expert parallelism. Expert parallelism distributes the 896 experts across multiple GPUs to avoid expert weight overflow on a single card. Long context requires enabling prefix caching and capping maximum sequence length, or the KV cache will rapidly consume remaining memory.

Path 2: Self-built inference service. Suitable for teams with custom kernels or deep fine-tuning needs. The key is to first get the quantize-dequantize path working, then gradually tune routing load balancing — MoE tail latency often comes from uneven expert load rather than insufficient compute.

Path 3: Hosted API. If you do not want to bear hardware costs, you can call a hosted service compatible with mainstream interface protocols. Standard and high-speed tiers are available, and the reasoning effort parameter typically offers multiple levels (low/high/max, default max). There is an easy mistake in multi-turn tool-calling scenarios: you must pass back the previous turn’s reasoning content along with tool-call records, otherwise the model loses tool execution context, causing repeated calls or broken logic.

Hardware threshold. One feasible configuration from third-party estimates is a server with 8 high-memory accelerators totaling about 2.3 TB of memory, at a total system cost of roughly $550K–$650K. This is a third-party estimate, not an official recommendation; actual requirements depend on context length, concurrency, and whether quantize-dequantize optimizations are enabled. For most teams, the more realistic approach is to validate capability at low concurrency first, then decide whether to invest in self-hosting.

5. Agent Capability Evaluation: Net Improvement from Session Signals

The Agent evaluation published alongside the release deserves separate analysis, because its methodology differs from static benchmarks. It is based on over 17,500 real long-horizon sessions, extracting judgments from four signal types: whether the user explicitly confirms task success, praise and complaints about the model, response quality after user corrections, and whether shell commands execute successfully. These signals are aggregated into a “net improvement” metric.

In results, the model (at max reasoning effort) shows +5.1% net improvement, ranking 1st among open-weight models and 13th overall; the previous generation at the same effort level scored +0.91%, 6th among open models. By category, Bash reliability is +7.9%, 1st among open models (previous generation: −4.5%, 13th), and it ranks 1st among open models in all three categories: Work, Code, and Chat.

The value of this methodology is its proximity to real usage: correction response and command execution success rates in long-horizon sessions reflect Agent usability better than single-turn QA accuracy. But it has limits — signals come from user behavior and carry selection bias; “net improvement” is relative to a baseline, not absolute capability. Developers should treat it as directional input, not the sole basis for model selection.

6. Deployment Checklist

Based on the analysis above, here are actionable recommendations:

1. Estimate KV cache before weights: In long-context scenarios, memory bottlenecks usually appear in the KV cache first, not the weights. Plan backward from maximum concurrency and maximum sequence length.
2. Reuse official recipes first: The parallelism and quantization configurations provided by inference frameworks are validated; the marginal benefit of custom kernels is low in early stages.
3. A/B test quantization precision: Compare output consistency between quantized and high-precision versions on your own task set, focusing on routing-intensive reasoning tasks.
4. Preserve full context in Agent calls: For multi-turn tool calling, always pass back reasoning content and tool records. This is the most common integration error.
5. Read sub-scores, not just totals: Sub-metrics like Bash reliability and correction response carry far more information for engineering decisions than overall leaderboard rank.

Opening trillion-scale MoE weights for download essentially lowers the barrier to capability acquisition from business negotiation to engineering competence. The real dividing line is no longer whether you can obtain the model, but whether you can get quantization, parallelism, caching, and context management right.