2.8T-Parameter MoE Deployment: Hardware Requirements and MXFP4 Quantization
This article examines the deployment implications of a recently released 2.8T-parameter MoE model, covering weight availability and licensing caveats, sparse architecture details, MXFP4 weight quantization, and the VRAM budget for 8×192GB accelerators.
1. How Open Is the Model?
According to reports, a mixture-of-experts (MoE) model with roughly 2.8T total parameters has recently made its weights available for download. The weights total about 1.56TB and are split into nearly 100 safetensors shards. The publisher had previously offered the model through an API and later released the weights, consistent with its earlier announcement.
The term “open source” needs clarification here. What has been made public is the model weights. Whether the training data and training code are also public depends on the official repository. Whether downloaded weights can be used commercially depends on the license terms; the fact that the weights are downloadable is not sufficient by itself.
2. Architecture: Total Parameters vs. Activated Parameters
Public materials indicate that the model has about 2.8T total parameters, while forward computation for each token activates only about 104B parameters. This sparse activation keeps inference compute requirements far below what the total parameter count would suggest, but VRAM must still hold all the weights.
The model contains 896 expert networks. Each token is routed to 16 experts, and 2 additional shared experts always participate in computation. The network has 93 layers in total, supports a context window of up to 1 million tokens, and includes a vision encoder for image inputs.
Compared with the base version, the main change in the Pro version is the replacement of some full-attention layers with sparse attention. The original text states that 24 layers were replaced. Sparse attention reduces computation for long contexts, changing computational complexity from growing with the square of sequence length to something closer to linear. Its main effect is to reduce compute overhead. The impact on KV cache depends on the specific implementation and should be determined from the actual configuration.
3. Quantization Format: What MXFP4 Means for Deployment
The weights are released in MXFP4 format. Each parameter uses 4-bit floating point, and every 32 elements share one scaling factor. Activations are computed in MXFP8. The publisher provides only this version of the weights; users cannot directly download BF16 or FP8 versions.
The theoretical storage for 4-bit weights is about 0.5 bytes per parameter, which corresponds to about 1.4TB for 2.8T parameters. Scaling factors and some layers that retain higher precision add extra overhead, so the total file size is slightly higher than this estimate. Actual usage should be based on the configuration files and file manifest in the repository.
The fact that weights are released in MXFP4 does not by itself prove that the model underwent quantization-aware training. Whether quantization adaptation occurred during training should be checked in the model card or technical report.
Whether MXFP4 delivers compute acceleration depends on the hardware. Some newer-generation accelerators natively support FP4 computation. Other models require software dequantization; in that case, VRAM savings still apply, but speed improvements are limited.
4. Hardware Requirements: Calculating the VRAM Budget
First, unify the units. In decimal terms, 1.56TB equals 1560GB.
Take a common configuration of 8×192GB accelerators as an example:
- If VRAM capacity is specified in decimal GB, 8×192GB equals 1536GB, which is less than 1560GB. The weights alone will not fit.
- If VRAM capacity is specified in GiB, 8×192GiB is about 1649GB, leaving about 89GB beyond the weights. That headroom must also cover KV cache, activations, framework runtime, and communication buffers, which is far from sufficient for long-context workloads.
