A 2.8T-Parameter Open-Weight Model: Architecture, Quantization, and Local Deployment

    This article examines a 2.8-trillion-parameter open-weight model. It covers the mixture-of-experts architecture and sparse attention, the MXFP4 and MXFP8 quantization approach, licensing conditions, and the memory and parallelism planning needed to run the model on your own hardware.

    1. From API Access to Downloadable Weights

    The model first launched as an API and web service. The vendor committed to releasing the full weights within about two weeks and did so on schedule. The weights are now available from a public model repository, with no application required. The repository contains 112 files, including 96 safetensors shards, for a total size of about 1.56 TB. The vendor provides only one low-bit quantized version. It does not release BF16 or FP8 weights.

    This release sequence is worth noting. Releasing an API first and the weights later is a common strategy for leading models. It gives the vendor time to observe real workloads and security risks through its online service. It also gives the community time to add support in inference frameworks. For developers, the key question is not when the weights were released but what they can do with them. The rest of this article addresses that.

    2. Architecture: How the 2.8T Parameters Are Organized

    1. Mixture of Experts and Sparse Activation

    The model uses a Mixture of Experts (MoE) architecture. It has 2.8T total parameters, but each token activates only about 104B of them. The network has 93 layers. It contains 896 routed experts, and each token is routed to 16 of them. Two shared experts take part in the computation for every token.

    MoE separates model capacity from inference compute:

    • During training, the model can absorb knowledge using far more parameters than a dense model would allow.
    • During inference, the compute per token (FLOPs) corresponds only to the small number of activated experts. The forward-pass cost is close to that of a dense model with around 100 billion parameters.
    • The trade-off is that all parameters must fit in memory. Inference compute is manageable, but the full model still has to fit in the memory pool.

    2. Sparse Attention for Million-Token Context

    Compared with the previous generation, the main architectural change is that 24 full-attention layers are replaced with sparse attention. The goal is to reduce compute and KV cache overhead for contexts up to one million tokens. Full attention cost grows quadratically with sequence length, which is impractical at that scale. Sparse attention computes only a subset of the key attention connections. This substantially reduces the cost of long sequences while maintaining modeling capability.

    For developers, this makes workloads such as long-document analysis and large codebase understanding feasible in self-hosted deployments. KV cache memory still has to be calculated separately when you plan capacity.

    3. Multimodal Design and Agent Positioning

    The model is natively multimodal. It includes a built-in vision encoder that accepts image input directly. The vendor describes it as an agent model, emphasizing tool calling, multi-step task planning, and execution rather than plain chat completion.

    According to the vendor’s self-reported benchmarks, the model performs strongly on terminal operation and software engineering tasks. It shows clear weaknesses on scientific reasoning benchmarks. Model selection should therefore match the task type rather than rely on a single leaderboard ranking.

    3. Quantization: What MXFP4 and MXFP8 Mean in Practice

    Weights are stored in MXFP4 (4-bit) format, and activations are computed in MXFP8 (8-bit) format. This is the result of quantization-aware training (QAT). Unlike post-training quantization, QAT simulates low-bit error during training, so the model weights adapt to the quantized representation. Accuracy loss is much smaller than with post-training quantization.

    The 1.56 TB size already reflects 4-bit quantization. At BF16 precision, 2.8T parameters would require about 5.6 TB, which is nearly impossible to distribute. This also explains why the vendor released only one version. For the community, low-bit weights are the only practical option.

    Low-bit inference places specific requirements on hardware and kernels. GPU architectures differ widely in their support for 4-bit matrix multiplication. Before deployment, confirm that your inference framework and GPU generation support the MXFP4 and FP8 data types. Otherwise, the model may dequantize weights before computation, which wastes performance.

    4. Licensing and Compliance: Open Does Not Mean Unrestricted

    This kind of partially open release usually comes with conditions. A common approach is a modified open-source license with two main restrictions. First, if a product or service built on the model exceeds a certain monthly active user threshold (for example, 100 million users), the developer must contact the vendor to obtain additional authorization. Second, derivative models must keep a source identifier in their names. The weight repository generally also requires users to follow an accompanying usage policy.

    Enterprise users should complete three steps before adopting the model. Read the full license text, not just a summary. Confirm whether your business scale reaches the commercial authorization threshold. Assess whether the naming and attribution requirements for fine-tuned derivative models affect your release process. Compliance costs should be part of the selection decision, not addressed after the fact.

    5. Local Deployment: Why an 8-GPU Setup Is the Starting Point

    1. Memory Estimation

    The 1.56 TB of weights means the memory pool must exceed 1.6 TB, even before counting activations and KV cache. On mainstream data center GPUs with 80 GB of memory, at least 20 cards are needed just to hold the weights. The official recommendation of starting from 8 GPUs typically corresponds to high-memory cards with 192 GB each (8 × 192 GB ≈ 1.54 TB). Once KV cache and overhead are added, that is already very tight.

    For planning, estimate memory as the weights plus a 30% margin. For long-context workloads, add the KV cache budget on top to determine the actual number of GPUs and the minimum memory per GPU.

    2. Parallelism Strategy

    A common configuration for an 8-GPU deployment combines tensor parallelism (TP), which splits each layer’s computation across the 8 GPUs, with expert parallelism (EP), which distributes the 896 experts across devices. MoE models suit EP well. Routed experts can be sharded naturally, and most communication occurs during the all-to-all expert dispatch step.

    This step places high demands on intra-node interconnect bandwidth. Prefer server configurations with high-speed interconnects such as NVLink-class links. For multi-node deployments, evaluate whether network bandwidth will become a bottleneck.

    3. Inference Framework and Optimizations

    Before deployment, confirm that your inference framework supports the model’s architecture, including sparse attention, multimodal input, and the MXFP4 weight format. Recommended optimizations include prefix caching to reuse the KV cache for long system prompts, continuous batching to increase thro