2.8 Trillion-Parameter Open-Weight Model: Architecture and Local Deployment Requirements
A newly released 2.8 trillion-parameter model pairs a mixture-of-experts architecture with sparse attention and low-precision training, but its roughly 1.56 TB of weights make local deployment a substantial infrastructure challenge. This article examines the weight release, MXFP4 compatibility, reported benchmark results, and the GPU memory, interconnect, and inference framework requirements developers need to evaluate.
A research organization has released the complete weights of its flagship model on its previously announced schedule. With 2.8 trillion parameters and approximately 1.56 TB of weight files, the model is described as the largest open-weight model released to date by parameter count.
The release fulfills a two-stage commitment: provide API access first, then publish the weights. However, terabyte-scale weights put local inference well beyond the hardware available to most individual developers.
Understanding the release requires looking at four connected areas: model architecture, quantization, benchmark performance, and deployment requirements.
1. Weight Release: Size and Access
The public repository contains more than 100 files, including nearly 100 weight shards, with a total download size of approximately 1.56 TB. The weights are available without an approval process, and mirror distribution channels are also provided.
The model initially launched as an online service before its complete weights became available on the promised date. This service-first, weights-later approach allows developers to collect real-world feedback while preparing, sharding, and verifying the weight files.
The download size is itself a practical barrier. Developers need sufficient high-speed storage—preferably NVMe SSDs—as well as reliable, gigabit-class connectivity and tools that support resumable downloads. Depending on sustained throughput and connection reliability, a full download can take days.
For enterprise teams, a common approach is to download the files once to shared storage and then distribute them within the cluster.
2. Architecture: How 2.8 Trillion Parameters Are Used
The model is a natively multimodal, agent-oriented system built on a mixture-of-experts (MoE) architecture.
Its main architectural characteristics are:
- 2.8 trillion total parameters, with approximately 104 billion activated per token. Sparse activation provides a large parameter capacity while limiting the computation required for each token.
- Approximately 896 routed experts, with 16 selected per token, plus two shared experts. The shared experts handle common knowledge and can help stabilize training and mitigate uneven expert utilization.
- 93 layers and a million-token-scale context window. A separate vision encoder processes image inputs.
- Sparse attention for long-context efficiency. Compared with the previous generation, some full-attention layers have been replaced with sparse attention, reducing attention computation and GPU memory requirements at very long context lengths.
The attention changes reflect a broader design direction for long-context models: use sparsity to extend context length while preserving the ability to retrieve important information.
From an engineering perspective, MoE and sparse attention serve complementary purposes. Together, they aim to expand parameter capacity, context length, and multimodal capability while keeping inference computation manageable.
3. Quantization: A Single Released Format
The released weights use MXFP4, while activations use MXFP8. These formats are the result of quantization-aware training (QAT), rather than post-training quantization.
That distinction matters. QAT exposes the model to low-precision representations during training, allowing it to adapt to quantization error. This can preserve accuracy more effectively than quantizing a model only after training.
Only this quantized version has been released; BF16 and FP8 weight versions are not provided. That has three practical implications:
- Inference frameworks must support MXFP4. Older engine versions may not be able to load the weights directly.
- Further quantization and fine-tuning require compatibility checks. Developers have less flexibility without a higher-precision weight release and should verify their toolchain before attempting customization.
- Hardware support affects the memory budget. Deployment requires compatible low-precision computation support or a runtime path that dequantizes the weights. Dequantization can increase GPU memory consumption.
The downloadable weight format therefore needs to be evaluated alongside the inference engine and target accelerators, not as an isolated storage choice.
4. Benchmarks: Strong Coding and Agent Performance, Uneven Reasoning
According to vendor-reported comparisons, the model performs particularly well on coding and agent tasks. Reported scores include approximately 65.7% on a terminal-based coding benchmark and 51.7% on a long-horizon software engineering benchmark, exceeding some contemporary closed-weight flagship models.
The results do not indicate a lead across every category. The model reportedly remains clearly behind leading closed-weight models on science-oriented terminal tasks and some difficult reasoning benchmarks.
The vendor summarizes its performance as leading on 16 of 37 benchmarks—a mixed competitive picture rather than a clean sweep. A third-party aggregate intelligence index ranks it fourth globally.
For developers, the more useful takeaway is its uneven capability profile. Coding, tool use, and multistep agent workflows appear to be strengths, making it a candidate foundation for agent applications. Scientific reasoning workloads still require careful evaluation.
5. Local Deployment: Why “Eight High-End GPUs” Needs Qualification
At approximately 1.56 TB, the weight files alone imply a GPU memory requirement above 1.5 TB if the weights are kept resident in their released representation. Runtime memory requirements add to that baseline.
Eight GPUs with 80 GB each provide only 640 GB of aggregate memory, which is far short of the weight size. On a simple capacity calculation, loading the weights would require roughly 20 or more GPUs in that memory class, or newer accelerators with substantially more memory per device.
An eight-GPU configuration should therefore be understood only as a possible entry point with sufficiently large-memory accelerators—not as a general minimum that applies to any high-end GPU.
Beyond raw capacity, deployment must account for several infrastructure requirements:
- High-speed interconnects. Distributing MoE experts across GPUs creates substantial cross-device traffic as tokens are routed to experts. NVLink-class connectivity is important to prevent communication from becoming a bottleneck.
- Expert and tensor parallelism. The inference framework needs appropriate parallel execution support, including expert-parallel scheduling to distribute approximately 896 routed experts across devices.
- KV cache capacity. Million-token-scale contexts can create substantial KV cache requirements. Serving long-context requests further reduces the memory available for other runtime needs.
For most individual developers, running the complete model locally is not practical. Three more feasible paths are:
- Use the hosted API. This avoids managing the full deployment infrastructure.
- Rent multi-GPU cloud instances. Teams can deploy on demand and combine the inference service with object storage, container platforms, monitoring, and alerting.
- Wait for more aggressive community quantization. Lower-precision variants may reduce hardware requirements, potentially at the cost of accuracy.
The key constraint is not simply obtaining enough GPUs. Memory capacity, interconnect bandwidth, and framework compatibility must all align.
6. Conclusion
The importance of this release is not that everyone can now run a 2.8 trillion-parameter model locally. Its value lies in giving the community complete weights that embody a combination of large-scale MoE routing, sparse attention, and low-precision training.
Researchers can investigate routing behavior. Enterprises can explore private customization subject to applicable licensing and compliance requirements. Inference framework developers can optimize support for the quantization format.
Competition among open-weight models is increasingly about more than benchmark scores. Practical adoption also depends on whether the surrounding ecosystem can make these models usable. GPU memory, communication bandwidth, and toolchain support remain three of the most immediate constraints.
