A 2.8T-Parameter Open-Weight Model: What to Assess Before Adoption
A vendor has released the full weights of a 2.8-trillion-parameter multimodal agent model, with a 1.56 TB download, a mixture-of-experts architecture, MXFP4/MXFP8 quantization, self-reported benchmarks, a custom license, and substantial GPU requirements. This article explains what technical teams should examine before choosing a deployment and licensing path.
1. What Was Released
In early November, a vendor fulfilled a public commitment by releasing the complete weights of a flagship large model. Anyone can download it without approval. The model has 2.8 trillion total parameters, and the download is about 1.56 TB across 112 files, including 96 safetensors shards. The repository was created two days before the formal release, was briefly closed on release day, and was restored in the evening. Synchronized copies are also available from major model hosting communities.
The parameters below follow the official model card. This article does not constitute procurement advice.
2. Architecture: How 2.8 Trillion Parameters Are Composed
This is a natively multimodal agent model trained from the previous-generation model. The model card discloses the following main structural details.
- Mixture-of-experts (MoE) structure: 2.8 trillion total parameters, with about 104B parameters activated per inference. MoE means the model distributes parameters across many expert subnetworks and invokes only a subset for each input.
- Expert configuration: 896 experts in total, with 16 dynamically selected per token. Two additional shared experts always participate in computation.
- 93 layers, with a 1 million token context window.
- Vision component: an independent vision encoder provides image-understanding capability.
Compared with the previous generation, only one structural change was made: 24 full-attention layers were replaced with sparse attention (DSA). Sparse attention attends to only a subset of historical positions, reducing compute overhead under very long contexts. Fewer changes make it easier to reuse existing inference optimizations and adaptation code, which is a conservative approach for continued training.
Weights use MXFP4 quantization, and activations use MXFP8. MXFP is a micro-scaling floating-point format that stores values in fewer bits and shares one scaling factor per group of data. This type of quantization is a product of quantization-aware training (QAT), meaning quantization error is simulated during training rather than compressed after training. The vendor released only this version, with no BF16 or FP8 variant.
This creates a trade-off for deployers. QAT usually retains more accuracy than post-training quantization. However, if an inference framework does not yet support MXFP4, teams must spend time on adaptation or wait for community support.
3. Performance: How to Read Self-Reported Data
The vendor provided a benchmark comparison table with the model. All results are self-reported. Independent third-party retesting takes time, so these numbers should be treated as references at this stage.
Based on the public numbers, results are mixed:
- Coding and agentic tasks are relatively strong, and the model outperforms some closed first-tier comparison models on certain code benchmarks.
- Science-related and some deep-reasoning benchmarks lag clearly behind the strongest competitors.
- The original text mentions that a third-party intelligence index gave the model a high ranking, but the specific name, version, and evaluation date require readers to verify independently. This article does not cite a specific rank.
Large-model evaluation should not rely on a single leaderboard. A leaderboard reflects the distribution of model capabilities on public prompts, while production environments care about whether that distribution matches their own business inputs. A more reliable approach is to build a private evaluation set from real business data, covering common requests, long-tail requests, and failure cases, define scoring criteria, and then run blind comparisons. Larger sample sizes produce more stable conclusions.
4. License: Open Weights Is Not Open Source
The most important item to examine separately in this release is the license. The repository uses a custom license agreement and is labeled as “other” on the model hosting platform. It is not a standard open-source license such as MIT or Apache, and its terms are broadly consistent with those of the previous-generation model.
The license structure can be summarized as a “permissive core plus two scale-based restrictions”:
- The core grant is relatively permissive, allowing free use, copying, modification, distribution, fine-tuning, and commercial use.
- Restriction one: if the model is provided as a service and revenue reaches the threshold stated in the original text, a separate agreement with the rights holder is required before commercial use.
- Restriction two: if a commercial product’s monthly active users exceed a certain scale, an additional agreement is likewise required.
These two restrictions target larger external businesses. Individual developers, research teams, and small-to-medium applications usually will not trigger them. However, the revenue figures and user thresholds given in the original text cannot be individually verified here; the original license text and official links must govern.
This model is often described as “open weights”: the code is readable and the weights are downloadable, but the rights holder reserves discretion over usage boundaries. Before adoption, enterprises should complete the following checks:
- Save the license version number and download date.
- Check whether their own revenue, user volume, and service form trigger the terms.
- Confirm whether redistribution or providing a fine-tuned product externally is involved.
- Have the legal department complete a review and retain records.
The above does not constitute legal advice.
5. Deployment: Estimating GPU Memory Requirements
The 1.56 TB weight volume directly determines the deployment threshold. The estimate is as follows: MXFP4 uses about 4 bits per parameter on average, so the weights for 2.8 trillion parameters are about 1.4 TB by themselves. Adding scaling factors, activations, KV cache, and runtime overhead makes actual GPU memory requirements higher.
Based on this estimate, if using single 80 GB accelerators, loading the weights alone requires roughly 18 to 20 cards. After accounting for KV cache and activations, the actual card count is usually higher. This number does not include parallel strategies or redundancy design; it only indicates the order of magnitude.
The original text also mentions an official claim of “starting from 8 cards,” but it does not specify the card type, whether multiple nodes are used, context length, or batch conditions. Eight 80 GB cards total 640 GB, which cannot hold more than 1.4 TB of weights. Therefore, this claim holds only when the card type, offloading method, and parallel conditions are all clearly defined. Readers should check the official deployment documentation to confirm the specific configuration.
Three practical conclusions follow:
- Consumer-grade hardware cannot carry this model; local operation is meaningful only for organizations with server-grade GPU clusters.
- For small and medium teams, a more realistic option is to rent cloud GPU instances on demand or call a hosted API directly.
- If self-hosting, teams need to determine the distributed inference framework, tensor parallelism and pipeline parallelism strategies in advance, and verify compatibility of the quantization format with the target framework.
6. How to Choose Among Three Adoption Paths
The table below can serve as a starting point for initial comparison.
| Approach | Upfront investment | Operational burden | Data compliance | Suitable scenarios |
|---|---|---|---|---|
| Hosted API | Low | Low | Depends on provider terms | Rapid validation, general Q&A |
| On-demand cloud GPU | Medium | Medium | Controllable, with dedicated instances required | Research fine-tuning, staged batch inference |
| Self-built cluster | High | High | Most controllable | Highly sensitive data, long-term high load |
Before choosing, clarify three questions: Can data leave the internal network? What is the average daily call volume? Does the team have the operational capability for distributed inference? Different answers lead to different conclusions.
7. Conclusion
This release sets a record for open-weight models in terms of parameter scale and download size. For developers, the barrier to accessing top-tier model capabilities has indeed been lowered, but there is still a large gap between downloading and deployment. First determine whether the need is research fine-tuning, private inference, or direct API calls, and then decide on the investment path. At the same time, as “open weights plus custom licenses” becomes a common model, enterprises should incorporate license compliance into the standard technical selection process.
