Deploying a 2.8T-Parameter Open-Weights MoE Model: 8 GPUs, MXFP4 Weights, and Practical Requirements
This article reviews the release of a 2.8T-parameter open-weights MoE model with 1.56TB of weights, explains why the real deployment barrier has shifted from downloading the model to running it, and covers model architecture, MXFP4/MXFP8 quantization, GPU memory and KV-cache planning, MoE parallelization, licensing limits, benchmark caveats, and a practical deployment decision path. It is intended to help engineers and technical decision-makers evaluate whether this model can run in their own environment and under what conditions it makes operational sense.
1. Release overview: a planned open-weights release
On November 4, 2025, a model vendor released the full weights of a new large language model in a public model repository, as promised. The model has 2.8 trillion total parameters, and the weight files total about 1.56TB. The vendor describes it as the strongest open-weights model available for download at the time.
The weights can be downloaded directly without applying for access. However, this “openness” comes with two important constraints:
- Running the model locally requires at least 8 GPUs/accelerators.
- Usage is subject to a custom license rather than a standard open-source license.
1.1 Timeline
- October 24: The model first became available through an API and a web interface. The vendor said the full weights would be released by November 4.
- November 2: The weight repository was created but remained closed.
- November 4, morning: The repository was still inaccessible.
- November 4, evening: The repository was officially opened. Anyone could download the weights without applying for access.
The gap between repository creation and official release was only two days. The fact that the repository was closed on the morning of November 4 and opened that evening suggests the release timing was deliberately controlled: first deliver the promised weights, then let the community verify them.
1.2 What is inside the weight package
- The repository contains 112 files. Among them, 96 are safetensors weight shards, each averaging about 16GB, totaling approximately 1.56TB.
- The remaining 16 files generally include supporting files such as model configuration, tokenizer, weight index, and vision encoder. The exact contents should be confirmed by the repository manifest.
- The model’s weights are directly downloadable, but local execution requires sufficient hardware and software support.
1.3 Why this release matters
- Record scale: This is the largest open-weights model released to date, with parameter count far exceeding previous comparable open-weights models.
- The barrier has shifted: Once model size enters the trillion-parameter range, the main constraints are no longer “Can I download it?” but “Do I have enough memory? Does my framework support it? Does the license allow my use case?”
- Engineering implications matter more than parameter count: The new problems this model creates for local inference frameworks are more important for developers than the headline parameter number.
The vendor positions the model as a natively multimodal agent model, trained further from a previous generation model in the same series. This article focuses on what happens after the weights become available: what is inside the release, how the license works, and whether general users can actually run it.
2. Model architecture: how 2.8 trillion parameters are organized
2.1 Parameter scale and activation scale: compute is saved, memory is not
The model uses a mixture-of-experts architecture. Key parameters are:
- Total parameters: 2.8 trillion.
- Activated parameters per token: approximately 104 billion, about 3.7% of total parameters.
- Expert configuration: 896 routed experts, with 16 experts selected per token. There are also 2 shared experts, which all tokens pass through.
- Layers: 93.
- Context length: 1 million tokens.
- Vision encoder: MoonViT-V2, supporting image-text input.
Compared with the previous generation, the architecture has one main change: in 24 layers, full attention was replaced with sparse attention to save compute for long-context use cases. Whether an inference framework already supports this structure directly determines real-world long-context performance.
The point of MoE is to separate “compute cost” from “memory usage”:
- Compute cost: For each generated token, only about 104 billion parameters are activated. The per-token compute is roughly comparable to a dense model at the 100-billion-parameter scale.
- Memory usage: Regardless of how many experts are activated, the weights of all 896 experts must remain resident in GPU memory. The model cannot predict which experts the next token will route to. If inactive experts were swapped out, they would need to be reloaded for each token, making the cost even higher.
This is the easiest point for ordinary developers to misjudge. Seeing “104B activated parameters” may lead to the assumption that one high-memory GPU can run the model. In reality, the bottleneck is GPU memory, not only compute.
2.2 Sparse attention: a targeted optimization for long context
In standard full attention, every token must compute relevance with all previous tokens. As sequence length grows, both compute load and KV-cache read load increase dramatically. In this model, 24 layers were changed to sparse attention, so each token only attends to a selected subset of historical content according to specific rules, rather than the entire context.
This has two sides for users:
- Benefit: In long-context scenarios, time-to-first-token and per-token generation latency can both be reduced significantly.
- Caution: Different layers may use different attention implementations. Some inference frameworks need to identify the model configuration layer by layer and select the corresponding execution path. Otherwise, the framework may fall back to full attention, severely degrading long-context performance.
Before production deployment, it is advisable to run stress tests with inputs of more than 100,000 tokens to confirm that long-context acceleration is actually working, rather than the request taking a generic path.
3. Quantization: only MXFP4 weights are officially provided
3.1 What MXFP4 is
The released weights use MXFP4 format, and activations use MXFP8 format. Both are quantization-aware formats introduced during training.
MX-series formats belong to block-level floating-point formats: every 32 elements form a group and share one scale factor, while each element in the group retains only a few bits. In MXFP4, each parameter occupies only 4 bits. The theoretical storage size is about one quarter of BF16 and about half of FP8. MXFP8 activations provide 8-bit precision during computation.
3.2 Three direct impacts on users
Hardware requirements move upward: Inference frameworks must support loading FP4 weights and FP8 computation. If the hardware does not support this, the weights must first be dequantized into higher precision before running. In that case, memory usage and compute consumption increase sharply, and the “8 GPUs to start” premise no longer holds.
Fine-tuning paths are limited: The vendor has released only this quantized version, with no BF16 or FP8 version. This also means there is no high-precision base model available for full-parameter fine-tuning. For local secondary development, parameter-efficient fine-tuning methods such as low-rank adaptation may be considered on top of the quantized weights, but framework support must be verified first, followed by a small-sample precision evaluation.
No precision reference is provided: Because no official high-precision version is available, precision validation can only be performed using the business’s own evaluation sets.
4. Hardware requirements: what an 8-GPU minimum really means
4.1 Memory math: start with a simple division
Start with a simple calculation to understand the scale:
Total weight size: 1.56 TB = 1560 GBMinimum GPU count: 8Per-GPU weight load: 1560 / 8 = 195 GB
Each GPU would need to hold about 195GB just for the weights. For example, if a common single-GPU memory configuration is 192GB, then 8 such GPUs provide only 1536GB total, which is theoretically not even enough to hold the weights. Therefore, the “8-GPU minimum” corresponds to higher-memory GPUs or a specific GPU combination.
Even if memory is just barely sufficient, additional headroom must be reserved for KV cache, activations, and communication buffers. A safer engineering configuration is to start with 16 GPUs, or use a hybrid approach such as 8 GPUs with tiered offloading.
4.2 KV cache: the real memory killer for long context
KV-cache usage can be roughly estimated as:
KV cache bytes ≈ 2 × layers × KV heads × head dimension × sequence length × bytes per element(The factor of 2 accounts for caching both K and V at each position.)
Two coefficients in this formula are not disclosed in the model card:
- KV heads: If grouped-query attention is used, this value may be much smaller than the number of query heads.
- KV storage precision: The model card does not specify this.
The numbers below are illustrative and only meant to convey the scale. Always measure before deployment:
Assume KV heads = 8, head dimension = 128, KV stored in FP8 (1 byte), layers = 93KV per token = 2 × 93 × 8 × 128 × 1 B = 190,464 B ≈ 186 KB1M-token single request = 186 KB × 1,000,000 ≈ 178 GiB
A single request using the full 1-million-token context can consume about 178GiB of KV cache, which is comparable to the total memory of 8 192GB GPUs. Real production workloads often involve multiple concurrent requests. Therefore, long-context capability does not mean “many long requests can be processed simultaneously.” KV-cache management is the key to capacity planning.
Common engineering approaches include:
- Paged KV cache to reduce fragmentation.
- Prefix reuse, so multi-turn conversations and shared system prompts are stored only once.
- KV quantization, such as the FP8 storage assumed above.
- Limits on per-request context length and quotas on concurrency.
4.3 Parallelism: MoE deployment cannot avoid all-to-all communication
MoE inference requires sending tokens to the GPUs that hold the target experts, computing on those GPUs, and then collecting the results. This is all-to-all communication. When 896 experts are distributed across 8 GPUs, each GPU needs to hold more than 100 experts on average. This communication is sensitive to inter-GPU interconnect bandwidth:
- With high-speed interconnect, communication overhead is relatively manageable.
- When only standard PCIe is available, communication can become a bottleneck and throughput can drop significantly.
Common deployment combinations include:
- Expert parallelism: Experts are split across GPUs. This suits models with many experts but depends heavily on inter-GPU interconnect.
- Tensor parallelism: A single matrix operation is split across multiple GPUs. This is suitable for multi-GPU single-node setups.
- Pipeline parallelism: Layers are split across different nodes, making cross-machine scaling easier.
- Prefill and decode separation: Prefill is compute-heavy, while decode is more memory-bandwidth-heavy. Separating them allows independent scaling.
4.4 Pre-download checklist
- Does the total GPU memory, after accounting for weights, leave at least 20% headroom?
- Are the GPUs connected by high-speed interconnect, or only PCIe?
- Does the inference framework support this model’s attention structure and MXFP4 weights?
- Does the framework support paged KV cache and prefix reuse?
- Does the hardware natively support FP4/FP8 computation?
- Is there at least 2TB of free local disk space for weights and temporary verification files?
- Is the download path stable? At 1Gbps, 1.56TB takes about 3.5 hours; at 10Gbps, about 21 minutes, assuming ideal bandwidth.
5. License boundaries: open weights are not the same as open source
This is the most easily overlooked part of the release, and the most important for enterprises.
5.1 Facts
- The model does not use standard open-source licenses such as MIT or Apache. Instead, it is accompanied by a vendor-specific license. The license is marked as “Other” in the model repository.
- Compared with the previous generation model in the same series, the license terms are identical word for word, except for the name.
5.2 Breakdown of the license
The core grant is permissive: Users may use, copy, modify, distribute, fine-tune, and commercially use the model. Individual developers, research institutions, and most small and medium-sized enterprises are not affected.
Two restrictions are layered on top, both aimed only at larger external-facing businesses:
- Revenue threshold: If a company offers the model as a service (MaaS) and has cumulative revenue of more than USD 20 million over a trailing 12-month period, it must sign a separate agreement with the licensor before commercial use.
- Monthly active user threshold: If a commercial product has more than 100 million monthly active users, separate authorization is also required. The exact metric should follow the original license text.
5.3 Why this is not “open source”
Open-source definitions require that a license must not restrict the field of use or commercial scenarios. This permissive license with two added conditions does not meet that requirement.
A more accurate term is “open weights”: the weights can be obtained and modified, but large-scale commercialization requires separate negotiation.
For practical business use:
- Teams that expose inference through self-hosted APIs and exceed USD 20 million in revenue should complete commercial authorization before launch.
- Consumer applications targeting massive user bases must also check the monthly-active-user clause.
- License terms may be updated together with the model. Enterprises should include the license text in compliance review processes and recheck it periodically.
6. How to read performance: interpreting vendor-reported results
6.1 Summary of vendor-reported results
The vendor’s results come from internal testing, compared with another vendor’s flagship closed-source model. They are for reference only:
| Benchmark area | This model | Comparison model | Result |
|---|---|---|---|
| Terminal coding tests (version 4.0) | 65.7% | 66.4% | Slightly lower |
| Long-horizon software engineering tasks | 51.7% | 46.3% | Ahead |
| Terminal-based science tasks | 38.1% | 63.3% | Clearly behind |
| High-difficulty PhD-level subject tests | 39.4% | 60.9% | Clearly behind |
| Number of leading results among 37 benchmarks | 16 | — | Mixed |
According to information from an independent evaluation organization, the model’s overall intelligence index is 55, ranking fourth globally.
6.2 Conclusion: strong in coding and agents, weaker in scientific reasoning
From the vendor’s reported data:
- Strengths: Terminal coding, long-horizon software engineering, and other coding and agent-related tasks. The model is roughly on par with top closed-source models and even surpasses them in some areas.
- Weaknesses: Scientific-reasoning tasks lag behind closed-source models by more than 20 percentage points. PhD-level subject problems are also clearly behind.
In one sentence: this model is more like a “working model” than a “problem-solving model.”
6.3 Recommendations for self-testing
The vendor’s prompts, temperature settings, and scoring methods are not public. Different teams may reproduce different results. It is recommended to run small-scale evaluations using your own business data:
- Fix the evaluation set and decoding parameters.
- Sample each example multiple times and use the average score.
- Manually spot-check key samples.
7. Deployment path: from “Can it run?” to “Can we afford to run it?”
7.1 Decision tree: first decide whether local deployment is necessary
Business scenario: If the main use cases are coding assistance, agent automation, and long-document processing, local deployment may deliver high value. If the main use case is scientific reasoning, compare alternative solutions first.
Hardware conditions: Can the current GPU count hold the weights and still leave at least 20% headroom? If not, evaluate the return on investment of adding GPUs.
Operations capability: Does the team have experience with distributed inference tuning, monitoring and alerting, and failure recovery? If not, it may be better to start with cloud resources.
License compliance: If revenue is approaching USD 20 million or monthly active users are approaching 100 million, plan commercial authorization in advance.
7.2 Cost tipping point between self-hosting and API
Whether self-hosting or API usage is cheaper depends on monthly call volume. A simplified formula can be used for estimation. The parameters are illustrative only; actual pricing should govern:
Self-hosting cost = hardware depreciation + power and facility costs + operations labor + engineering effortAPI cost = input token volume × input price + output token volume × output priceSet the two equal to calculate the monthly cost tipping point.If monthly call volume is below the tipping point -> API is cheaper, because hardware and operations investment are avoided.If monthly call volume is above the tipping point -> self-hosting has a cost advantage.
For businesses near the tipping point or with highly variable call volume, it may be reasonable to start with on-demand cloud resources and decide whether to self-host based on actual load.
7.3 Service monitoring: watch these four metrics
- Time to first token (TTFT): Determines interaction experience and deserves special attention in long-context scenarios.
- Time per output token (TPOT): Determines the smoothness of streaming output.
- Throughput (tokens/s): Determines output per unit of compute and is the basis for capacity planning.
- GPU memory utilization and queue time: Reflect whether KV cache is saturated and whether requests are queuing.
Logging services should record request parameters and outputs. Monitoring and alerting should notify the team when memory or latency exceeds thresholds. Only then can the system move from “it can run” to “it can run stably.”
If offering generative AI services to the public, complete any applicable regional compliance requirements, security assessments, and related obligations.
8. Summary
Open-weights models have entered the trillion-parameter era: 2.8 trillion parameters and 1.56TB of weights give the phrase “strongest downloadable model” real physical meaning.
The barrier has shifted from downloading to running: An 8-GPU minimum means each GPU must hold about 195GB of weights. Ordinary hardware requires new capacity planning.
Quantization determines hardware selection: Only MXFP4 weights are provided. Hardware and framework support are both required.
The license has additional conditions: Revenue and monthly-active-user thresholds mean the model is suitable for most small and medium-sized enterprises and researchers, while large-scale commercialization requires advance coordination.
Performance is specialized: Coding and agent capabilities are strong, while scientific reasoning still lags. Model selection should be based on your own business evaluations.
