Deploying a 2.8T Parameter Open-Weight Model: What 1.56TB Means for Developers
The release of a 2.8T parameter open-weight model marks a major milestone for ultra-large Mixture of Experts (MoE) architectures. This article analyzes what developers and enterprises need to know about its internal structure, mixed benchmark performance, custom licensing terms, and the practical realities of local deployment, including an 8-GPU minimum requirement and native MXFP4 quantization.
From API to Open Weights
The model was initially launched in late October via an API and web interface, with the developers promising that full weights would be released within two weeks. In early November, the weights appeared in public model repositories as scheduled, available for download without application: 112 files, 96 safetensors shards, totaling approximately 1.56TB.
This is the largest open-weight model released to date. For developers following large language model technology, the release of the weights raises three key questions: What is the internal structure of the model? How does the license define usage boundaries? And can ordinary teams actually run it locally?
Model Architecture: Making 2.8T Parameters Feasible for Inference
A total of 2.8T parameters does not mean that all 2.8T parameters are calculated during every inference pass. The model employs a Mixture of Experts (MoE) architecture with 896 expert networks, activating only 16 per token, alongside 2 always-active shared experts. This results in approximately 104B activated parameters per forward pass. This design of “large total parameters, small activated parameters” is the mainstream approach for current ultra-large models to balance compute costs with model capacity.
Other key specifications include 93 network layers, a 1-million token context window, and a built-in vision encoder capable of processing image inputs. It is positioned as a natively multimodal agent model, continuously trained from its previous generation. Compared to its predecessor, the only structural change is replacing 24 full-attention layers with sparse attention mechanisms to reduce computational overhead in long-context scenarios.
It is crucial to note that the weights are stored in MXFP4 and activations are computed in MXFP8. This is a product of quantization-aware training (QAT). The developers have only released this single version, with no BF16 or FP8 weights available. This means users cannot manually “de-quantize” the model to obtain higher precision weights. Fine-tuning must be performed on this low-precision foundation, placing higher demands on the numerical stability of training frameworks.
Benchmark Results: A Mixed Bag
According to the vendor’s self-reported comparative data:
- Terminal-Bench 4.0: Scored 65.7%, slightly below Competitor A’s 66.4% but above Competitor B’s 58.2%.
- SWE Marathon: Scored 51.7%, surpassing Competitor A’s 46.3%.
- Terminal-Bench-Science: Scored only 38.1% (Competitor A scored 63.3%).
- HLE-Diamond: Scored 39.4% (Competitor A scored 60.9%).
The vendor claims that out of 37 benchmarks, the model outperforms the strongest competitor in 16. Overall, the conclusion is clear: the model excels in coding and agent-related tasks, but still lags in scientific and certain reasoning tasks. A third-party evaluation agency gave it an intelligence index of 55, ranking fourth globally. For users, the takeaway is to cross-validate model selection against specific business scenarios (e.g., code generation, multi-step tool calling, long document processing) rather than relying on a single total score.
Licensing: Open Weights vs. Open Source
This is perhaps the most critical aspect of the release to discuss. The repository uses a custom license, marked as “other” on model platforms, rather than a standard open-source license like MIT or Apache.
The core terms are indeed MIT-like: it allows free use, copying, modification, distribution, fine-tuning, and commercial use. However, it appends two restrictions targeting large-scale external businesses:
- If providing the model as a service and the company’s cumulative revenue over the past 12 months exceeds $20 million, a separate agreement must be signed with the publisher before commercial use.
- If a commercial product’s monthly active users (MAU) exceed a certain threshold, it also triggers additional agreement requirements.
In other words, this is a classic “open weights” model. It sets virtually no barriers for individual developers, research teams, and small-to-medium commercial applications, but large-scale commercial use requires negotiation. Enterprises building products on this model should carefully evaluate whether their revenue and user scale will breach these terms and seek legal counsel if necessary.
Local Deployment: An 8-GPU Minimum
The 1.56TB weight size directly dictates the deployment barrier. Estimating at MXFP4 precision, the 104B activated parameters correspond to the VRAM required for a single inference pass. When adding the overhead for KV cache, context buffers, and the vision encoder, a single GPU—even the latest 80GB+ VRAM specifications—cannot fully support it. Therefore, the vendor recommends an 8-GPU minimum.
For teams planning local deployment, here are practical recommendations:
- Hardware Planning: At least 8 high-VRAM accelerator cards connected via NVLink or a high-speed interconnect bus to form a single-node multi-GPU topology. Multi-node deployments will require additional network bandwidth budgeting.
- Download and Verification: The 96 shard files should be downloaded in parallel using resumable download tools. After downloading, verify the overall hash to avoid loading failures caused by corrupted shards.
- Inference Framework Selection: You need a mainstream inference engine that supports MXFP4 quantization and enables multi-GPU sharding with expert parallelism. An illustrative loading process is as follows:
# Pseudocode illustration: Multi-GPU loading of ultra-large MoE weightsfrom vllm import LLMllm = LLM(model="local/path/to/model",tensor_parallel_size=8, # 8-way tensor parallelismquantization="mxfp4", # Native MXFP4 weightsmax_model_len=131072, # Truncate context based on VRAM budgetenable_expert_parallel=True, # MoE expert parallelism)
- Context Trade-offs: The 1-million token context is theoretically usable, but the KV cache grows linearly with context length. In actual deployment, adjust
max_model_lento a level sufficient for your business needs based on your remaining VRAM. - Cloud Alternatives: If you lack 8-GPU hardware, you can rent GPU instances on-demand from cloud providers. Object storage services can be used to store weight shards, and container platforms paired with high-performance storage volumes can simplify the orchestration of multi-node inference services. Many cloud platforms also offer managed large model inference services, eliminating local operational costs.
Conclusion and Future Outlook
The release of 2.8T parameter open weights signifies that ultra-large MoE models have entered the “downloadable” era. However, it also draws three clear boundaries: the uniqueness of quantized weights limits precision flexibility; the custom license sets negotiation thresholds for large-scale commercial use; and the 8-GPU hardware requirement leaves local deployment to teams with real compute budgets.
For individual developers, cloud APIs remain the most cost-effective way to experience the model. For research institutions, these weights provide a rare sample for studying ultra-large MoE structures. For enterprise teams, a comprehensive TCO evaluation is necessary to balance license compliance and compute costs. As technologies like sparse attention and low-bit quantization-aware training mature, the barrier to opening ultra-large models will gradually decrease. However, the “trade-off between scale and usability” will remain the core proposition of large model engineering for a long time to come.
