2.8T Open-Weight Model: Architecture, Benchmarks, Licensing, and Deployment
A new 2.8T-parameter open-weight multimodal agent model has been released with full weights and a 1.56TB download size, making it one of the largest downloadable open-weight models available. This article analyzes its architecture, benchmark strengths and gaps, custom license restrictions, and the hardware and cloud deployment requirements that enterprises must evaluate before adoption.
A research team has recently released the full weights of its latest ultra-large parameter model. With 2.8T total parameters and a 1.56TB download size, the model is regarded as one of the largest open-weight models currently available for download. The release has sparked discussion in the open-source community and raised new questions about model architecture evolution, licensing norms, and enterprise deployment. This article examines deployment practices for such ultra-large models across four dimensions: technical architecture, performance evaluation, license compliance, and deployment barriers.
Architecture: MoE, Sparse Attention, and Multimodal Integration
The model is positioned as a natively multimodal Agent model and was further trained from the previous-generation base model. Its architecture reflects several core trends in large model development.
Sparse Activation and Expert Networks
The model has 2.8T total parameters but activates only about 104B parameters per inference. This is a typical Mixture-of-Experts architecture with 896 expert networks. For each token, 16 experts are selected and activated during inference, along with 2 shared experts that participate in every computation. This design allows the model to greatly expand knowledge capacity and expressive capability without significantly increasing computation per inference. For developers, this means inference latency and cost can be effectively controlled, but GPU memory usage remains a major challenge.
Attention Evolution and Long Context
Structurally, the model makes a key adjustment compared with the previous generation: some layers, such as 24 layers, replace traditional full attention with sparse attention (DSA). As context length extends to 1 million tokens, traditional full attention would cause compute consumption to grow quadratically. Introducing DSA substantially reduces computational complexity in long-context scenarios and provides compute support for Agent tasks such as long-document processing and repository-level code understanding.
Quantization-Aware Training and Multimodal Capabilities
The model weights use the MXFP4 format, while activations use MXFP8. These are produced through quantization-aware training (QAT). Notably, the team released only this quantized version and did not provide higher-precision versions such as BF16 or FP8. In addition, the model integrates a vision encoder and has native image understanding capability, moving further toward a multimodal Agent direction. The single quantized release reduces storage size but also imposes certain constraints on downstream fine-tuning and hardware adaptation.
Performance and Benchmark Analysis
Evaluating an ultra-large model requires looking beyond scale to real performance across benchmarks. Based on the team’s self-reported data, the model shows an uneven performance distribution.
Strong Coding and Agent Performance
The model shows significant advantages in specific coding benchmarks. For example, it scores 65.7% on a terminal coding test and 51.7% on a long-horizon marathon test, surpassing some leading closed-source models in the industry. This indicates strong competitiveness in complex logical reasoning, code generation, and multi-step tool calling, making the model well suited for building intelligent coding tools or automated code review Agents.
Gaps in Science and Deep Reasoning
However, the model does not lead across all categories. Its performance is relatively weak on scientific reasoning tests and some high-difficulty logic tests, with scores far below leading closed-source models of the same tier. Across dozens of benchmarks, the model is strong on coding and Agent tasks but still needs improvement in scientific knowledge and some deep reasoning areas. This also suggests that enterprise users must evaluate models against their own business scenarios—such as code generation versus knowledge Q&A—rather than relying blindly on parameter scale.
License: Open Weights with Commercial Boundaries
One of the most notable aspects of this release is its license. The model does not use standard open-source licenses such as MIT or Apache 2.0. Instead, it carries a custom license marked as “other” on the platform. Although its base terms are similar to MIT and allow free use, modification, distribution, and commercial use, it adds two restrictions for large-scale businesses:
- Revenue limit: Companies that offer model-as-a-service and have cumulative revenue exceeding $20 million over 12 months must sign a separate agreement with the research team before commercial use.
- Traffic limit: Commercial products with more than 100 million monthly active users also require separate authorization.
This strategy of “open weights rather than true open source” is becoming common among leading model vendors. It allows vendors to benefit from community engagement and brand exposure brought by the open-source ecosystem while protecting their commercial interests through business thresholds. For enterprises planning to integrate the model into core business operations, it is essential to review their own business scale and revenue expectations, conduct legal and compliance assessments in advance, and avoid potential infringement risks.
Local Deployment and Compute Requirements
With 1.56TB of weight files and 2.8T parameters, the model has an extremely high barrier to local deployment.
Hardware and GPU Memory
Official guidance states that running the model locally requires at least eight high-end GPUs. Assuming 80GB of memory per GPU, an eight-GPU system provides 640GB total GPU memory. Considering weight loading—which requires hundreds of GB even under MXFP4—KV cache, and system runtime overhead, an eight-GPU configuration only barely crosses the threshold. For longer context or higher-concurrency inference, the compute cluster must scale further.
Model Loading and Distribution
The weights are split into nearly 100 shards. The total 1.56TB data volume poses challenges for network bandwidth and storage systems during download and distribution. During local cluster deployment, the distributed file system must efficiently read these shards; otherwise, model loading time can become excessive and severely affect service cold-start efficiency.
Enterprise Cloud Deployment Recommendations
For most developers and small and medium-sized enterprises without large compute clusters, running such a model in a self-built data center is unrealistic. A more practical approach is to rely on public cloud infrastructure. For example, teams can use Baidu AI Cloud’s heterogeneous computing platform and container services to dynamically schedule GPU resources based on business traffic; use cloud object storage to support fast distribution and mounting of very large weight files; and use cloud monitoring and alerting to track GPU memory utilization and inference latency in real time. Through cloud-native architecture design, enterprises can access cutting-edge open-weight model capabilities while minimizing hardware procurement and operations and maintenance costs.
Conclusion
The arrival of this 2.8T-parameter open-weight model marks an important step for the open-source model community in terms of model scale and multimodal Agent capabilities. However, its special license terms, single quantized version, and high local deployment cost also create practical obstacles for developers. Before adopting such a model, enterprises should comprehensively evaluate technical architecture fit, business scenario requirements, legal and compliance risks, and compute costs, and rationally choose between self-hosting and cloud deployment to maximize technical benefits and business value.
