2.8T-Parameter MoE Model Released as Open Weights: Quantization, Memory, and the 8-GPU Deployment Threshold
A 2.8T-parameter mixture-of-experts model has been released with open weights on a public model hosting platform. The model activates about 104B parameters per token, uses 896 routed experts plus shared experts, and adds sparse attention in some layers. Its weights are stored in MXFP4, totaling about 1.56 TB across 112 files, while activations use MXFP8. This article examines what the release means for developers and technical decision-makers: model structure, quantization constraints, licensing considerations, vendor-reported benchmark results, and practical local or cloud deployment requirements, including why an 8-GPU setup is the starting point.
On the evening of November 4, 2025, an AI team made good on an earlier commitment by publishing the complete weights of a new flagship model on a mainstream open-model hosting platform. The files can be downloaded without an application process. According to the public information provided, this is the largest open-weight model released to date.
- The model has 2.8T total parameters and uses a mixture-of-experts (MoE) architecture.
- The weights are stored in a 4-bit floating-point format. Across 112 files, they total approximately 1.56 TB.
- The vendor states that local deployment requires at least eight accelerators.
For developers and technical decision-makers, “largest” is only the surface story. The more important questions are: what is actually inside the weights, what the license permits, and whether existing hardware can handle the model. This article analyzes those three questions. All performance figures are vendor-reported and have not been independently reproduced; they should be treated as reference points only.
1. Release Overview: A Promise Delivered, but “Open” Has Conditions
From online service to open weights
- October 24: The model first became available through an API and a web interface. At that time, the team promised to release the complete weights before November 4.
- November 2: The model repository was created on a community platform, but it remained closed on the day it appeared.
- November 4: The repository was still inaccessible that morning, but it officially opened that evening. It contains 112 files in total, including 96 weight shards, adding up to about 1.56 TB.
What has been released is the weight files themselves. The model card, configuration files, and inference code should be verified against the actual contents of the repository.
What “open” means here
In industry usage, “open weights” usually means that the trained parameter files are made publicly available, allowing users to download them and run the model in their own environments. It does not mean that everything is open:
- Training data, data mixtures, and complete training code are usually not released.
- License terms may place conditions on commercial use, redistribution, derivative model naming, and other usage scenarios.
Before downloading a large model for deployment, reading the license section of the model card should be the first step in any production-oriented evaluation. This point is discussed further in Section 5.
2. Structure: How the 2.8T Parameters Are Assembled
2.8T total parameters, but only about 104B activated per token
According to the model card, this is a native multimodal agent model continued from the previous generation. Its key structural facts are:
- Total parameters: 2.8T. This includes a large number of inactive parameters, such as expert parameters that are not selected for the current token.
- Activated parameters per token: about 104B, less than 4% of the total.
The value of MoE lies in this separation of model capacity and computation. Model capacity grows with the number of experts, while the compute cost for each token depends only on the experts that are activated.
896 experts, with each token visiting only 18
The model has 896 routed experts. For each token, 16 routed experts are selected, and 2 shared experts always participate in computation:
- Expert scale: About 2.8T parameters are distributed across 896 routed experts, meaning each routed expert has roughly 3B parameters on average.
- Activation ratio: Only 16 out of 896 routed experts are activated for each token, or about 1.8%. Including the 2 shared experts, about 18 experts participate in computation for each token in practice.
The router is also part of the model. Its gating network must be optimized jointly with the experts during training. Expert load balancing is therefore a common challenge in both training and inference for this type of model.
Compared with the previous generation, only one major change
Compared with the previous generation, the main structural change is limited to one area: 24 layers that originally used full attention have been changed to sparse attention. With 93 layers in total, roughly one quarter of the layers now use sparse attention.
In standard attention, each token must attend to all historical tokens pairwise. The computational cost grows quadratically with context length, and the KV cache grows linearly with sequence length. In contexts reaching the million-token level, attention over long sequences can become a clear bottleneck. Sparse attention reduces this cost by limiting the set of tokens that each token can “see.”
However, sparsity means that each token no longer interacts with all historical tokens. Preserving long-distance dependencies is therefore a core issue that must be validated for this type of design. Common evaluation methods include:
- Needle-in-a-haystack information retrieval;
- Long-document reasoning across multiple documents;
- Cross-file understanding in very large codebases.
Native multimodal: vision encoder and agent capabilities
The vision component uses a vision encoder developed by the team itself, giving the model image-understanding capability. For agent applications, this means the model can:
- Read screenshots of user interfaces;
- Parse charts and diagrams;
- Combine document images with other inputs to complete tasks.
However, images must first be encoded into visual tokens before being passed to the language model. High-resolution images can significantly increase context usage, so resource planning for long multimodal tasks must account for this additional cost.
3. Weight Precision: MXFP4 and a Single Quantized Release
MXFP4 weights and MXFP8 activations
The model uses MXFP4 for weights and MXFP8 for activations. Both were produced through quantization-aware training.
| Format | Bit width | Applied to | Characteristics |
|---|---|---|---|
| MXFP4 | 4-bit | Model weights | Uses block-wise scaling factors; balances storage and bandwidth requirements |
| MXFP8 | 8-bit | Runtime activations | Higher precision than weights; helps control quantization error propagation |
| BF16 | 16-bit | Standard full-precision weights | Wider numerical range; roughly 4x the size of MXFP4 |
| FP8 | 8-bit | Common training/inference precision | Roughly 2x the size of MXFP4 |
Quantization-aware training allows the model to adapt to low-precision representations during training, which is usually more effective than post-training quantization at preserving accuracy. Four-bit weights also directly reduce memory usage and memory-bandwidth pressure.
Only one quantized version is officially provided
The official release contains only this quantized version. No full-precision or 8-bit alternative is provided. For users, this has direct implications:
- No easy conversion back to full precision: Low-precision weights have already lost precision information and cannot be restored to a true full-precision version.
- Converting to full precision is impractical: If the weights are manually dequantized into a 16-bit format, their size would expand to roughly 6 TB, making local deployment even less feasible.
- Runtime support matters: Whether the bandwidth benefits of 4-bit quantization can be fully realized depends on the level of support for this format in the inference framework and hardware. If support is insufficient, the runtime may fall back to “dequantize first, then compute,” greatly reducing the bandwidth advantage.
4. Benchmarks: How to Read Vendor-Reported Results
The official comparison table reports the following coding and agent-related results:
| Test | This model | Commercial flagship model A | Commercial model B |
|---|---|---|---|
| Terminal task benchmark | 65.7% | 66.4% | 58.2% |
| Long-horizon software engineering task | 51.7% | 46.3% | — |
| Terminal science task | 38.1% | 63.3% | — |
| High-difficulty general reasoning test | 39.4% | 60.9% | — |
The vendor claims that the model leads model A in 16 of 37 benchmarks. A third-party independent evaluation gave it an intelligence index of 55, ranking fourth globally. Putting these numbers together leads to three observations.
First, the strengths are concentrated in coding and agent tasks. The terminal task result is roughly on par with model A, while the long-horizon software engineering task is 5.4 percentage points higher. This aligns with the model’s positioning as an agent-oriented system.
Second, the weaknesses are equally clear. The model trails by 25.2 percentage points on the terminal science task and by 21.5 percentage points on the high-difficulty general reasoning test. These are substantial gaps in the table. If a business is centered on scientific computing, complex reasoning, or knowledge-intensive question answering, coding performance alone is not enough to justify model selection.
Third, vendor-reported results still need independent validation. There are at least three information gaps here:
- Vendors usually choose benchmarks and prompting methods that are favorable to their own models.
- The performance of coding and agent tools depends heavily on tool calling, execution sandboxes, and prompt engineering. The same model may perform very differently under different agent frameworks.
- Public benchmarks also carry the risk of contamination from training data.
It is advisable to run blind tests using a private evaluation set that matches your own business scenarios. In addition to accuracy, teams should also record first-token latency, cost per request, and success rate for long-running tasks.
5. Licensing and Compliance: Two Prerequisites Beyond Open Weights
Open weights do not mean unrestricted use. The larger a model is, the more likely its license is to define usage boundaries. When evaluating such a model, the following points should be checked carefully:
- Commercial scope: Whether commercial use is allowed, and whether there are trigger conditions such as daily active users or revenue thresholds.
- Redistribution: Whether the weights can be packaged into your own products or distributed as part of images.
- Derivative models: How fine-tuned or continued-trained models should be named and labeled.
- Usage restrictions: Whether applications in specific domains are prohibited.
- Compliance responsibility: What responsibilities the user bears regarding content safety, data privacy, intellectual property, and other compliance areas.
For individual developers, these terms may have limited practical impact. For enterprises, they directly determine whether the model can enter production. It is best to complete license review with legal and security teams during the project initiation phase and retain records of that review.
6. Local Deployment: Why Eight GPUs Is the Starting Point
Where the memory requirement comes from: a simple estimation formula
The memory requirement for local deployment of a large model can be broken down into three parts:
Total memory ≈ weight usage + KV cache + runtime overheadWeight usage ≈ 1.56 TB (4-bit quantized weights)KV cache ≈ KV size per token × context length × concurrent requestsRuntime overhead ≈ activation buffers + communication buffers + framework overhead(usually about 5%–15% of total memory)
For example, with an accelerator card that has roughly 180 GB of memory per card:
Weights per GPU = 1.56 TB ÷ 8 ≈ 195 GB
The weights alone already exceed the capacity of a single card. This means that the vendor’s “eight cards” most likely refers to a single-node, eight-GPU configuration. For cross-node deployment, multi-machine network bandwidth can become the bottleneck.
KV cache: the real cost of long context
Weights are only the entry ticket. With 93 layers, even if grouped-query attention is used and the KV cache is compressed with 8-bit or 4-bit precision, the KV cache for a single request with million-token context can still be very large. In real production environments, it is usually difficult to allow one request to consume the full million-token context. Common strategies include:
- Setting a reasonable maximum context length based on business needs;
- Enabling paged KV cache management to reduce memory fragmentation;
- Enabling prefix caching reuse for high-frequency prompts;
- Using continuous batching to improve concurrent throughput.
MoE compute profile: compute is affordable, but memory reads are heavy
With about 104B activated parameters, the forward computation per token is roughly 2 × 104B FLOPs, or about 0.2 TFLOPs. A single GPU typically delivers hundreds of TFLOPs to PFLOPs of compute, so compute is not the main bottleneck. Memory bandwidth is.
During decoding, every generated token requires reading the relevant expert weights from memory. Performance is therefore constrained by how many bytes each GPU can read per second.
The following estimation script can be used as a rough reference. Actual deployment should be tuned with load testing:
WEIGHT_BYTES = 1.56e12 # Total 4-bit quantized weight sizeTP = 8 # Tensor parallelism degree, i.e., number of GPUsN_LAYERS = 93def estimate(weight_bytes=WEIGHT_BYTES, tp=TP, layers=N_LAYERS,bytes_per_token=512 << 20, ctx_len=32768, concurrency=8,overhead=0.1):weight = weight_bytes / tp # Weight per GPUkv = bytes_per_token / layers * ctx_len * concurrency / tpreturn {"weight_gb": weight / 1e9,"kv_gb": kv / 1e9,"total_gb": (weight + kv) * (1 + overhead) / 1e9,"per_gpu_gb": (weight + kv) * (1 + overhead) / tp / 1e9}if __name__ == "__main__":print(estimate())
Parallel strategy: tensor parallelism versus expert parallelism
Deploying an MoE model usually requires balancing two parallelism strategies:
| Dimension | Tensor parallelism | Expert parallelism |
|---|---|---|
| Split target | Weight matrices within each expert | Individual experts |
| Communication pattern | Multiple collective communications per layer | Mainly All-to-All communication for token dispatch and collection |
| Load balance | Naturally balanced | May be skewed; if hot experts concentrate on certain GPUs, those GPUs can become bottlenecks |
| Typical use case | High-speed interconnect within a node | Many experts; optimizing memory efficiency |
With 896 experts, there is ample room for expert parallelism. In practice, a common approach is to mix tensor parallelism and expert parallelism inside a node, while keeping high-communication operations within the node as much as possible when crossing nodes. Shared experts need to be present on every GPU; otherwise, additional communication may be introduced.
From storage to serving: an engineering workflow
- Download: Weight shards over hundreds of gigabytes should support resumable downloads and hash verification per shard, avoiding the situation where corruption is discovered only after the full download is complete.
- Distribution: When multiple replicas need to be loaded, it is better to synchronize the weights to internal object storage or file storage first, then let compute nodes pull them locally. This reduces pressure on public network egress.
- Loading: Large weight files should be placed on local high-speed storage, with parallel shard loading enabled. For repeated restarts, memory-mapped files or resident mappings can help reduce cold-start time.
- Serving: The inference engine should be containerized and deployed on a container platform or GPU node pool, with access through a unified API gateway.
- Observability: Logging and monitoring systems should track memory usage, batch size, first-token latency, throughput, and request error rates.
Containerization, combined with orchestration and scheduling, is a key step in moving an ultra-large model from “it can run” to “it can be operated.”
7. Cloud Deployment: You Do Not Need to Build Your Own Eight-GPU Cluster
The hidden costs of local eight-GPU deployment go beyond the GPUs themselves. They also include multi-GPU servers, high-speed interconnects, power, cooling, and ongoing work on drivers, inference frameworks, and quantization kernel adaptation.
For long-term, stable inference workloads, self-built resources often have a better unit cost. However, if a workload is bursty, validation cycles are short, or the team lacks dedicated operations staff, using on-demand or elastic GPU instances on cloud platforms such as Baidu AI Cloud may be more suitable:
- Elasticity: Rent the number of GPUs needed for peak demand, then release them after the peak passes.
- Fast validation: Complete private evaluation early, then decide whether a long-term investment is justified.
- Managed operations: Image distribution, scaling, and monitoring can be handled by platform capabilities.
- Elastic storage: Large model weights can be stored in object storage and loaded on demand.
- Parallel workloads: Distributed training or batch inference jobs can benefit from parallel file storage for stable throughput.
Conclusion: Look at the Task First, Then the Parameter Count
This weight release is clearly symbolic: a multimodal agent model with more than a trillion parameters is now available to developers in downloadable form for the first time. But assessing this “gift” rationally requires answering three questions:
- Structure: 2.8T total parameters, about 104B activated parameters, 896 experts, and sparse attention define its cost model. The compute may be affordable, but keeping memory bandwidth fed is the real challenge.
- Precision: 4-bit quantization makes the model deployable in size, but it also makes the runtime environment a hard constraint.
- Benchmarks: Coding and agent capabilities are its strengths, while science and complex reasoning are clear weaknesses.
The final decision should depend on the business task and private evaluation results, not on parameter count or leaderboard rankings. For most enterprises, a more pragmatic path may be “validate in the cloud on demand, process private data locally, and make long-term plans for stable workloads,” striking a balance between capability and cost.
