2.8T-Parameter MoE Open Weights: Deployment and Licensing Explained
This article examines a 2.8T-parameter MoE model released as open weights, covering its sparse architecture, low-precision format, benchmark interpretation, custom license restrictions, and practical deployment requirements.
1. What was released, and why it matters
When a Mixture-of-Experts (MoE) model with 2.8T parameters releases full open weights, totaling roughly 1.56TB, with an 8-GPU minimum deployment footprint, the practical question for developers and enterprises is no longer just “Can I download it?”
There are three more concrete questions:
- What is structurally special about the model?
- What does the license prohibit?
- Can it run on your hardware, and is it economically viable?
This article does not focus on any specific product. Instead, it uses this class of “trillion-scale open-weight models” as a case study and breaks the issue down layer by layer.
1.1 Timeline: a weight release that delivered on a promise
The release sequence followed a familiar pattern for this type of model:
- In late October, the model usually appears first as an online API and web interface, while promising to release full weights within about ten days.
- On November 4, the weights appeared in a public model hosting repository and were downloadable without application or approval.
- The repository had been created a few days earlier, remained closed that morning, and was formally opened that evening.
- The release contained 112 files and 96 shards, totaling about 1.56TB. In some regions, the files could also be synced from mirror communities.
For developers, three points matter most:
- Complete weights: This is not a distilled version, a quantization preview, or a limited “open API” release. It is the full model that can be placed on your own storage.
- Immediate availability: There is no application process and no queue. Download access is effectively zero-friction.
- Unprecedented scale: This is the largest open-weight model publicly released so far, substantially larger than earlier comparable models.
One caveat is important: figures such as 2.8T parameters, 1.56TB size, and 8-GPU minimum deployment come from public information. Performance results come from self-reported testing by the model author. This article treats them as inputs to be verified, not as final conclusions.
2. Model architecture: how 2.8T parameters are calculated
2.1 Total parameters versus activated parameters: a typical sparse MoE
According to the public model card, this is a native multimodal agent model continued from the previous generation. Its key structural details are:
- Total parameters: 2.8T
- Activated parameters per token: 104B, about 3.7% of total parameters
- Experts: 896
- Experts selected per token: 16
- Always-active shared experts: 2
- Network depth: 93 layers
- Context window: 1 million tokens
- Additional component: a visual encoder for image input
This design follows a clear evolution path in large models:
- Dense models: every token passes through all parameters. Capability growth and inference cost rise almost in lockstep.
- MoE models: total parameters determine knowledge capacity, while activated parameters determine compute per token. These two factors are decoupled.
Many models follow this idea. The differences are mainly in scale, expert granularity, and shared-expert design.
2.2 How many experts are used in one forward pass?
Assuming each token is routed to 16 experts:
- 16 ÷ 896 ≈ 1.79%
Including the 2 shared experts, a single forward pass involves only 18 experts out of 898:
- 18 ÷ 898 ≈ 2%
This does not mean the other 98% of parameters are “stored but unused” during inference. To represent a complete knowledge space, all expert weights usually need to reside in GPU memory.
This is the origin of the rule:
- Total parameters determine memory footprint
- Activated parameters determine compute demand
That distinction is the basis for the memory estimation in the next section.
2.3 The main change from the previous generation: attention structure
Among the 93 layers, 24 layers replace full attention with sparse attention, referred to as DSA.
- Full attention layers are responsible for global information integration.
- Sparse attention layers allow each token to focus only on key positions over a longer range, reducing computation as context length grows.
This is a “mixed stacking” design: it preserves global modeling capability while controlling long-context cost.
For agent-oriented models that often need to process hundreds of thousands of characters of context in a single request, this design directly affects latency and cost per request.
2.4 What it means that only one low-precision version was released
The official release only provides weights in MXFP4 and activations in MXFP8. No BF16 or FP8 version is provided.
There are two reasons behind this:
- MX formats are microscaling low-precision data formats. Multiple elements share one scaling factor, usually grouped by 32 elements.
- MXFP4 means each parameter uses 4 bits.
- MXFP8 means each parameter uses 8 bits.
- These weights are the result of Quantization-Aware Training (QAT). The model adapts to low-precision representation during training rather than being compressed after training.
This makes storage estimation straightforward:
Storage size ≈ 2.8T × 4 bit ÷ 8 ≈ 1.4 TB
Adding scaling factors, embedding layers, and other auxiliary tensors brings the total to about 1.56TB, which matches the download size.
There is also a practical reason for not releasing a BF16 version: if MXFP4 weights are dequantized back to BF16 for inference, the model still has only 4-bit effective precision, while the file size expands to roughly four times the original. Almost no one would use it that way.
The tradeoff is real:
- Low precision reduces storage and bandwidth pressure.
- But it shifts the burden to the inference engine and hardware, because not all inference frameworks and accelerators can natively handle these formats efficiently.
Before evaluating hardware, you should confirm whether your inference stack supports the format.
3. Capability boundaries: how to read vendor-reported benchmark scores
According to the model author’s published comparison data, which is self-reported, the model shows a clear imbalance:
- Relatively strong coding and agent benchmarks
- Terminal-Bench 4.0: 65.7%, slightly below 66.4% from a leading proprietary model, but above 58.2% from another proprietary model.
- SWE Marathon: 51.7%, clearly higher than the former’s 46.3%.
- Clearly weaker science and reasoning benchmarks
- Terminal-Bench-Science: 38.1%, versus 63.3% for the former.
- HLE-Diamond: 39.4%, versus 60.9% for the former.
- Mixed overall picture
- The author claims the model outperforms the comparison model on 16 out of 37 benchmarks.
- A third-party evaluation assigned it a composite intelligence score of 55 and ranked it near the top.
3.1 The capability profile revealed by the data
Overall, this can be understood as a model that is:
- Strong in coding and agentic tasks
- Relatively weak in scientific reasoning and deep reasoning
This aligns with its positioning. A model designed around agent tasks tends to invest more training resources in:
- Tool calling
- Multi-step planning
- Code and terminal operations
The tradeoff is weaker performance on tasks requiring rigorous scientific derivation and deep hierarchical reasoning.
3.2 Why these scores should be discounted
At least four factors matter when interpreting these results.
First, who ran the evaluation.
Self-reported data usually selects prompts, sampling parameters, and evaluation configurations most favorable to the model. These results may differ from independent third-party re-testing.
Second, the benchmark itself.
Many benchmark questions have already appeared widely in public corpora, so data contamination is possible. Changing prompt format alone can cause noticeable score fluctuations. A one-point difference is often not enough to prove a real capability gap. For example, it is hard to claim a clear winner between 65.7% and 66.4%.
Third, index methodology.
Composite intelligence scores are usually aggregated from multiple institutions’ model data. Weight settings differ from independent evaluations, so they are better used as a reference coordinate rather than as a final selection criterion.
Fourth, the advantage of open weights.
Unlike closed models that only expose an API, open weights allow anyone to reproduce evaluations on their own task sets. This is one of their most practical values.
3.3 A practical recommendation
Instead of staring at leaderboards, build your own evaluation set.
Extract 50 to 200 representative tasks from real business scenarios, covering:
- Code generation and repair
- Long-document understanding
- Tool calling
Then compare candidate models using the same prompts and scoring criteria. The conclusions will be much closer to your actual needs than any single leaderboard number.
4. License: open weights are not the same as open source
This is one of the most important parts of the release to examine separately.
The license type listed in the hosting repository is usually classified as Other/custom, not as MIT, Apache-2.0, or another standard open-source license. The repository carries a custom model license, and compared with the previous generation, the terms are almost identical word for word except for the name.
4.1 The main grant is MIT-style, but with additional restrictions
The core license text is basically consistent with an MIT-style grant. It allows free use, copying, modification, redistribution, fine-tuning, and commercial use.
However, two additional restrictions are attached, and they mainly apply only to large-scale external commercial operations:
Model-as-a-Service (MaaS) and revenue threshold
Companies that provide the model as a service externally and have cumulative revenue exceeding USD 20 million within a 12-month period must sign a separate agreement with the model publisher before commercial use.Monthly active user threshold
Commercial products whose monthly active users exceed a specified scale must meet additional conditions. The exact threshold should be confirmed in the license text.
4.2 Who is affected by these restrictions?
| Use case | Affected? | Notes |
|---|---|---|
| Personal study, academic research | Basically not affected | Additional terms mainly target external commercial services |
| Internal enterprise private deployment | Generally not affected | This is not “offering services externally,” but the full text still needs review |
| Small and medium SaaS products | Generally not affected | Revenue and MAU thresholds are usually not reached |
| Large-scale model services, large commercial products | Separate negotiation required | A separate agreement must be signed before commercial use |
4.3 What is the difference between “open weights” and “open source”?
Open weights mean you can obtain the full parameters and perform inference, fine-tuning, distillation, and deployment yourself.
Open source usually also implies that the license meets the open-source definition and does not impose field-of-use or scale restrictions.
The additional terms mean this license is conditional regarding “commercial freedom.” For most teams, the impact is limited. But for enterprises planning to commercialize the model at scale, this is a legal boundary that must be clarified during project initiation.
4.4 Three practical compliance suggestions
Review line by line and archive a version snapshot
Licenses may change with model versions. When downloading weights, also archive the license text and its version number.Bring in legal review early
If external service delivery is involved or product growth is fast, legal review should happen early to avoid being caught by scale-related terms after product launch.Pay attention to matters outside the license
Beyond the license itself, enterprises must also consider data compliance, generated-content safety, and supply-chain traceability. These issues are unrelated to which model you choose, but they are mandatory for enterprise deployment.
5. Can it run locally? The 8-GPU memory and bandwidth math
This is the question ordinary developers care about most.
The direct conclusion is: 8 GPUs are only the starting point, not the comfort zone.
- “2.8T parameters” refers to total parameters.
- “1.56TB” refers to disk footprint.
- “8 GPUs” refers to the minimum deployment scale.
These three numbers do not measure the same thing.
5.1 How is GPU memory allocated?
The memory budget can be split into four parts:
- Weights: about 1.4TB. Across 8 GPUs, that is roughly 175GB per GPU. The full download, including scaling factors and auxiliary tensors, is about 1.56TB, or roughly 195GB per GPU.
- KV cache: proportional to context length and batch size.
- Activations and temporary buffers: related to batch size and sequence length.
- Framework overhead: communication buffers, memory fragmentation, framework runtime usage, etc. It is usually necessary to reserve more than 10%.
The per-GPU memory requirement can be written as:
Per-GPU memory ≈ (weights + KV cache + activations and temporary buffers) ÷ number of GPUs × (1 + overhead ratio)
Using a typical 80GB accelerator as an example:
1.56 TB ÷ 8 GPUs ÷ 80 GB ≈ 2.4×
This does not yet include KV cache or framework overhead.
Even if per-GPU memory increases to 141GB, 8 GPUs provide about 1.1TB in total, which still cannot hold 1.56TB of weights.
Therefore, “8-GPU minimum deployment” actually corresponds to several situations:
First, accelerators with about 192GB per GPU
Eight GPUs provide about 1.5TB in total, barely enough to hold the weights, leaving very little room for KV cache.
Second, conventional 80GB-class accelerators
You need at least 16 GPUs, or you must offload part of the weights to host memory and local storage. Offloading introduces significant latency and is mainly suitable for offline batch processing.
5.2 During decoding, bandwidth is the real bottleneck, not compute
When an MoE model generates tokens, each token activates only about 104B parameters. Using a simplified MXFP8-style approximation of about 1 byte per activated parameter, reading the activated expert weights requires roughly 100GB-class bandwidth per token.
The theoretical lower bound for generating a single token is approximately:
Time per token ≈ activated parameters × bytes per parameter ÷ aggregate GPU memory bandwidth≈ 104B × 1 byte ÷ 8 TB/s ≈ 13 ms→ theoretical upper bound ≈ 75 tokens/s
This estimate assumes an aggregate bandwidth of 8TB/s. In practice, you must also account for:
- Cross-GPU communication introduced by expert parallelism
- Inference-engine scheduling overhead
- Sampling and batching overhead
Real throughput is usually significantly lower than the theoretical upper bound.
This explains a common confusion:
- “Can run” and “runs fast” are two different things.
If the weights fit in memory, inference can start. But whether latency requirements are met depends on bandwidth, parallelism strategy, and batching policy.
5.3 What if you can only assemble eight 80GB GPUs?
Weight offloading is a feasible path.
Inactive expert weights can be placed in host memory or even local NVMe storage, and swapped into GPU memory on demand.
The idea can be described with pseudocode:
# Weight offloading: only swap an expert's weights into GPU memory when it is routeddef moe_forward_with_offload(hidden, router, expert_store, gpu_cache, k=16):scores = router(hidden) # compute routing scores for each tokentopk = scores.topk(k, dim=-1).indices # each token selects only 16 expertsexpert_ids = topk.unique() # deduplicate experts actually used in this batchfor eid in expert_ids:if eid not in gpu_cache:gpu_cache[eid] = expert_store.fetch_to_gpu(eid) # host memory/NVMe -> GPU memoryoutputs = [gpu_cache[eid](hidden) for eid in topk] # compute only selected expertsreturn weighted_sum(outputs, scores) # weighted sum by routing scores
This path works, but the cost is frequent data movement. PCIe bandwidth is usually only a small fraction of GPU memory bandwidth, often tens of times lower. If too many experts need to be swapped in, throughput can drop sharply.
Therefore, this approach is more suitable for:
- Low-frequency invocation
- Offline batch processing
It is not suitable for online services facing large numbers of users.
5.4 Quick feasibility table
| Scenario | Hardware configuration | Feasibility | Suitable use |
|---|---|---|---|
| Single node, 8 high-memory GPUs | 8 × about 192GB with high-speed GPU interconnect | Feasible | Online inference, agent services |
| Multi-node, 16+ GPUs | 16 × 80GB or more with high-speed network | Feasible | High-concurrency online inference |
| Single node, 8 × 80GB + weight offloading | 8 × 80GB plus large host memory/NVMe | Starts but slow | Offline evaluation, low-frequency batch processing |
In addition, MoE inference requires distributing different experts across different GPUs and exchanging tokens at each computation step. This places high demands on GPU-to-GPU and node-to-node bandwidth.
- Single-node multi-GPU deployment should prioritize high-speed GPU interconnect.
- Multi-node deployment relies on high-speed networking such as RDMA.
Otherwise, communication becomes the new bottleneck.
6. Using cloud capabilities to reduce deployment friction
For most enterprises, building and maintaining such an inference cluster internally over the long term is not economical.
A more practical approach is to use generic cloud capabilities in stages:
- Object storage: store 1.56TB-class weights and model versions as distribution sources or backups.
- Container platform: package inference services and scale elastically according to load.
- Load balancing and API gateway: provide unified ingress, rate limiting, authentication, and multi-version canary release.
- Logging, monitoring, and alerting: track first-token latency, throughput, GPU memory utilization, and other key metrics to detect anomalies quickly.
- Elastic GPU compute: turn fixed hardware procurement into on-demand resource usage.
Major cloud platforms, including Baidu AI Cloud, generally provide these generic capabilities. They can help teams move from proof-of-concept validation to limited production without building a dedicated data center.
One important reminder: the cost of large-scale model services mainly comes from GPU compute. Storage, networking, and other supporting costs are relatively manageable.
Cost estimation should focus on:
- Cost per million tokens
- GPU count required at peak concurrency
Not just how large the weight files are.
7. Deployment checklist
| Dimension | Check item | Key criterion |
|---|---|---|
| License | Whether commercial scale triggers additional terms | Monthly revenue or MAU approaching thresholds; legal review completed |
| Hardware | Per-GPU memory, GPU interconnect, node-to-node bandwidth | Whether weights fit in memory; whether communication becomes a bottleneck |
| Inference engine | Support for MXFP4/MXFP8 and expert parallelism | Whether it can efficiently use low-precision hardware capability |
| Performance | First-token latency, decode throughput, long-context behavior | Whether it meets business SLA requirements |
| Quality | Performance on a self-built evaluation set | Comparison results against candidate models |
8. Conclusion: selection should return to three questions
When a 2.8T-parameter, 1.56TB open-weight model appears, “open” is no longer a simple black-and-white label. It becomes a collection that must be reviewed item by item:
- Architecture: sparse MoE pushes the logic of “total parameters determine memory, activated parameters determine compute” to the extreme, while low-precision quantization shifts the barrier to hardware and inference engines.
- License: “open weights” does not mean “unconditional commercial use.” Scale-related terms in the custom license must be included in compliance evaluation early.
- Deployment: “8 GPUs” is only the minimum entry ticket. Real cost depends on the combination of memory, bandwidth, and interconnect.
For developers and enterprises, the most important thing is not chasing the highest benchmark score. The three questions that matter are:
- What capabilities does the business actually need?
- What does the license allow?
- What scale can the existing infrastructure support?
Answer these three questions clearly, and trillion-scale open-weight models can become real productivity, rather than just a 1.56TB file sitting on a disk.
