2.8T-Parameter Model Weights Released: What the 1.56 TB Open-Weight Package Means
A vendor has released the full weights of a 2.8T-parameter mixture-of-experts multimodal model as a 1.56 TB package split across 96 shards. This article examines the model architecture, 4-bit floating-point quantization, the practical limits of its 1M-token context window, vendor-reported benchmark results, custom license terms, and what it really takes to serve the model locally on eight GPUs.
1. What happened: a large weight release that followed a promise
In late October, a vendor launched a new flagship model through an API and a web interface. At the same time, it promised to release the full model weights within two weeks.
On November 4, the weights appeared in a repository on a public model hosting platform and could be downloaded without an application process. According to the repository record, the repository had been created two days earlier and was still inaccessible during the day; it was opened that night.
The weights are distributed as shards. There are 96 shards in total, with a combined size of about 1.56 TB. This is one of the largest publicly released model weight sets so far, and it is considerably larger than previous open-weight models.
Once the weights were available, developers had three main questions:
- What is inside the model package?
- Does the license allow commercial use?
- Can the model be run locally?
This article focuses on those three questions. All performance figures cited in this article come from the vendor’s own reports and are identified as such.
2. Inside the weight package: how 2.8T parameters are organized
2.1 MoE structure: separating total capacity from compute
The model is a native multimodal model designed for agent tasks. It was trained further from the previous generation. According to the model card, the core specifications are:
| Item | Specification | Meaning |
|---|---|---|
| Total parameters | 2.8T | Source of the full weight size; determines memory and VRAM requirements |
| Activated parameters per token | About 104B | The portion actually used when generating one token; determines the lower bound of speed |
| Expert structure | 896 routed experts + 2 shared experts | Each token selects 16 routed experts |
| Layers | 93 | — |
| Context length | 1 million tokens | Theoretical maximum; practical deployment requires tradeoffs |
| Vision encoding | Built-in vision encoder | Native multimodal input with mixed image and text |
| Attention | Some layers use full attention; some use sparse attention | Reduces attention compute cost under long context |
This is a typical mixture-of-experts, or MoE, architecture.
During decoding, each token activates only about 1.8% of all experts (16/896) and computes only about 3.7% of total parameters (104B/2.8T).
From this, three engineering conclusions follow:
Capacity comes from total parameters; speed comes from activated parameters.
Whether the model fits depends on the 2.8T total parameters. The lower bound of per-token generation speed is mainly determined by reading the roughly 104B activated parameters.Batch throughput is usually better than single-user latency.
MoE models are suited to concurrent requests. Experts selected by different tokens in the same batch can be combined, improving hardware utilization. If the system serves only one interactive session, much of the hardware may sit idle.Routing can amplify hardware overhead.
Different tokens in the same batch may select very different expert sets. After being split across GPUs, this can create uneven load. Techniques such as expert parallelism and redundant experts are practical ways to mitigate this problem.
2.2 Quantization format: official release only provides 4-bit
The weights use 4-bit floating-point weights and 8-bit floating-point activations.
The weight values have lower numerical precision, but computation still keeps floating-point exponent bits. This gives a wider dynamic range than pure integer quantization and is suitable for large models with widely varying parameter distributions. The 8-bit activations are a compromise between compute accuracy and hardware efficiency.
This scheme is the result of quantization-aware training. Compared with post-training quantization, quantization-aware training simulates quantization error during training or continued training, giving the model a chance to compensate. Accuracy loss is usually smaller.
One easily overlooked fact is that the vendor did not release BF16 or FP8 versions.
A size calculation makes the reason clear:
Activated parameters at 4-bit are about 52 GB.
If all 2.8T parameters were stored in BF16, the package would need about 5.6 TB, more than three times the current size. At 4 bits, the theoretical lower bound is 2.8T × 0.5 byte ≈ 1.4 TB. The actual size is about 1.56 TB. The extra space roughly comes from routers, shared experts, normalization parameters, and other parts that may still be stored at higher precision, as well as shard metadata.A 208 GB reference point is also useful.
If the roughly 104B activated parameters were held in BF16, the high-precision container would be about 208 GB. This is the part a traditional BF16-style deployment would need to hold in GPU memory for the active path. It also highlights why 4-bit weights can significantly reduce the deployment footprint.
This also explains a commonly misunderstood question: why not release a BF16 version?
By estimate, a BF16 version would exceed 5 TB. That is impractical for download, storage, and distribution. What developers usually need is not original precision, but a model that fits into limited GPUs.
There is a cost. 4-bit weights can only be dequantized into higher-precision formats. They cannot be restored to the original precision, and the quantization scheme cannot be redesigned locally. If an inference framework does not support this floating-point 4-bit format, the weights must first be converted into a higher-precision container, and memory usage may grow significantly.
2.3 Long context: the compute cost behind 1M tokens
Compared with the previous generation, the architecture changed in one main area: full attention in some layers was replaced by sparse attention.
Attention compute grows quadratically with sequence length. In a million-token context, the KV cache can take more space than the model weights. Sparse attention computes attention only among selected positions, reducing the overhead of long context.
But 1M tokens is only a theoretical upper limit. It does not mean the model is immediately usable at that length.
In practice, the available context is determined by:
- GPU memory capacity minus the space used by weights;
- The latency budget for each request.
Engineering deployments usually start with 64K to 256K context and reserve 1M tokens for cases that genuinely require extremely long documents or large codebases.
3. How to read the benchmarks: vendor-reported results need context
The vendor published a set of comparison results. The following figures are vendor-reported. This article lists the model’s own scores and summarizes the comparisons:
| Benchmark type | Score | Relationship with reference model |
|---|---|---|
| Terminal tasks | 65.7% | Slightly below the reference model |
| Long-horizon software engineering tasks | 51.7% | Above the reference model |
| Scientific benchmarks | 38.1% | Clearly below the reference model |
| High-difficulty knowledge and reasoning | 39.4% | Clearly below the reference model |
The vendor also says that, among 37 benchmarks, the model outperformed the reference model on 16. It also cites a third-party evaluation that ranked the model fourth globally by a composite intelligence index.
Taken together, the table and the vendor’s statement lead to a clear conclusion: this is a specialized model. Its coding and agentic performance is close to, or partly above, the top tier. Its scientific and high-difficulty reasoning performance is clearly behind. A mixed record is a more accurate description.
Specific points for interpreting these numbers:
Distinguish exam scores from production scores.
Benchmarks measure average performance under specific task distributions. Enterprises care about success rate, rework rate, and end-to-end cost on their own tasks. Those need to be measured on internal evaluation sets.Pay attention to how scoring works.
Terminal tasks and software engineering benchmarks often use pass/fail tests. These are harder to game with surface-level tricks than multiple-choice questions and are more useful for evaluating agent behavior.Use the agentic form to compensate for single-step reasoning weaknesses.
Weaknesses in scientific and reasoning benchmarks can be partly offset at the product layer with retrieval augmentation, tool calls, and result verification. However, if a task depends heavily on rigorous single-step reasoning, this should be treated as a clear negative factor during model selection.Vendor-reported data needs independent reproduction.
Inference configuration affects results. Sampling parameters, context length, prompt templates, and whether tools are used can all change scores. Before production use, teams should run their own tests rather than directly trusting the vendor’s comparison table.
4. License: open weights are not the same as open-source software
The license is the most important point to discuss separately in this release.
The repository is attached with a custom license. On the model hosting platform, it is marked as license:other, not MIT, Apache, or another standard open-source license.
This custom license is word-for-word identical to the previous generation’s license, except for the name. Following convention, it constrains only the model weights and does not constrain inference code or business code written around the model.
The main grant is close to MIT-style permission: free use, copying, modification, distribution, fine-tuning, and commercial use. The threshold is not high.
However, custom licenses usually include additional clauses. Before production use, enterprises should check at least the following points. The full repository license text remains the final reference:
Attribution and notices:
When distributing or providing a service, must copyright notices be preserved, must a copy of the license be included, or must modified versions be marked in a specific way?Name use:
Can modified or fine-tuned models keep the original name? Can trademarks or model names be used in marketing?Downstream pass-through:
When redistributing or embedding the model in a commercial product, must the license also be provided to end users?Use restrictions:
Are there prohibited or restricted industries or use cases?Liability and warranty:
Does the license state that the model is provided “as is” and without warranties?
One point is often overlooked: because the terms are word-for-word identical to the previous generation, teams that have already completed a compliance review for the previous generation can reuse their conclusions without starting over.
It is also worth noting that the industry has stricter definitions of “open source.” For example, the Open Source Initiative released OSAID 1.0 in 2024. In addition to open weights, it requires disclosure of training data, code, training methods, and other information. By that standard, a weights-only release is an open-weight model, not fully open-source software. Regardless of which definition is used, the license is the first compliance gate.
5. Local deployment: why 8 GPUs is the starting point, not the comfortable point
“Local deployment requires at least 8 GPUs” is the most practical barrier of this release.
When the numbers are broken down, the key issue is not the number of GPUs. It is the combination of VRAM, host memory, and bandwidth.
1. VRAM: 8 GPUs are not enough to hold all weights.
If the 1.56 TB package is split evenly across 8 GPUs, each GPU would need to hold about 195 GB. That exceeds the memory capacity of current mainstream single GPUs.
If all weights must reside in VRAM, using 141 GB per GPU would require at least 12 GPUs.
Therefore, “8 GPUs to start” implies a premise: most weights are stored in host memory and paged into GPU memory as needed. The GPUs mainly handle computation.
2. Host memory and storage: the machine should start with at least 1.5 TB of memory.
Holding the remaining weights usually requires at least 1 TB of system memory, and 1.5 TB is a safer starting point.
The weight files themselves require about 1.6 TB of local storage. Local NVMe SSDs are recommended, with extra space reserved.
Adding the motherboard, CPU, and power supply, the total machine cost is not much lower than the cost of the 8 GPUs themselves.
3. Speed: decoding is bottlenecked by memory bandwidth.
Autoregressive decoding requires reading the activated expert weights for every generated token.
Below is an illustrative estimate. It explains the order of magnitude only; actual results depend on testing.
All weights resident in VRAM:
At 4 bits, each token needs to read at least about 52 GB of weights. Assuming 8 GPUs provide an aggregate bandwidth around 10 TB/s, and single-GPU memory bandwidth is typically above 3 TB/s, reading the weights alone takes about 5 ms per token. The theoretical upper bound is about 190 tokens/s. After adding communication overhead, sampling, and KV cache reads and writes, actual generation speed is usually only a fraction of this limit.Weights placed in host memory:
The bottleneck becomes host memory bandwidth. With memory bandwidth in the hundreds of GB/s range, reading the weights for one token takes about 0.1 to 0.5 seconds. In practice, measured speed is often only single-digit tokens/s, making interactive use difficult.
4. Power and deployment conditions.
A machine with 8 high-performance accelerator cards usually consumes several thousand watts. It needs stable mains power, datacenter-grade cooling, and noise isolation. It is not suitable for an ordinary office desk.
The following is an illustrative configuration. It is not the real syntax of any framework, and field names vary by framework:
# Example local-deployment outline for a very large MoE model.# Field names vary by framework.model_path: /models/moe-large # Weights are about 1.56 TB; local NVMe storage is recommendedquantization: fp4_weight_fp8_activate # 4-bit floating-point weights, with 8-bit activations during computationtensor_parallel_size: 8 # Tensor parallelism across 8 GPUsexpert_parallel_size: 8 # Expert parallelism; routed experts are distributed across GPUscpu_offload: true # Most expert weights stay in host memory and are paged in as neededhost_memory_gb: 1024 # 1 TB or more of host memory is recommendedmax_model_len: 65536 # Set context length according to need; do not start at the maximumkv_cache_dtype: fp8 # Lower-precision KV cache to save more GPU memorygpu_memory_utilization: 0.90
Overall, 8 GPUs is a minimum configuration that can get the model running. It is not a comfortable configuration.
This scale is better suited to a small number of shared nodes that serve multiple teams through an API, rather than local deployment for each individual user.
6. Self-hosting or using a service
For very large open-weight models, enterprises do not have to choose strictly between self-hosting and using a service. They can first evaluate three dimensions:
| Dimension | Lean toward self-hosting open weights | Lean toward using a model service |
|---|---|---|
| Data compliance | Data cannot leave the internal network, or physical isolation is required | Data can be processed through compliant channels |
| Usage pattern | Long-term, high-concurrency, stable load | Highly variable demand, or short-term demand is hard to forecast |
| Team capability | Has GPU cluster and inference operations experience; needs customization or fine-tuning | Wants fast launch and elastic scaling |
If self-hosting is chosen, three things need to be prepared: hardware, framework, and operations.
Hardware:
Configure host memory, GPU count, and storage as described in Section 5, and reserve room for expansion.Framework:
The inference framework must support floating-point 4-bit weights, expert parallelism, CPU offload, and paged KV cache. A single-node validation is recommended first.Operations:
Monitor GPU memory and host memory usage, paging frequency, per-request latency, and throughput. Frequent paging is often the first sign of performance degradation.
For many teams, a more pragmatic approach is to deploy the open-weight model in a controlled environment as a shared inference node that serves multiple business teams. This can satisfy data residency requirements while amortizing per-token cost through batching.
For teams that do not want to build and operate their own hardware, cloud service providers can lower the entry barrier. Baidu AI Cloud, for example, offers related choices across heterogeneous computing, containerized inference, and model services. Teams should verify regional availability and terms. Ultimately, the choice should be based on evaluation results on your own tasks, not on parameter count or leaderboard position.
7. Summary
This weight release sends a clear signal: open-weight models have entered the trillion-parameter era, and the engineering threshold has risen accordingly.
The 2.8T total parameter count and roughly 104B activated parameters in an MoE architecture determine the core tradeoff: capacity depends on total parameters, while speed depends on memory bandwidth.
The 4-bit quantization makes the download size practical, but it also ties deployment to supported inference frameworks.
The custom license is mainly permissive, but commercial teams still need to check the additional clauses item by item.
“8 GPUs to start” is essentially the minimum configuration under a memory-tiered architecture.
For developers facing trillion-parameter open-weight models, three questions matter more than whether the model can be downloaded:
- Is commercial use allowed in your business? Start with the license.
- Can your hardware and framework handle it? VRAM, host memory, and bandwidth must be calculated together.
- Does it actually fit your use case? Coding and agentic tasks are worth evaluating first, while scientific and rigorous reasoning scenarios require careful validation.
