2.8T Open-Weight Model License and Local Deployment Guide
This guide explains the full release of a 2.8T-parameter open-weight multimodal model and covers what users need to know before deploying it locally: model structure, 4-bit microscaling quantization, custom license restrictions, vendor-reported benchmark results, VRAM and GPU-count estimates, download and verification steps, and a practical selection checklist. It is intended for teams evaluating private deployment, API-based use, or hardware planning for large sparse Mixture-of-Experts models.
1. Release Overview: Full Weights for a 2.8T Open-Weight Model
On November 4, the complete weights of a newly released open-weight model were made available as previously promised. The model has approximately 2.8 trillion total parameters and about 1.56 TB of weight files, making it one of the largest open-weight models released so far. Earlier, on October 24, the model had already been made available through an API and a web interface. At that time, the team promised to release the full weights before November 4, and it delivered on that commitment.
A few details about the release process are worth noting:
- Release timing: The weight repository was created on November 2. It was still inaccessible on the morning of November 4 and became publicly available that evening. No application or review was required.
- File scale: The repository contains 112 files in total, including 96 sharded weight files with a combined size of about 1.56 TB. If a mirror is available through the model hosting platform in your region, it may also be used to synchronize the files.
- What was released: The release includes the trained model parameters. These weights can be loaded locally, fine-tuned, and redistributed. However, the training data, training code, data mixture, and detailed training documentation were not released.
- Technical focus: Because the model uses a Mixture-of-Experts, or MoE, architecture, local deployment depends not only on GPU memory but also on expert routing, inter-card interconnect, KV cache behavior, and quantization format.
This article focuses on what happens after the weights become available: what the model contains, how its license should be interpreted, how much VRAM and how many GPUs are required for local deployment, and whether ordinary users can realistically run it.
2. Model Structure: How 2.8T Parameters Are Organized
2.1 Total and Active Parameters: Two Different Budgets for Sparse Models
According to the model documentation, the model is a natively multimodal agent-oriented model. It was trained further from the previous generation. Its total parameter count is 2.8 trillion, but only about 104 billion parameters are activated for each token, or roughly 3.7% of the total.
This sparse design creates two separate considerations:
- Compute depends on active parameters: For each generated token, the amount of multiply-accumulate work is roughly proportional to the number of activated parameters. Using the common estimate of about 2 floating-point operations per parameter, and excluding the attention portion, each token requires approximately 2.1×10¹¹ floating-point operations. This is comparable to a dense model in the 100-billion-parameter range.
- Storage depends on total parameters: Even for a single request, all expert weights must be resident in memory or able to be swapped in quickly when needed.
This separation between “cheap to compute” and “expensive to store” is key to understanding the deployment conclusions later in this article.
2.2 Expert Configuration: 896 Experts, Only 18 Activated per Token
Each layer of the model uses a hybrid MoE design combining fine-grained experts with shared experts:
- There are 896 routed experts. For each token, the routing network selects 16 of them.
- There are also 2 shared experts, and every token passes through both.
- In total, 18 experts are activated per token.
Fine-grained experts make routing combinations more flexible. The same token can activate completely different expert subsets at different layers. The tradeoff is higher engineering complexity, mainly in three areas:
- Routing overhead: For every token and every layer, the model must compute scores for all 896 experts and select the top 16.
- Load balancing: Popular experts can become bottlenecks. During training, auxiliary losses or similar techniques are usually needed to keep expert load balanced.
- Expert parallelism: During deployment, experts are split across multiple GPUs. Each token must exchange data with the GPUs holding its target experts, so inter-card interconnect bandwidth directly affects inference efficiency.
2.3 Layers, Context Length, and Multimodality
The model has 93 layers and supports a context length of 1 million tokens. Vision is handled by a self-developed visual encoder, making the model natively multimodal. Text and images are modeled within the same model rather than through a separate external vision module.
The visual encoder shares the same inference runtime as the text layers. This has two implications:
- Image processing introduces additional preprocessing overhead and visual tokens.
- Multimodal capabilities can be used directly in agent tasks. For example, the model can interpret webpage screenshots, interface elements, and charts before performing actions.
2.4 Structural Difference from the Previous Generation: Sparse Attention in Some Layers
Compared with the previous generation, only one structural change was made: full attention in 24 layers was replaced with sparse attention.
The core idea of sparse attention is that each token only computes attention with a filtered subset of historical tokens instead of all historical tokens. In million-token contexts, full attention computation grows quadratically with sequence length. After replacing 24 layers with sparse attention, the attention overhead in those layers is reduced significantly.
One important point is that sparse attention saves compute and bandwidth, but it does not reduce context storage. The key-value, or KV, cache for historical tokens usually still needs to be fully retained or only lightly compressed. Therefore, sparse attention relieves the compute pressure caused by long contexts, but it does not eliminate the VRAM pressure.
2.5 Quantization Format: Released Directly in 4-Bit Weights
The weights use a 4-bit microscaling floating-point format, while activations use an 8-bit microscaling floating-point format.
Microscaling floating point works by grouping numerical values into blocks of 32 elements. Each block uses a shared scaling factor and a small number of mantissa bits. This preserves the flexibility of floating-point dynamic range while compressing storage and bandwidth to a level close to 4-bit integers.
This quantization is the result of quantization-aware training, meaning the model learned with quantization error present during training. Compared with post-training quantization, quantization-aware training usually keeps precision loss smaller.
Two points are easy to overlook:
- Only this one version is officially provided. There is no BF16 or FP8 version. If users need another precision format, they must dequantize locally. This cannot restore the original precision, and VRAM usage may increase significantly.
- Fine-tuning directly on 4-bit weights requires caution. The model has already adapted to quantization error during inference. Continued training should use methods suitable for low-precision base models and include sufficient validation. It should not be assumed to be equivalent to a full-precision base model.
3. Performance: What to Watch When Reading Vendor-Reported Data
The comparison data provided in the model documentation is vendor-reported. The comparison models are contemporaneous closed-source flagship models.
| Evaluation area | This model | Comparison model |
|---|---|---|
| Terminal task execution benchmark | 65.7% | 66.4% |
| Long-horizon software engineering task benchmark | 51.7% | 46.3% |
| Scientific task execution benchmark | 38.1% | 63.3% |
| Hard composite knowledge reasoning benchmark | 39.4% | 60.9% |
The vendor also states that this model leads on 16 out of 37 benchmarks. The results are mixed: it clearly leads on long-horizon software engineering tasks, is roughly tied on terminal task execution, and lags significantly on scientific tasks and hard composite reasoning.
Third-party evaluation information mentioned in the original material is not used as a basis for conclusions because it lacks verifiable organization names, report titles, release dates, and links. If such information is cited, it should be based on official public reports.
When reading this type of data, three points are useful:
- Distinguish vendor-reported data from third-party testing: Vendor-reported results are often optimized for specific prompts and evaluation procedures. Third-party tests usually have non-public questions and more standardized processes. If the conclusions differ, testing in your own business scenario should take priority.
- Match results to task type: The model performs relatively strongly on coding and agent tasks, making it suitable for code assistance and long-workflow automation. It is weaker on scientific and complex reasoning tasks, where results may require human review or cross-validation with other tools.
- Pay attention to large gaps: For example, the scientific benchmark is 25 percentage points behind the comparison model. In domains requiring rigorous reasoning, this model may not be the first choice.
4. License: Open Weights Do Not Mean Open Source
4.1 A Custom License Is Used
The model uses a custom license. On the model hosting platform, it is labeled as “Other.” It is neither a standard open-source license such as MIT or Apache 2.0, nor a dedicated open-weight license.
From the text, the license appears to be word-for-word identical to the previous generation’s license terms, with only the name changed. This suggests that the open-weight strategy follows the previous generation rather than being changed specifically for this release.
The model card, license file name, and version date should be based on the repository content released that day. If the repository does not provide complete license text, commercial evaluation should be paused until the official text in the README, LICENSE file, or model card is confirmed.
4.2 MIT-Style Grant Plus Two Additional Restrictions
After removing the name, the main body of the license is MIT-style. It permits free use, copying, modification, distribution, fine-tuning, and commercial use. The real difference is two additional restrictions layered on top, both aimed at external business operations that reach a certain scale:
- Model-as-a-Service, or MaaS: Companies whose primary business is offering model services externally and whose cumulative revenue over the past 12 months exceeds 20 million US dollars must sign a separate agreement with the model team before commercial use.
- Large-scale commercial products: When a commercial product’s monthly active users exceed a defined threshold, separate authorization is also required.
The intent is clear: the license is open to small teams and individuals, while the model team retains the right to negotiate separately with commercial services that have already reached significant revenue or user scale.
However, public information does not clearly disclose the specific monthly active user threshold, counting method, or calculation period. Therefore, “defined threshold” should not be interpreted as having no restriction. Before commercialization, the agreement text must prevail and legal review is required. In particular, users should confirm:
- Whether monthly active users refers to end users, accounts, enterprise tenants, or actual API calls.
- Whether internal users and external users are treated differently.
- Whether the count is based on daily, weekly, monthly, or rolling 12-month statistics.
- Whether revenue includes custom projects, hybrid cloud deployments, private delivery, and similar income.
- Which forms of service are considered MaaS.
4.3 “Open Source,” “Open Weights,” and “API-Only” Are Different
The industry usually distinguishes three levels of model openness:
| Type | Usually includes | Additional restrictions |
|---|---|---|
| Open-source model | Weights, training code, data or detailed data description, under a standard open-source license | Very few |
| Open-weight model | Trained model parameters | Often includes usage restrictions |
| API-only model | Accessible only through an API | Defined by the service provider |
Under the common definition of open source, software should allow free use for any purpose without additional restrictions. A license that adds conditions based on revenue or user scale does not meet this standard and should be called “open-weight” rather than “open-source.”
4.4 Pre-Commercial Use Checklist
- Revenue scope: Check whether MaaS revenue includes custom projects, hybrid cloud deployments, or similar income, and whether monthly tracking and record-keeping are required.
- Monthly active user scope: Check whether the count is based on end users or accounts, whether internal and external users are distinguished, and remember that public information does not provide specific numbers or definitions. The agreement text must be used as the reference.
- Business model: Determine whether providing model capabilities for customers to call, embedding the model into a SaaS product, or further developing and selling based on the model falls into a restricted category.
- Redistribution: Check whether weights can be provided to customers for deployment inside the customer’s environment and whether secondary distribution requires additional conditions.
- Derivative models: Check whether fine-tuned models inherit all obligations under the original license.
- Responsibility boundaries: Service terms, user agreements, and data processing policies should remain consistent with the model license.
5. Local Deployment: How Much VRAM and How Many GPUs Are Needed?
5.1 Estimating VRAM Requirements
Using several public parameters, a conservative estimate can be made:
- Weights: 2.8 trillion parameters × 0.5 bytes ≈ 1.4 TB. This does not include unquantized components such as the visual encoder and vocabulary-related parameters.
- KV cache: With 93 layers, 8 KV heads, head dimension 128, and a half-precision cache, each token requires roughly 0.37 MB. For 1 million tokens, the KV cache requires about 355 GiB.
- Framework overhead: Runtime components, communication buffers, and memory fragmentation can add overhead. A 15% margin is included.
The conclusion is that supporting a single 1-million-token request requires at least approximately 1.9 TB of VRAM.
The following illustrative code can reproduce the estimate and compare different GPU memory tiers. The code uses GiB as the output unit. Vendor-stated capacities such as 80 GB, 141 GB, and 192 GB often differ from GiB, so actual procurement and deployment should be based on the visible VRAM reported by the GPU vendor and the inference engine.
# Illustrative capacity estimate for order-of-magnitude judgment onlydef estimate_vram(total_params=2.8e12, # total parametersbytes_per_weight=0.5, # 4-bit microscaling floating point, 0.5 bytes per parameterlayers=93,kv_heads=8, # assumed value; confirm with the model configurationhead_dim=128,cache_bytes=2, # KV cache precision in bytesseq_len=1_000_000,):weight_gib = total_params * bytes_per_weight / 1024**3kv_per_token = layers * kv_heads * head_dim * 2 * cache_byteskv_gib = kv_per_token * seq_len / 1024**3need = (weight_gib + kv_gib) * 1.15 # about 15% framework overheadprint(f"Weights {weight_gib:.0f} GiB | KV cache {kv_gib:.0f} GiB")for capacity in (80, 141, 192):cards = -(-need // capacity) # round upprint(f"{capacity} GiB per card: at least {int(cards)} cards")estimate_vram()
With typical assumptions, the output is as follows:
Weights 1304 GiB | KV cache 355 GiB80 GiB per card: at least 24 cards141 GiB per card: at least 14 cards192 GiB per card: at least 10 cards
This shows that the idea of “starting with 8 GPUs” only holds under two conditions:
- Each GPU has VRAM close to 192 GiB, and the context length is much smaller than 1 million tokens.
- KV cache quantization or other compression methods are enabled, and concurrency is very low.
For stable service with long contexts and concurrent requests, practical planning usually requires around 12 to 16 GPUs. Additional VRAM should also be reserved for batching, visual encoding, runtime caches, and communication buffers.
VRAM requirements also vary significantly with context length and KV head count. The following is a rough sensitivity reference:
| Context length | Assumed KV heads | Estimated KV cache | Notes |
|---|---|---|---|
| 128K tokens | 8 | About 46 GiB | Suitable for shorter contexts or a single request |
| 1M tokens | 8 | About 355 GiB | Corresponds to the default assumptions in the code above |
| 1M tokens | 16 | About 710 GiB | If the actual KV head count is higher, VRAM pressure increases significantly |
Before deployment, users should check the model configuration file for the number of layers, KV heads, head dimension, KV cache precision, and whether KV cache compression is supported. The calculation should then be repeated with those confirmed values.
5.2 Two Bottlenecks Beyond VRAM
1. Compute and interconnect
- Decoding, or token-by-token generation: This is mainly limited by memory bandwidth. Each token must read at least about 52 GB of activated expert weights. The aggregate memory bandwidth across 8 GPUs needs to reach the TB/s scale.
- Prefilling, or processing long inputs in one pass: Prefilling 1 million tokens requires approximately 2×10¹⁷ floating-point operations. Even if 8 high-end GPUs provide more than 10 PFLOPS in total, the ideal time is around 20 seconds. In practice, it will take longer.
- Expert parallel communication: Routing tokens across GPUs also consumes bandwidth. If inter-card interconnect is weak, communication can become the main bottleneck.
2. Single-GPU VRAM
Consumer GPUs with 8 GB to 24 GB of VRAM cannot meet even 1% of the total requirement. For local inference of such large-scale models, offloading to system memory or SSD storage can only achieve extremely slow operation and has little practical value.
6. Download and Deployment: A Practical Checklist
6.1 Preparation
- Storage: Reserve at least 2 TB of high-speed local storage. During download and shard merging, two copies of the data may exist at the same time. Multi-copy or multi-version environments should scale this estimate accordingly.
- Download: Prefer the command-line download tool provided by the model hosting platform and enable resumable downloads. If a regional mirror is available, it may also be used.
- Verification: Official files usually include a per-file hash list. When there are many shards, full verification is especially important. If the official release does not provide a
checksums.txtfile, this step can be skipped, but it is still advisable to record file sizes, shard count, and the repository version.
# Check the hash list; failed files must be re-downloaded# If checksums.txt is not provided, skip this stepsha256sum -c checksums.txt 2>&1 | grep -v ': OK'
6.2 Loading and Serving
- Runtime selection: Choose an inference engine that supports the model architecture, including sparse attention, Mixture-of-Experts, and microscaling floating point. For multi-GPU scenarios, prefer a combination of tensor parallelism and expert parallelism.
- Quantization consistency: The weights and activations are already quantized. Do not stack additional quantization in the runtime, as this can introduce double error.
- Context and cache: Set the context limit based on actual business needs. Enable KV cache quantization and avoid allocating full VRAM for peak scenarios all at once.
- Multimodal resources: Reserve separate compute resources and VRAM quota for the visual encoder so that image requests do not squeeze out text decoding resources.
6.3 Cloud Deployment
If cloud deployment is required, general cloud capabilities can support a layered design:
- Object storage can be used for weight distribution and version archiving.
- A container platform can be used to orchestrate multi-GPU inference instances.
- Logging, monitoring, and alerting services can track VRAM usage, batch queue time, and throughput metrics.
6.4 Continuous Tuning
After deployment, continuously monitor VRAM usage, first-token latency, per-GPU throughput, and routing load distribution. Use these metrics to adjust parallelism strategy and batching parameters. Avoid designing configurations only around theoretical peak values while ignoring the actual traffic pattern.
7. Selection Checklist: Who Should Consider This Type of Model?
| Dimension | Key considerations | Recommendation |
|---|---|---|
| License | Whether revenue or monthly active users cross additional authorization thresholds | If the threshold may be crossed, do not launch an external service before commercial terms are finalized. Monthly active user definitions must follow the agreement text. |
| Hardware | Single-GPU VRAM tier and available GPU count | Recalculate requirements using target context length and concurrency with the illustrative code above. |
| Quantization | Whether only the 4-bit weight version is available | If full-precision fine-tuning or precision comparison is required, perform small-sample validation first. |
| Capability | Whether the model’s strengths are coding and agent tasks | Add human review for scientific and complex reasoning scenarios. |
| Ecosystem | Whether the inference engine supports the model architecture | Perform full functional and performance testing before launch. |
Based on these considerations, three typical scenarios can be described:
- Large teams with private deployment needs: This typically requires building an inference resource pool at the thousand-GPU level. It is often more practical to expose model capabilities as an inference service than to place a copy of the weights on every machine.
- Small and medium-sized teams: Instead of building a cluster, it is usually more efficient to call the model through an API and focus on business integration.
- Individual developers: Running the model locally has little practical value. Shared compute resources or on-demand cloud compute can be used for small-scale experiments.
8. Conclusion
The full release of this 2.8T-parameter open-weight model represents a new step in the scale of open-weight models. However, “open” does not mean “without barriers”:
- License: It uses a custom license that combines an MIT-style grant with additional restrictions for scaled commercial use.
- Deployment: 8 GPUs are only the starting line. Long-context and high-concurrency scenarios typically require around 12 to 16 high-end GPUs or more.
When evaluating this type of model, start with the restrictive clauses in the license, calculate VRAM requirements using your own real context length and concurrency, and finally run tests with data from your own scenario. Only when all three steps are completed can the model be considered truly usable.
