2.8T Open-Weight Model Release: Architecture, License, and Deployment Analysis
A major Chinese LLM vendor has released the full weights of a 2.8-trillion-parameter flagship model under a custom open-weight license. This article breaks down what technical teams actually need to know: how the 2.8T sparse Mixture-of-Experts architecture is organized (104B active parameters, 93 layers, 896 routed experts, sparse attention, and 4-bit weights with 8-bit activations), how to read the vendor's self-reported benchmark results, what the license does and does not permit for commercial use, and whether a single node with 8 GPUs can serve the default 1.56TB of weights. It also walks through the VRAM and throughput math behind that question, offers a build-versus-API cost comparison, and gives selection guidance for individuals, mid-sized companies, and large enterprises.
1. Overview: The Weights Are Out, but This Is Not Standard Open Source
On November 4, a Chinese LLM vendor released the full weights of a flagship model, following through on an earlier promise. The model had first launched via API and a web interface in late the previous month, with a commitment to open the weights within a month. The weight repository was created on a major model-hosting platform on November 2 and opened on the evening of the 4th. The repository is publicly downloadable with no application required. Of its 112 files, 96 are weight shards totaling about 1.56TB. This is one of the largest open-weight models available to download to date, far larger than previously released open-weight models.
For a technical team, only three questions are worth answering:
- What is inside: how the 2.8T parameters are organized, and what the sparsification and quantization bring;
- How the license works: whether open weights mean you can freely use it commercially;
- Whether 8 GPUs are enough: where the memory boundary lies between the default 1.56TB of weights and a single node with 8 GPUs.
One point deserves emphasis: the model uses a custom open-weight license, not a standard open-source license. If you intend to commercialize it as a model-as-a-service offering or as part of a large commercial product, you must check the revenue and monthly-active-user thresholds.
2. Model Architecture: What Makes Up the 2.8T
2.1 Overall Specifications
According to the model card, this is a natively multimodal Agent model, continued-trained from the previous flagship generation:
| Dimension | Specification | Engineering implication |
|---|---|---|
| Total params / activated per token | 2.8T / 104B (~3.7%) | Capacity and compute are decoupled |
| Architecture | MoE, 93 layers | Fine-grained expert routing |
| Expert configuration | 896 routed experts, 16 selected per token, plus 2 shared experts | Up to 18 experts involved per token |
| Attention | 24 layers of sparse attention, remaining 69 layers full attention | Reduces long-context cost |
| Context | 1M tokens | Long documents and long-chain tasks |
| Vision | Built-in vision encoder, understands images | Native multimodal |
| Quantization | 4-bit weights, 8-bit activations (fine-grained block floating point) | The only official version; no BF16/FP8 variant |
Compared with the previous generation, only one published spec changed: what were 24 full-attention layers are now sparse attention.
2.2 Sparse MoE: More Capacity, Less Compute
A dense model activates all of its parameters for every token, so capacity and compute grow together. A Mixture-of-Experts (MoE) model splits the feed-forward layers into a large number of “experts” and activates only a small subset for each token, which makes it possible to scale capacity substantially while keeping compute bounded. With an activation ratio of about 3.7%, this model means you pay memory for 2.8T parameters but compute for only 104B. That is the premise for understanding the deployment question later.
Two design points stand out in the expert configuration:
- Fine-grained routing: with 896 experts and only 16 selected at a time, the division of labor is finer and parameter combinations are more flexible. The cost is that routing decisions carry more weight; when expert load is uneven, a “weakest-link” effect can appear, so engineering practice needs expert parallelism paired with routing constraints.
- Shared experts: unlike the per-token routed experts, 2 shared experts are always active for every token. They typically carry general-purpose knowledge, letting the routed experts focus on differentiated capabilities.
2.3 Sparse Attention: Lightening the Load for Million-Token Contexts
In standard self-attention, every token attends to all previous tokens, so both compute and the KV cache grow with sequence length. Sparse attention lets most tokens attend only to a local window and a small amount of key history, bringing the attention cost of long sequences close to linear. At the same time, 69 full-attention layers are kept for global information exchange, balancing efficiency with long-range dependencies. A common implementation approach looks like this:
# Illustration: an alternating layer attention strategy (structural intent only, not the official implementation)def layer_attention_type(layer_idx, num_layers=93, sparse_layers=24):sparse_stride = num_layers / sparse_layersis_sparse = int(layer_idx / sparse_stride) != int((layer_idx + 1) / sparse_stride)return "sparse" if is_sparse else "full"
Note that sparsification reduces the attention compute; the KV cache for every token still has to be stored, so the memory pressure of long context does not disappear on its own.
2.4 Quantization: 4-Bit Weights and 8-Bit Activations
Weights use a 4-bit fine-grained block floating-point representation, where a group of values shares one scale factor; activations use 8-bit floating point. Fine-grained blocking keeps numerical outliers contained within small groups and is a common technique for preserving accuracy at low bit widths. Quantization-aware training simulates quantization error during training so the model learns to adapt to lower precision, and generally loses less than post-training quantization. Both formats are block floating-point schemes with configurable block size, so the estimates below assume “an extra 0.25 bit of scale factor per weight.” This is only an order-of-magnitude assumption; defer to the weight-file metadata.
The vendor releases only this single version—4-bit weights and 8-bit activations—which has three direct consequences:
- Memory-friendly: storing 2.8T parameters in BF16 would take about 5.6TB; 4-bit needs about a quarter of that;
- No fallback precision: if you want a BF16 or FP8 version, you can only dequantize it yourself, and the size and memory footprint multiply several times over, while accuracy cannot exceed the original weights;
- Compute-format requirements: at inference time the weights usually have to be restored to 8-bit or 16-bit to participate in computation, which requires both the runtime and the kernels to support fine-grained block scaling—this is the first compatibility condition to verify when selecting a stack.
3. Performance: How to Read the Vendor’s Self-Reported Scores
Start with the vendor’s self-reported comparison data. The main point of comparison is a leading international closed-source model (referred to here as Model A):
| Benchmark dimension | This model | Reference Model A | Relative position |
|---|---|---|---|
| Terminal coding benchmark 4.0 | 65.7% | 66.4% | Slightly lower |
| Long-horizon software engineering benchmark | 51.7% | 46.3% | Ahead |
| Scientific terminal task benchmark | 38.1% | 63.3% | Clearly behind |
| High-difficulty multidisciplinary reasoning benchmark (curated subset) | 39.4% | 60.9% | Clearly behind |
| Composite intelligence index (independent third-party evaluator) | 55 | — | 4th globally |
The vendor states that of 37 benchmarks, 16 beat Model A—an overall mixed record. The model is stronger on coding and Agent tasks and weaker on scientific and some reasoning tasks.
Three points matter when reading this kind of data:
- Source: apart from the composite intelligence index, which comes from an independent third party, all figures are vendor-reported. The reference model, prompt templates, and evaluation configuration are chosen by the vendor, so they should not be taken as direct cross-model conclusions.
- Distinguish the aggregate from the sub-items: the composite index aggregates many capabilities, so for selection you should look more at the sub-item closest to your scenario. If you are building a coding assistant, focus on the coding results; if you are doing scientific analysis, watch out for its clear weaknesses.
- Test under unified conditions: re-run with your own eval set, under exactly the same decoding parameters and tool configuration.
A self-built eval set should cover:
- Sample size: at least 300 items; when the number of tasks is small, report a range rather than a single point;
- Hard-item share: 20% to 40%, otherwise there is not enough discrimination;
- Repetitions: at least 3 per task, reporting the mean, standard deviation, or confidence interval;
- Task weighting: weight by business distribution—for example, coding 40%, Agent 30%, reasoning 20%, safety 10%.
# Illustration: a multi-round evaluation flow under unified conditions (pseudocode)TASKS = [{"id": "T001", "prompt": "...", "judge": judge_exact_match, "weight": 0.4},{"id": "T002", "prompt": "...", "judge": judge_unit_tests, "weight": 0.3},{"id": "T003", "prompt": "...", "judge": judge_llm_rubric, "weight": 0.3},] # tasks drawn from a real business distribution; validators decide automaticallydef evaluate(model_endpoint, tasks, repeats=3):scores = []for r in range(repeats):for t in tasks:out = model_endpoint.generate(t["prompt"], temperature=0.0, max_tokens=8192)scores.append(t["judge"](out))return {"mean": mean(scores), "std": std(scores)} # watch the variance, not a single run
If your workload heavily involves scientific computing or complex reasoning, validate with your own eval set before committing compute.
4. License: Open Weights Are Not Open Source
This is the single point most worth calling out on its own. The repository carries a custom model license, and the hosting platform labels it as “other” rather than a standard open-source license. The license is word-for-word identical to the previous generation, with only the name changed.
Its core is a permissive, MIT-like grant: free to use, copy, modify, distribute, fine-tune, and use commercially. On top of that, the license adds two restrictions that apply only to large-scale external businesses:
- Model-service revenue threshold: a company whose business is offering the model as a service and whose cumulative revenue over 12 consecutive months exceeds USD 20 million must sign a separate agreement with the model provider before commercial use;
- Large-scale commercial products: a commercial product whose monthly active users reach a certain scale likewise requires a separate license (the exact threshold is governed by the license text).
The fact that it is word-for-word identical means the compliance conclusions from the previous generation carry over directly. Mainstream open-source licenses are typically characterized by not placing extra restrictions on fields of use; this license adds conditions based on the entity’s size and revenue, so the more accurate description is “open weights” rather than standard open source.
4.1 Who Is Affected
- Essentially unaffected: individual developers, researchers, the vast majority of small and medium-sized businesses, and model-service providers below the revenue threshold;
- Requires a separate agreement: model-service providers that reach the revenue threshold, or commercial products whose monthly active users reach a certain scale;
- Passed on with the weights: derivative models that are modified, fine-tuned, and redistributed are typically still bound by this license, so a copy of the license should accompany distribution.
4.2 Compliance Self-Check Example
First, pin down the definitions: the revenue threshold is triggered only for “offering the model as a service”; the MAU threshold is triggered for “commercial products”; the two need to be evaluated in combination with the business model.
# Illustration: a self-check flow based on the license terms (values follow the license text; not legal advice)def check_license(model_as_service=False, # whether the model is offered as a servicerevenue_12m_usd=0.0, # cumulative revenue over 12 consecutive months (USD)product_mau=0, # monthly active users of the commercial product hosting the modelmau_threshold=None, # MAU threshold defined by the licensedistribute_derivatives=False,):if model_as_service and revenue_12m_usd > 20_000_000:return "a separate commercial agreement with the model provider is required"if mau_threshold is not None and product_mau > mau_threshold:return "an additional license from the model provider is required"if distribute_derivatives:return "distribution allowed, but a copy of the license must be included"return "free to use and commercialize within the scope of the license"
Three recommendations in practice:
- Confirm the definitions: whether revenue is counted per entity or per product line, and how MAU is measured, are governed by the license text; seek legal advice when necessary;
- Keep records: retain the model version, a snapshot of the license text, and compliance-assessment records for auditing;
- Hedge risk: for businesses that may cross the revenue or MAU line, isolate the access layer architecturally so the model layer can be swapped.
In addition, whether or not you reach the thresholds above, it is advisable to follow common conventions:
- Do not imply official endorsement in external marketing;
- Disclose the base model and any important modifications;
- Do not use the model’s output for unlawful purposes.
5. Local Deployment: Are 8 GPUs Enough, and How to Run It Stably
5.1 First, Correct a Common Intuition
Many people assume an MoE only needs to load the active 104B parameters. That is wrong: routing depends on the input, and any token can hit any expert. So all 2.8T parameters must be available on demand during inference. VRAM capacity is determined by the total parameter count; compute is determined by the active parameters.
5.2 The VRAM Ledger
1.56TB is the download size; at runtime, VRAM has to add extra overhead on top:
| Item | Estimated value | Notes |
|---|---|---|
| Weights | ~1.49TB | 4-bit params + scale factors, ~1487.5GB |
| KV cache | Tens to hundreds of GB | Assuming 256B for K and V combined per layer per token, 1M tokens ≈ 22.2GB/request; actual depends on KV head count, head dimension, and precision, and grows with concurrency |
| Activations and vision encoder | A few GB to tens of GB | The vision encoder is resident separately; batch size and resolution affect activations |
| Runtime and communication buffers | Tens of GB | Dequantization buffers, parallel communication buffers, inference-engine overhead |
| Total | ~1.5TB+ | Varies by implementation |
The difference between the 1.56TB download size and the ~1.49TB weight estimate usually comes from storage format, sharding, metadata, alignment, checksums, and file-organization overhead. This article’s estimates use decimal GB/TB; defer to the weight metadata.
# Illustration: single-node multi-GPU VRAM capacity estimation (in GB, for capacity judgment only)def estimate_vram(params_B, # total parameters (B), e.g. 2800w_bits=4, scale_bits=0.25, # weight bit width, extra scale-factor overhead per weightkv_bytes_per_token=93 * 256, # KV footprint of all layers per token (bytes, assumed)seq_len=1_000_000,batch=1,extra=80, # activations, vision encoder, runtime, and communication buffersnum_gpus=8, gpu_mem=192, # GPU count and per-GPU memory):weights = params_B * (w_bits + scale_bits) / 8kv_cache = kv_bytes_per_token * seq_len * batch / (1 << 30)total = weights + kv_cache + extracapacity = num_gpus * gpu_memreturn {"weights": round(weights, 1),"kv_cache": round(kv_cache, 1),"extra": extra,"total": round(total, 1),"capacity": capacity,"remaining": round(capacity - total, 1),}# roughly {'weights': 1487.5, 'kv_cache': 22.2, 'extra': 80, 'total': 1589.7, 'capacity': 1536, 'remaining': -53.7}print(estimate_vram(params_B=2800))
By the default example, 8×192GB needs about 1589.7GB against a capacity of about 1536GB—a gap of about 53.7GB—so you cannot simply conclude that “8 GPUs can run it.” A more sound conclusion is: 8 GPUs may start up only under specific boundary conditions.
- If you load only the weights, 8×192GB has only about 48.5GB left for KV cache, vision, runtime, and communication buffers;
- If the KV cache is about 22GB, the extra overhead must be compressed to within about 26GB;
- With a 1M-token context, high concurrency, and high-resolution vision tasks stacked together, 8×192GB is insufficient;
- Only with a short context, low concurrency, a disabled or trimmed vision branch, and extreme runtime optimization might it barely start.
So with the default million-token context and extra=80, 8 GPUs are not recommended; high-concurrency production usually needs more GPUs, more nodes, or host-memory pooling with an accepted drop in throughput.
5.3 Throughput Estimate: Why the Weights Must Stay in VRAM
During decoding, generating each token requires reading the expert weights activated for that step: 104B × 0.5 bytes ≈ 52GB. The estimate below separates the bandwidth sources to avoid conflating aggregate HBM with host-memory bandwidth.
| Weight location | Bandwidth assumption | Per-token transfer | Theoretical floor | Expected at 30% effective utilization |
|---|---|---|---|---|
| All in VRAM | 8-GPU aggregate HBM ~25TB/s | 52GB | ~2.1ms, ~480 token/s | hundreds, ~140 token/s |
| ~30% in host memory | 8 host channels aggregate ~100GB/s | ~15.6GB | ~156ms, ~6.4 token/s | ~2 token/s |
| All in host memory | ~100GB/s | 52GB | ~520ms, ~1.9 token/s | <1 token/s |
The 30% effective-utilization figure here reflects real engineering losses: an MoE’s all-to-all expert dispatch communication, dequantization, scheduling overhead, and batch bubbles all pull down measured throughput. So when “all in VRAM,” it is not right to summarize as “tens of token/s”; a more reasonable expectation is hundreds of token/s. If the inference framework is poorly optimized or cross-node communication becomes the bottleneck, it can still fall below that range.
The key to fitting this model into 8 GPUs is precisely that 4-bit weights cut the bytes read per token to a quarter of BF16. Even on the same 8 GPUs, BF16 weights could not all stay resident in VRAM, and speed would drop sharply.
5.4 Deployment Checklist
- License first: check the revenue and MAU thresholds to confirm whether a separate agreement is needed, and retain assessment records;
- Official docs and engine compatibility: first verify the official deployment documentation, inference-engine version, fine-grained block quantization format, and sparse-attention support;
- Inventory: downloading ~1.56TB requires matching storage, and total node VRAM should be at least on par with weights plus KV cache;
- Parallelism strategy: prefer tensor parallelism plus expert parallelism, keeping the communication domain on fast intra-node interconnects where possible. If you need cross-node, cross-machine all-to-all is sensitive to network latency and bandwidth;
- Capacity isolation: set independent queues and quotas for requests of different context lengths and modalities so a single long request does not drag down the whole system;
- Observability: build separate monitoring and alerts for time-to-first-token, inter-token latency, throughput, VRAM usage, and expert load distribution.
For enterprises that need data to stay within their own domain and to run stably over the long term, the weights can sit in object storage, an inference cluster can be brought up on a container platform, and log and monitoring services can keep tabs on operations. General-purpose cloud capabilities—object storage, message queues, container platforms, log services, monitoring and alerting—can cover these pieces. But such very large models place heavy demands on the underlying compute platform and inference-optimization capabilities, so be sure to validate with real testing before self-building.
6. Selection Advice: Are Open Weights Worth It
The real value of open weights is control: data stays within your domain, it is customizable, it is not locked to a single vendor, and it can satisfy compliance audits. The cost is the hidden expense of self-hosting:
- Fixed investment: high-end accelerator cards, high-bandwidth hosts and networking, data-center space and power;
- Operations burden: inference-engine adaptation, performance tuning, fault handling, security hardening;
- Technical barrier: an engineering team with distributed-inference experience.
The cost can be roughly estimated as:
self-hosted monthly cost ≈ hardware depreciation (over 3 years) + power and data center + bandwidth and storage + operations staffAPI monthly cost = input tokens × input price + output tokens × output price
For example, if self-hosting costs about 150,000 per month and the blended API price works out to about 5 per million tokens, self-hosting and API costs break even at around 300 million tokens per month. This example is only to illustrate the method; substitute real prices and hardware depreciation figures. When call volume is low and volatile, the API is more economical; when call volume is high and steady and there are data-compliance requirements, self-hosting has the edge.
Recommendations by scenario:
| Team type | Recommendation |
|---|---|
| Individual / small team | Prefer the official API or a third-party hosted service; to try it out, download the quantized version and run it offline on a machine with enough VRAM |
| Mid-sized enterprise | First compare candidate models with your own eval set; if call volume is high and steady, consider renting 8-GPU or larger nodes for inference |
| Large enterprise / industry customer | Deploy privately under compliance requirements, focusing on inference optimization, capacity planning, and expert load monitoring |
A suggested rollout path in four steps:
- Self-check against the license;
- Size the hardware against the VRAM and throughput ledger;
- Validate quality with your own eval set;
- Run a small canary and then scale up gradually.
7. Conclusion
Three key conclusions from this weight release:
- Architecture: sparse MoE, sparse attention, and 4-bit quantization together push the deployment threshold for a trillion-parameter model down to around a single node with 8 GPUs, but under the default million-token context, 8×192GB is not generous; the cost is that there is no higher-precision official version;
- Performance: coding and Agent tasks approach the top international closed-source models, while science and some reasoning tasks still lag noticeably, so selection should be based on specific evaluations;
- License: this is open weights with revenue and MAU conditions, not standard open source; before commercializing, check whether your business triggers an additional agreement.
As for the margin question between 1.56TB and 8 GPUs, this article is only an order-of-magnitude estimate based on published specs; actual performance should defer to the official deployment documentation and real testing. For most teams, the safer path is to first validate business value with the API, then decide whether to self-host, and to run small-scale tests based on the VRAM, throughput, and license ledgers in this article.
