2.8T Open-Weight Model: Architecture, License, and Deployment Costs

    A Chinese model developer has released the full weights of a 2.8T-parameter MoE model as 1.56TB of downloadable tensor shards. This article breaks down what the model card actually specifies — 104B activated parameters per token, 896 routed plus 2 shared experts, 93 layers, a 1M-token context, and 4-bit microscaling floating-point weights — and turns those numbers into deployment math: how much VRAM eight GPUs need, why the active weight set is ~52GB rather than 104GB, how to size KV cache, and where offloading destroys throughput. It also covers how to read vendor-reported benchmark scores, and why the custom license (hosted under an "other" tag) still permits commercial use for most organizations while requiring a separate agreement above $20M in trailing-12-month revenue for model-as-a-service offerings or 100M monthly active users for commercial products. Includes estimation scripts and a cloud-first deployment path.

    Introduction: The Weights Landed. Don’t Start With the Benchmarks.

    On November 4, 2025, a Chinese model research organization released the full weights of its next-generation ultra-large-scale model, as previously promised. The publisher states this is among the largest open-weight models publicly released as of that date; that claim comes from the publisher’s own materials and has not been independently verified here. The weight repository contains 112 files in total, 96 of them tensor shards, amounting to roughly 1.56TB, downloadable without any application process.

    For engineering teams, three questions matter more than benchmark rankings once weights are out:

    • What is actually inside the model: parameter composition, architectural changes, and quantization format.
    • How the license works: open weights are not open source, and commercial use carries extra conditions.
    • Whether an individual or a team can run it: what is required locally versus in the cloud.

    This article covers architecture, quantization, performance, licensing, and deployment cost for ultra-large open-weight models, and focuses on the practical questions: can it run on eight GPUs, what to do when VRAM runs short, and how to judge commercial use under the license.

    1. Reading a 2.8T Parameter Count: Sparse MoE, Activated Parameters, and Quantized Deployment

    According to the model card, this is a natively multimodal Agent model, produced by continuing training from the previous generation. The key figures:

    Dimension Spec Engineering implication
    Total parameters 2.8T Sets the model’s knowledge capacity, and the on-disk size of the weights
    Activated parameters per token 104B Determines compute per forward step
    Expert structure 896 routed experts + 2 shared experts Each token selects 16 routed experts
    Layers 93 Affects KV cache, communication, and inter-layer scheduling
    Context length 1M tokens Requires sparse attention and KV cache management to keep long-context cost in check
    Vision encoding In-house vision encoder Unified image and text input; natively multimodal

    1.1 Parameters split into a “warehouse” and a “shelf”

    The core of a sparse mixture-of-experts architecture is dividing model parameters into two parts:

    • Expert parameters: idle during normal computation, like inventory sitting in a warehouse.
    • Activated parameters: the portion each token actually touches, like the amount you move per pickup.

    Compute per token can be approximated as:

    C ≈ 2 × activated parameters + attention and other overhead

    Plugging in 104B gives roughly 2.1×10¹¹ FLOPs per token. Generating a 32K-token response requires about 6.7×10¹⁵ FLOPs for the forward pass alone. At 150 TFLOPS for 4-bit weights and an assumed 35% effective utilization, that is roughly 128 seconds. Processing a 1M-token input needs about 66 minutes for the prefill stage alone, before KV cache and attention overhead.

    So the 2.8T total parameter count determines the capability ceiling and the storage bill, while the 104B activated count determines generation speed. Deployment requires both numbers; looking at only one leads to bad estimates.

    1.2 Routing load is the hard part of MoE deployment

    A 16/898 routing ratio is about 1.8%, meaning that on average fewer than 2% of experts in a layer participate in a single token’s computation. That creates two deployment problems:

    • Load imbalance: hot experts can accumulate large numbers of tokens, so under expert parallelism some GPUs stay saturated while others idle.
    • Communication overhead: when experts are spread across GPUs, every layer requires token routing communication, and insufficient inter-GPU bandwidth becomes the bottleneck.

    These mechanisms are largely fixed during training and usually cannot be changed once the weights are published.

    1.3 Sparse attention: buying back compute for long context

    Compared with the previous generation, the main architectural change is that 24 layers of full attention were replaced with sparse attention. Full attention makes the current token attend to the entire history; sparse attention lets it attend only to a subset. The longer the context, the larger the savings. The model card does not disclose the specific sparse pattern, but the direction is clear: a 1M-token context only becomes practically useful once attention cost has been pushed down.

    1.4 Native multimodality: the vision path costs extra VRAM

    The vision encoder has its own parameters, which must stay resident in VRAM. Multimodal inference also splits high-resolution images into tiles that expand into large numbers of visual tokens, which then enter the language model and add to the KV cache. This is easy to leave out when sizing multimodal service capacity.

    2. 4-Bit Quantization Is the Default Form, Not an Option

    The weights use a 4-bit microscaling floating-point format, with 8-bit floating-point activations. Both are the result of quantization-aware training, which simulates low-precision computation during training so the model adapts to quantization error early. That is why 4-bit inference typically loses less accuracy than ordinary post-training quantization.

    Three caveats matter:

    1. The publisher released only this one version. There is no BF16 or FP8 release.
    2. The default form is the quantized form. To run inference, fine-tune, or distill at higher precision, you have to dequantize yourself — and dequantization cannot recover information lost during quantization.
    3. Hardware must support the format. 4-bit microscaling floating point includes block-shared scale factors and is not a simple integer W4A16 scheme, so older hardware may fall back to an emulation path with lower performance.

    By contrast, the KV cache can usually still be stored in FP16, BF16, or FP8 even when weights are 4-bit. KV cache precision is relatively independent and can be configured according to hardware support.

    3. How to Read the Performance Numbers: Vendor-Reported, Not Independently Verified

    The figures below come from the publisher’s self-reported evaluation materials and were not independently reproduced here. The reference models are closed-source flagships; exact versions and measurement conventions follow the publisher’s materials.

    Benchmark This model Reference A (closed) Reference B (closed)
    Terminal command task benchmark (4.0) 65.7% 66.4% 58.2%
    Long-horizon repository-level software engineering benchmark 51.7% 46.3% —
    Science subset of the terminal task benchmark 38.1% 63.3% —
    Hard academic reasoning QA set 39.4% 60.9% —

    The publisher states that the model outperforms Reference A on 16 of 37 benchmarks. A third-party composite capability index is also cited, giving the model a score of 55 and fourth place globally, but that index is quoted by the publisher and has not been independently checked here.

    Three takeaways from these self-reported numbers:

    • Capability tracks the scenario closely: the model is stronger on coding and Agent tasks, weaker on science and some reasoning tasks. It does not lead across the board.
    • The reference frame matters: the accurate description is “one of the strongest open-weight models,” not the strongest among all models.
    • Self-reported scores carry inherent bias: sampling parameters, the Agent scaffolding, prompt templates, and retry counts all move the numbers.

    Before committing, run a small-scale validation on your own business data covering at least: your hardest business questions, multi-turn tool calls, long-context retrieval, and scientific computation or hard reasoning.

    4. Licensing: Open Weights Are Not Open Source, and Commercial Use Needs a Check

    The license deserves separate scrutiny. The repository ships a custom model license from the publisher, and the hosting platform labels it “other” rather than MIT, Apache, or another standard open-source license.

    The license grants free use, copying, modification, distribution, fine-tuning, and commercial use, so individuals and small-to-medium teams generally face few constraints. Organizations running business at scale need to watch two additional conditions:

    1. Model-as-a-service clause: companies offering the model as a service, with cumulative revenue above $20M over the trailing 12 months, must sign a separate agreement with the publisher before commercial use.
    2. Commercial product monthly active users clause: commercial products with more than 100M monthly active users also require a separate agreement.

    The code below only illustrates the decision logic and is not legal advice. Actual commercial use must be based on the license text, its version number, and any supplementary terms.

    1. def can_use_commercially(model, org):
    2. if model.license_type != "other":
    3. return True, "Standard open-source license; follow its terms"
    4. if org.provides_model_as_a_service:
    5. if org.revenue_last_12m_usd > 20_000_000:
    6. return False, "A separate commercial agreement with the publisher is required"
    7. if org.commercial_product_mau > 100_000_000:
    8. return False, "A separate commercial agreement with the publisher is required"
    9. return True, "Evaluate against the commercial terms in the license text"

    Note that the thresholds target organization size, not technical use. Two teams calling the same API can face completely different constraints depending on their scale. Before commercial use, verify the full license text, supplementary terms, trademark usage, model naming, external messaging, and whether derivative models carry the same license logic. If you cannot obtain the complete license text, large-scale commercial use is not advisable.

    5. Can It Run on Eight GPUs? Three Bills: VRAM, Compute, Context

    The short answer: running it locally is not impossible, but “it can be computed” is not the same as “it runs well.”

    5.1 The VRAM bill: eight GPUs is not comfortable

    1.56TB of weights spread across eight GPUs means each card must hold about 195GB. Mainstream per-card VRAM today ranges from roughly 80GB to 141GB, so “eight GPUs as a starting point” implies one of two conditions:

    • Each card has very large VRAM; or
    • Weights are split between VRAM and host memory.

    With 141GB per card across eight cards, total VRAM is about 1128GB. Even reserving only 10% for framework and activation space leaves about 1015GB usable, below the 1.56TB of weights. This is exactly why the default startup script reports that the weights cannot be fully resident: total VRAM does not cover them.

    When VRAM runs out, the inference framework offloads some layers to host memory. The cost is that every generated token requires moving weights from memory, and throughput can drop to a few tokens per second. That profile suits offline batch processing far better than interactive online serving.

    5.2 The compute bill: about 52GB of active 4-bit weights

    At roughly 0.5 bytes per parameter, 4-bit weights put the 104B activated parameters at about 52GB, not 104GB. The theoretical decode ceiling is:

    decode ceiling ≈ effective VRAM bandwidth ÷ 52GB

    Scenario Aggregate bandwidth Ideal ceiling Estimated real throughput
    Single node, 8 GPUs at 3TB/s per card ~24TB/s ~460 tokens/s ~160–280 tokens/s
    2 nodes, 16 GPUs ~48TB/s ~920 tokens/s ~320–550 tokens/s
    8 cards with ~80GB each — — Weights don’t fit; offloading required; throughput may fall to single digits

    Real throughput is usually dragged down by communication overhead, scheduling latency, and expert-parallel efficiency, so capacity planning cannot rely on the theoretical ceiling. And if the weights cannot stay fully resident, the bandwidth estimates above do not apply at all.

    5.3 The context bill: reserve room for the KV cache

    KV cache size can be estimated as:

    KV bytes = layers × KV heads × head_dim × 2 × bytes of precision × tokens

    With 93 layers, 8 KV heads, head_dim of 128, and BF16, each token needs about 0.36MiB:

    • 100K tokens ≈ 35GiB
    • 1M tokens ≈ 350GiB

    Real systems also add image caches, request fragmentation, and headroom. On an eight-GPU box, if the weights already fill VRAM, long-context concurrency drops noticeably.

    5.4 A reusable estimation script

    1. import math
    2. def estimate(num_gpus=8, mem_per_gpu_gb=141.0, bw_per_gpu_tbs=3.0,
    3. weights_tb=1.56, active_params_b=104, seq_len=100_000,
    4. kv_bytes_per_token=380_928):
    5. total_mem = num_gpus * mem_per_gpu_gb
    6. weight_gb = weights_tb * 1000
    7. min_gpus = math.ceil(weight_gb / (mem_per_gpu_gb * 0.9))
    8. fits = weight_gb <= total_mem * 0.9
    9. kv_free_gb = max(total_mem * 0.9 - weight_gb, 0)
    10. read_gb_per_token = active_params_b * 0.5
    11. bandwidth = num_gpus * bw_per_gpu_tbs * 1000
    12. tps_upper = bandwidth / read_gb_per_token
    13. tps_est = tps_upper * 0.45
    14. kv_gb = seq_len * kv_bytes_per_token / (1024 ** 3)
    15. return {
    16. "weights_fully_resident": fits,
    17. "min_gpus_for_weight_residency": min_gpus,
    18. "offloaded_weights_gib": round(max(weight_gb - total_mem * 0.9, 0), 1),
    19. "estimated_tokens_per_second": round(tps_est, 1),
    20. "kv_for_this_length_gib": round(kv_gb, 1),
    21. "vram_available_for_kv_gib": round(kv_free_gb, 1)
    22. }

    6. Choosing a Deployment Path: Validate in the Cloud, Then Decide on Private Deployment

    Scenario Recommended path Cost to watch Latency to watch Compliance to watch
    Want to try the model’s capabilities Official API or a cloud model service Per-token pricing, low entry cost Depends on network and the provider License limits are usually absorbed by the service provider, but still worth confirming
    Data must stay in your own environment Dedicated cloud instances or a self-built multi-node cluster 8-GPU-class hardware and operations cost Affected by interconnect and parallel efficiency Check the custom license and data terms
    Need fine-tuning Private deployment + parameter-efficient fine-tuning Only 4-bit weights exist, so full-parameter training is unrealistic Long training cycles and heavy VRAM pressure Derivative models must be assessed against the same license logic
    Edge or single GPU Not recommended Weights and activated parameters far exceed a single-card budget Inference speed unusable Not applicable

    For most teams, the pragmatic route is cloud first, then local:

    1. Validate with a cloud model service: close the business loop at low cost and complete scenario evaluations on real data. Some cloud providers offer API access to mainstream open-weight models along with private deployment options; check what their actual products support.
    2. Once you confirm the model is not replaceable, consider private deployment: it also requires identity and access control, audit, a gateway, monitoring, and object storage. The cost is not only in the GPUs.

    Operational baseline after launch

    • Gateway layer: multi-model routing and fallback, so the primary model reaching capacity does not become a single point of failure.
    • Object storage: weights, checkpoints, and evaluation datasets, with multi-availability-zone redundancy.
    • Container platform: containerized inference services that scale with load.
    • Monitoring and alerting: time to first token, token throughput, VRAM utilization, expert load distribution, and request failure rate.

    Sample launch flags

    1. inference-server serve \
    2. --model-path /models/open-weight-moe \
    3. --tensor-parallel-size 8 \
    4. --expert-parallel-size 8 \
    5. --max-model-len 32768 \
    6. --kv-cache-dtype auto \
    7. --gpu-memory-utilization 0.9

    Exact flag names depend on the inference engine you use — check its documentation. Three settings matter most when tuning:

    1. Tensor parallel degree: bounded by the number of GPUs per node.
    2. Expert parallel degree: bounded by interconnect bandwidth.
    3. Maximum context length: set it from your actual P95 business length rather than defaulting to the full 1M, otherwise KV cache is reserved at maximum length and concurrency falls.

    7. Conclusion: The Decision Checklist

    The real value of this weight release is not the benchmark ranking but that it puts three key parameters on the table: 2.8T total parameters, 104B activated parameters, and 1.56TB of weights. Each maps to one bill you have to work out:

    • The capability bill: sparse MoE lets “large” and “fast” be measured separately.
    • The compliance bill: under a custom license, large-scale external business must clear revenue and monthly-active-user thresholds, with the license text as the authority.
    • The cost bill: eight GPUs is only the starting point. VRAM, bandwidth, and context management together determine final throughput.

    A three-step approach:

    1. Run a small-scale scenario evaluation first, then decide whether to go deeper.
    2. Review the license terms in parallel and confirm whether you fall under the model-as-a-service or monthly-active-user limits.
    3. Use the estimation script above to budget hardware, then compare self-hosting against cloud.