2.8T-Parameter Open-Weight Model Release: License and 8-GPU Deployment Analysis

    This article examines a 2.8T-parameter open-weight model release from a practical deployment perspective. It covers the model’s sparse MoE architecture, 104B activated parameters per token, native multimodal and agent-oriented design, MXFP4/MXFP8 quantization constraints, memory and KV Cache requirements, and the implications of its custom MIT-style license. It also explains why an 8-GPU setup is the minimum practical starting point for many high-memory configurations and what teams should verify before moving from evaluation to production.

    1. The release: open weights with conditions

    On November 4, a Chinese AI team released the full weights of a flagship model as planned: 2.8T total parameters and roughly 1.56TB of weight files. The team described it as “the strongest downloadable open model currently available.” These weights had previously been offered through online services and APIs. The team promised to publish the full weights within two weeks, and they appeared on schedule in a public repository on a major model-weight hosting community. No application was required for download. The repository contains 112 files, including 96 safetensors shards.

    This is the largest downloadable open-weight model disclosed so far, roughly an order of magnitude larger than any previous release. However, being downloadable does not mean it is free to use in all cases, and it certainly does not mean it can run locally without significant planning. This article focuses on three questions after the release: what is inside the model, how the license works, and whether ordinary teams can deploy it.

    2. Model structure: what 2.8T total parameters and 104B active parameters mean

    2.1 Only a small subset of parameters is activated per token

    According to the official model card, this is a native multimodal Agent model continued from the previous generation base model. Key structural parameters are:

    Dimension Value
    Total parameters 2.8T (trillion-scale)
    Activated parameters per token 104B
    Experts 896 routed experts + 2 shared experts
    Experts selected per token 16 routed experts
    Layers 93
    Context length 1 million tokens
    Vision encoder Custom vision tower, supports image input
    Weight / activation precision 4-bit micro-scaled floating point / 8-bit micro-scaled floating point

    This is a typical fine-grained sparse Mixture of Experts (MoE) design. Its core characteristic is decoupling total capacity from per-token computation: memory must accommodate 2.8T parameters, while compute mainly handles 104B parameters. This asymmetry drives the deployment conclusions: memory is the hard constraint, while compute is relatively easier to satisfy.

    1. # Example: routing and expert merging in a fine-grained MoE layer
    2. import torch
    3. def moe_forward(x, router, experts, shared_experts, top_k=16):
    4. # x: [tokens, hidden]
    5. logits = router(x) # [tokens, 896]
    6. w, idx = logits.softmax(dim=-1).topk(top_k, dim=-1) # select 16 experts per token
    7. w = w / w.sum(dim=-1, keepdim=True) # normalize weights
    8. y = shared_experts(x) # 2 shared experts processed by all tokens
    9. for t in range(x.shape[0]):
    10. for weight, e in zip(w[t], idx[t]):
    11. y[t] += weight * experts[int(e)](x[t])
    12. return y

    2.2 One structural change from the previous generation

    At the structural level, the main change from the previous-generation base model is the replacement of full attention in 24 layers with sparse attention (DSA). This is not done to improve benchmark scores. It is intended for long-context use. Attention cost grows with the square of sequence length. At million-token context lengths, dense full attention across every layer would make attention itself more expensive than the FFN. Sparse attention brings this cost back into a manageable range. The tradeoff is that sparse attention depends more heavily on high-quality inference kernels. If the inference engine does not support the required pattern, it may fall back to a dense path, and long-context costs can rise significantly.

    2.3 Engineering implications of multimodal and Agent capabilities

    Native multimodality means the vision encoder and language backbone work together in the same forward pass. Deployment must load the vision tower in addition to the language weights. Images are encoded into many visual tokens that enter the context and consume KV Cache budget. Agent workloads imply very long multi-turn conversations and frequent tool-call concatenation. These scenarios place much higher demands on time-to-first-token (TTFT) and long-session stability than ordinary chat.

    3. Why only the 4-bit version is available

    The official release includes only MXFP4 weights with MXFP8 activations. There is no BF16 version and no conventional FP8 version. The reason is that the model was trained with quantization-aware training (QAT): it converged under low-precision noise near the end of training, and the weight distribution has already been flattened into a range that the 4-bit micro-scaled format can represent.

    The benefits are direct: at the same capacity, file size drops to about one quarter, more experts can fit in a single device’s memory, and memory-bandwidth pressure is significantly reduced.

    However, this introduces three side effects that must be evaluated in advance:

    1. Fine-tuning is difficult. Low-precision weights are effectively the final form. Full-parameter or large-scale fine-tuning on top of them usually requires dequantization to higher precision, which sharply increases memory requirements and implementation complexity. In practice, parameter-efficient methods such as low-rank adaptation (LoRA) are more feasible, or small-step tuning focused on activation precision.
    2. Precision-sensitive tasks may lose accuracy. QAT preserves most capabilities, but high-entropy numerical reasoning, long-chain logic, and scientific computing tasks may still amplify quantization errors.
    3. Dequantization overhead may occur. Micro-scaled formats require reading scale factors by block. If the inference kernel does not natively support 4-bit compute, it may dequantize to 8-bit or 16-bit at runtime, reducing throughput gains. Before selection, verify whether the target inference framework natively supports the format and sparse attention.

    4. Memory math: 1.56TB and the “8-GPU minimum”

    4.1 How much memory do the weights require?

    A rough calculation: 2.8T parameters × 0.5 bytes per parameter ≈ 1.4TB. Adding the vision encoder, token embeddings, runtime buffers, and format-alignment overhead, a final file size of about 1.56TB is consistent.

    Spread across GPUs:

    • 1.56TB ÷ 8 GPUs ≈ 195GB per GPU — so the implicit premise of an “8-GPU minimum” is access to the latest high-memory accelerator cards with memory capacity near the 200GB level.
    • With 80GB or 141GB cards, the weights alone would require at least 12 to 16 cards to remain fully resident.
    • If memory is insufficient, the system can still offload some expert layers to host memory or local NVMe storage. However, every decoding step would need to move tens of gigabytes over PCIe, and single-request latency would degrade to a level unsuitable for interactive use.

    Conclusion: an 8-GPU configuration is the minimum threshold for this technical route, not a recommended production configuration.

    4.2 KV Cache is the real bottleneck for long context

    Weights are a fixed cost. KV Cache is a cost that grows linearly with concurrency and context length. A rough formula is:

    1. def kv_cache_bytes(layers, kv_heads, head_dim, tokens, bytes_per_elem, batch=1):
    2. # Factor 2 accounts for K and V separately
    3. return 2 * layers * kv_heads * head_dim * tokens * batch * bytes_per_elem

    Using 93 layers and a 1-million-token context, assuming a small number of KV heads per GPU, head dimension 128, and 8-bit KV storage, the KV Cache for a single million-token context can reach tens to hundreds of gigabytes (the exact value depends on the head count and head dimension disclosed in the model card). This means:

    • The “nominal capability” of 1 million tokens and the number of serviceable concurrent requests are two different things. Long-context workloads will generally support only low concurrency.
    • Chunked prefill, prefix-cache reuse, KV quantization, and paged memory management should be enabled; otherwise KV Cache can quickly exhaust memory.
    • Multimodal inputs require additional visual-token accounting. More images mean less available context.

    4.3 Intuitive understanding of compute and latency

    Decoding is memory-bandwidth bound. For every generated token, the system must read at least 104B activated parameters in 4-bit form, roughly 52GB of reads. With the aggregate bandwidth of 8 GPUs, the theoretical single-request upper bound may be in the hundreds of tokens per second range, but communication overhead, routing imbalance, and sparse-attention fallback can reduce real-world performance. The right way to improve throughput is to increase batch size: weight reads are amortized across multiple requests, and throughput can rise nearly linearly with batch size until the workload becomes compute-bound.

    Prefilling is the opposite: it is compute-bound. Pushing 1 million tokens in at once, estimating only from activated parameters, requires approximately 2 × 104B × 1e6 ≈ 2 × 10^17 floating-point operations, or about 200 PFLOPs. This is several orders of magnitude larger than a single decode step. Therefore, for long-document workloads, the first optimization target is always TTFT, not tokens/s.

    5. License: open weights do not mean open source

    This is the most important point to consider separately. The repository uses a custom license, labeled as license: other on the hosting platform, rather than a standard open-source license such as MIT or Apache. The license text is word-for-word identical to the previous-generation model, with only the name changed.

    The main grant is MIT-style: free use, reproduction, modification, distribution, fine-tuning, and commercial use. However, there are two gates, and both apply only to large-scale external business use:

    Trigger condition Specific requirement
    Offered as model-as-a-service (MaaS) with cumulative revenue exceeding $20 million over a trailing 12-month period A separate agreement must be signed with the rights holder before commercial use
    Commercial product monthly active users exceed the scale threshold defined in the license text A separate license or agreement must be obtained

    (The specific value for the second item is defined in the license text and is not speculated here.)

    Engineering and legal teams should focus on four points:

    1. Internal use is largely unrestricted. Internal R&D, offline evaluation, and non-scale product integration fall within the MIT-style grant.
    2. Scale thresholds are hard red lines. Teams building MaaS resale or consumer products with tens of millions of MAU must monitor trailing-12-month revenue and MAU as part of compliance, not make a one-time judgment.
    3. License and model card are separate. Technical documentation does not cover legal terms. Legal review must read the license text and retain version snapshots, because license terms may change with model versions.
    4. Redistribution is transitive. If you distribute fine-tuned weights, you must include the license text and modification notes.
    1. # Example: enforce license gates before release
    2. GATES = [
    3. {"name": "maas_revenue", "limit_usd": 20_000_000, "window_months": 12},
    4. {"name": "product_mau", "limit": None}, # fill in the value from the license text
    5. ]
    6. def release_check(is_maas, trailing_12m_revenue_usd, product_mau, license_id):
    7. if license_id != "other":
    8. return {"ok": True, "note": "standard open-source license; follow its original terms"}
    9. if is_maas and trailing_12m_revenue_usd > GATES[0]["limit_usd"]:
    10. return {"ok": False, "reason": "MaaS revenue exceeds threshold; a separate commercial agreement is required"}
    11. limit = GATES[1]["limit"]
    12. if limit is not None and product_mau > limit:
    13. return {"ok": False, "reason": "product MAU exceeds threshold; a separate license is required"}
    14. return {"ok": True, "note": "within the open grant"}

    6. How to read vendor-reported benchmark tables

    The vendor’s horizontal comparisons (all vendor-reported) show a clear specialization pattern:

    • Terminal / command-line operation benchmark: 65.7%, slightly below the same-generation closed-source flagship’s 66.4%, but clearly above another closed-source flagship’s 58.2%.
    • Long-horizon software engineering benchmark: 51.7%, above the comparison model’s 46.3%.
    • Hard science question benchmark: 38.1%, compared with 63.3% for the comparison model.
    • Hard academic QA benchmark: 39.4%, compared with 60.9% for the comparison model.

    The vendor claims leadership in 16 of 37 benchmarks. A third-party independent evaluation summary gives an intelligence index of about 55, ranking fourth globally. The conclusion after reviewing the items is clear: strong in coding and Agent tool use; weak in science and deep-chain reasoning.

    The right way to read these tables:

    • Treat vendor-reported data only as directional signals, not selection criteria.
    • Re-test using private evaluation sets based on your own tasks, focusing on the three to five request types that are truly frequent in your business.
    • Pay attention to metric definition differences (whether tools are provided, whether the task is multi-turn, whether pass@1 is used); cross-leaderboard comparisons are almost always unfair.
    • Put cost, latency, context length, and accuracy in the same table. The value of being fourth depends on whether your scenario falls within its strengths.

    7. Deployment decision checklist

    Scenarios where this model may be appropriate: local or private deployment, Agent and coding tasks as the main workload, access to 8+ GPUs with large memory, and revenue/MAU below the license red lines.

    Scenarios to avoid: insufficient memory without tolerance for offload-induced latency; workloads centered on scientific computing or rigorous mathematical reasoning; frequent full-parameter fine-tuning; consumer products that must reach tens of millions of MAU in the short term.

    Cloud or self-hosting: purchasing a full 8-GPU large-memory accelerator server and supporting data-center infrastructure is a large one-time investment, while model iteration is extremely fast. For most teams, it is more prudent to first rent large-memory GPU instances in the cloud to validate performance and concurrency curves, then decide whether self-hosting is more cost-effective. Baidu AI Cloud GPU cloud servers can serve as a compute foundation. When combined with container platforms for elastic inference service scaling, object storage for distributing TB-scale weights, and logging/monitoring/alerting for TTFT, throughput, and memory utilization, this can form a complete inference operations loop. Before self-hosting, run a small-scale stress test and measure three numbers: whether a single GPU can hold all required weights, how many concurrent requests KV Cache can support, and whether long-context TTFT is acceptable. Only then should capacity planning proceed.

    Inference service architecture recommendations:

    1. Use hybrid tensor parallelism and expert parallelism, choosing the split granularity based on single-GPU memory, while avoiding expert-distribution skew that causes some GPUs to fill up first.
    2. Separate prefill and decode deployment: long-document requests go to a prefill pool, interactive requests go to a decode pool, and the two do not block each other.
    3. Reuse prefix caches for system prompts and tool definitions; this yields the largest benefit in Agent scenarios.
    4. Build fallback paths and tiering: difficult requests go to the large model, while high-frequency long-tail requests go to a smaller or distilled model, with routing controlling overall cost.

    8. Summary

    The real value of this open-weight release is not that “another large model has been released.” It is that it puts three practical problems of sparse MoE on the table at once: 2.8T parameters create a 1.56TB memory hard floor; 4-bit QAT makes the model deployment-friendly as a final form rather than a starting point for fine-tuning; and the custom license means there is a compliance gate between “can download” and “can use commercially.” Understand these three points before deciding whether to prepare eight GPUs.