Open-Weight 2.8T MoE Model: Deployment and License Explained

    An in-depth look at deploying and licensing an open-weight 2.8T-parameter MoE model with 104B active parameters. It explains why open weights differ from open source, why capacity (2.8T) and compute (104B active) drive different bottlenecks, what an MXFP4/MXFP8 QAT-only release means in practice, how to read vendor-reported benchmarks, and how to budget VRAM for an 8-card minimum setup. It also covers deployment essentials such as parallelism, expert-routing long tails, multimodality, and cold start, decodes the custom license's revenue and MAU thresholds, and offers a decision checklist for when self-hosting makes sense.

    1. Open Weights Is Not Open Source: Define the Name and Boundaries First

    A China-based team released a native multimodal flagship model as an online service in late October and committed to opening the full weights by November 4. The repository was created on November 2; it remained closed that morning and was officially opened that evening. It contains 96 weight shards totaling about 1.56TB. The weights can also be obtained from model hosting platforms, but it is recommended to download from the official repository and verify the checksums, since third-party re-hosted copies carry a risk of tampering.

    One clarification first: because the original materials did not provide a verifiable model name, version number, or the official name of the license, this article does not assign names on its own. The relevant parameters, license terms, and download sources should all be based on the final disclosure in the official repository.

    The concepts also need to be aligned here: open weights is not the same as open source. Open weights means users can load, run inference on, fine-tune, and redistribute the trained parameter files, but it usually does not include the complete training code, data mixture, cleaning pipeline, distributed training framework, or reward model. So “can it be reproduced” and “can it be used” are two different questions. For enterprises, what really matters is: can it be deployed privately, can it be used commercially, and can it be fine-tuned for a domain?

    2. 2.8T Parameters Is Not Compute; 104B Active Is the Per-Token Compute

    According to the model description, it is a native multimodal Agent model continued-training from the previous-generation base. Key architectural details:

    • Total parameters: 2.8T; active per token: 104B — a sparse mixture-of-experts (MoE) structure
    • 896 experts total, routing 16 per token, plus 2 always-on shared experts
    • 93 layers, of which 24 use sparse-attention-style mechanisms and the rest keep full attention
    • Context length up to 1 million tokens; the vision encoder is a self-developed V2 that supports image input

    These numbers explain the “big but fast” paradox: 2.8T is capacity, 104B is compute. Capacity determines memory (VRAM) and bandwidth usage; compute determines the per-token compute cost. As a result, the inference bottleneck for this kind of model is usually not floating-point compute but memory capacity, memory bandwidth, expert-routing communication, and long-context KV Cache management.

    A common misconception needs correcting: sparse attention does not necessarily mean the KV Cache becomes smaller. It may reduce the attention compute for some layers and may lower compute and memory-access pressure through a sparser access pattern, but whether it actually reduces KV memory depends on whether the inference framework implements KV compression, eviction, paged caching, sparse indexing, or context compression. Without a matching implementation, long context will still consume significant memory.

    3. Only a 4-Bit Version Is Released: QAT Works, but It Is Not Lossless and Does Not Mean Hardware Natively Supports It

    The model weights are MXFP4 and the activations are MXFP8, a product of quantization-aware training (QAT). QAT simulates low-precision numerical behavior during training, so it typically outperforms post-training quantization (PTQ), but it is not lossless. Especially on long-chain reasoning tasks such as multi-step math, strict-format output, and precise retrieval over long documents, it needs to be validated separately.

    More importantly, the vendor only provides this single 4-bit version, with no BF16 or FP8 intermediate. This means: you cannot quantize further to trade for higher concurrency; the fidelity ceiling is locked to this version; and if a team wants full-parameter fine-tuning, continued pre-training, or distillation, there is no high-precision starting point — only dequantization approximation, and dequantization cannot recover the discarded information.

    So compatibility checks are mandatory before deployment: confirm whether mainstream inference frameworks such as vLLM and SGLang support MXFP4 and MXFP8, and which minimum version is required; confirm whether the accelerator cards natively support these low-precision formats. Blackwell-class hardware, for example, usually has better native support; other architectures may need dequantization, a fallback to BF16, or a simulation path. If there is no native support, the earlier judgment that “the bottleneck is mainly memory bandwidth rather than compute” may change, and throughput, latency, and deployment cost all need to be re-evaluated.

    4. How to Read Vendor Self-Reported Scores: Look at the Weaknesses First, Then Verify the Scaffold and Third Parties

    The vendor’s published comparison table is a self-reported result, roughly as follows:

    Evaluation area This model Closed-source flagship Note
    Terminal-command coding 65.7% 66.4% Roughly even, slightly lower
    Long-horizon software engineering 51.7% 46.3% Relatively ahead
    Science reasoning 38.1% 63.3% Clear gap
    Multi-discipline hard Q&A 39.4% 60.9% Clear gap

    The vendor also claims that it beats the comparison model on 16 of 37 benchmarks, and cites a third-party evaluation giving an intelligence index of 55, ranked fourth globally. Because the original text did not disclose the complete benchmark names, the third-party report link, or the evaluation date, this information can only serve as a lead and cannot be a procurement basis.

    Four principles are recommended when reading leaderboards:

    1. Distinguish self-reported from third-party. Vendor-reported prompt templates, sampling parameters, retry counts, and best-of-n strategies all affect the score. Third-party evaluations are usually more conservative; the larger the gap, the more you should defer to the third party.
    2. Look at weaknesses, not the average. If your business relies on scientific computing, strict logical reasoning, or highly reliable format output, a weakness number like 38.1% is more relevant than a composite average.
    3. Re-run on your own business data. It is recommended to use 200 to 500 real samples for manual blind evaluation, with a fixed evaluation set and scoring script, so it can be re-run across version iterations.
    4. Agent-type scores depend heavily on the scaffold. Terminal and software-engineering evaluations are strongly tied to file-editing approach, command-execution sandbox, failure retries, and rollback strategies. The same model with different scaffolds can differ by double digits.

    5. 8 Cards to Start: Do the Memory Math Yourself Before Talking About Production

    First, correct the unit conversion: 1.56TB is decimal, roughly equal to 1.42TiB, where 1TiB = 1024^4 bytes. Converted to GiB, that is about 1453GiB. At 8 cards with 192GiB each, total memory is 1536GiB, leaving a theoretical margin of about 83GiB. It looks like there is still about 80GB of room, but this does not mean you can easily run long context and high concurrency.

    Because framework runtime overhead, CUDA communication buffers, activations, and the KV Cache all consume memory. If runtime overhead is calculated at 5% per card, the 8 cards deduct about 77GiB, leaving only single-digit GiB of usable space; at 8%, there is almost no KV budget left. If the framework implementation is not efficient enough, the actual margin will be even smaller.

    An illustrative capacity calculation:

    1. weight_gib = 1.56e12 / 1024**3 # about 1453GiB
    2. # can also be written as weight_gib = 1.42 * 1024
    3. cards = 8
    4. per_card_gib = 192
    5. runtime_overhead_gib = 0.05 * per_card_gib # empirical value, adjust from 3%~8%
    6. kv_per_token = estimate_kv(L, d, layers_full_attn, layers_sparse_attn)
    7. budget_kv = cards * per_card_gib - weight_gib - cards * runtime_overhead_gib
    8. max_total_tokens = budget_kv / kv_per_token
    9. if budget_kv <= 0:
    10. raise "cannot even serve the minimum context, must add cards or adjust the deployment plan"

    The estimate_kv function here needs the team to estimate based on the actual model configuration, number of layers, head dimension, context length, paging strategy, and sparse-attention implementation. If budget_kv is less than or equal to 0, the current number of cards cannot even support basic service and you must add cards.

    So “8 cards to start” should be understood as the minimum configuration that can load and run, not a production configuration that can serve. To support 100k-token-level long documents, 20+ concurrency, or high-resolution image input, you usually need a memory pool at the 12-to-16-card level.

    6. Deployment Engineering Essentials: Parallelism, Long Tails, Multimodality, and Cold Start

    First, prioritize tensor parallelism (TP). TP splits each layer’s weights across multiple cards, and every card participates in every layer’s computation; communication volume is large, so it should be deployed over high-speed interconnect within the same node. When crossing nodes, expert parallelism (EP) is usually more cost-effective than wide-span TP, because MoE’s All-to-All communication is bandwidth-sensitive but relatively tolerant of latency.

    Second, expert routing causes load imbalance. Of 896 experts, each token selects 16; different tokens activate different experts, so actual memory and communication volume have a long tail. Stress tests cannot use only short prompts; they must include very long documents, very long codebases, multi-turn conversations, and multimodal input.

    Third, multimodality amplifies prefill cost. The vision encoder and high-resolution images push up the number of visual tokens, and time to first token (TTFT) rises noticeably. In capacity planning, image-resolution policy, input-size limits, and preprocessing overhead must all be decided together.

    Fourth, do not ignore weight loading time. Pulling and loading 1.42TiB of weights from object storage can make cold start take minutes to tens of minutes. Production environments should use local high-performance storage caching, along with warm instances, readiness probes, and canary releases, to avoid long unavailability during scale-out.

    7. License: Beyond a Permissive Core, There Are Two Scale Thresholds

    The model repository carries a custom license, marked as “other” on the hosting platform, not a standard open-source license. The original text did not disclose the official name or full link of the license; readers should refer to the license file in the official repository. This article is only a point-by-point interpretation and does not constitute legal advice; the terms are subject to the original official agreement, and consult legal counsel for specific applicability.

    From the existing description, the license structure is close to a “permissive core + large-scale commercialization exception”: the core is close to an MIT-style grant, allowing free use, copying, modification, distribution, fine-tuning, and commercial use; but there are two situations that may trigger separate authorization:

    1. Providing the model “as a service” externally, with cumulative revenue over $20 million over 12 consecutive months — a separate agreement must be signed before commercial use.
    2. Using it in a commercial product, with monthly active users exceeding an officially agreed threshold, requires separate authorization. The specific MAU number was not disclosed in the original text and should be based on the official agreement.

    To judge whether your enterprise is restricted, you can answer three questions:

    1. Is the delivery form private deployment, or providing inference service externally? Weights landing in the customer’s environment usually does not equal “model as a service,” but whether it constitutes one still needs to be judged in light of the customer contract, service model, and vendor interpretation.
    2. Has the business line’s relevant revenue over the past 12 months crossed the threshold? Note that it is cumulative revenue, not profit and not single-product revenue.
    3. Has the commercial product’s MAU crossed the threshold? Internal tools and offline batch processing usually do not trigger it, but it still needs to be judged against the license text.

    The compliance recommendation is: legal and platform teams should complete the license comparison at the project-initiation stage, not patch it before launch. It is recommended to record the license version number, checksum, download time, and source repository; re-compare the license text on subsequent version iterations; and, if the revenue or MAU threshold may be triggered, reserve a business-communication window in advance.

    8. Deployment Decision Checklist: Which Scenarios Suit Self-Hosting and Which Do Not

    Scenarios suitable for self-hosting include: strict data-compliance requirements, such as finance, healthcare, and government; stable and predictable traffic; a need for domain fine-tuning and deep customization; and long context as a hard requirement, such as whole-codebase, long-contract, or long-medical-record analysis.

    Scenarios not suitable for self-hosting include: highly volatile traffic with sparse peaks; teams lacking distributed-inference operations experience; tasks that are essentially short Q&A, where 30B-to-100B-class models usually offer better cost-effectiveness; and needs for the strongest scientific reasoning, where it is still recommended to prioritize evaluating closed-source flagships or hybrid invocation.

    If you decide to self-host, it is recommended to proceed in the following order:

    1. Validate quality at small scale. Get it running with a minimum viable configuration, do manual blind evaluation on a real business set, compare with the existing solution, and only scale up after confirming quality meets the bar.
    2. Do capacity and performance stress tests. Test in tiers — short context with high concurrency, long context with low concurrency, and multimodal mixed load — and record TTFT, per-token latency, throughput, and memory watermark.
    3. Build a cost model. Amortize GPU hours, storage, network, and operations labor into cost per million tokens, compare with API pricing, and find the break-even traffic.
    4. Build engineering guardrails. Fix the evaluation set and regression scripts, add structured validation and retries to model output, set tiered memory and latency alerts, and prepare canary and rollback paths for version upgrades.
    5. Define a fallback plan. When the flagship model is unavailable, there should be a secondary model or API as a fallback to avoid a single point of dependency.

    9. Conclusion: Look at the Memory Math and the License First, Then the Leaderboard Scores

    The real signal of this weight release is not that “the strongest open model can be downloaded,” but that the focus of open-weight models is shifting from “can you get it” to “can you afford to use it once you have it.” 2.8T capacity, 104B active, 896-choose-16 sparse routing, 24 sparse-attention layers, and 4-bit quantization-aware training — this combination pushes per-token compute into an acceptable range, but at the cost of a higher memory threshold, a single precision version, and limited deep fine-tuning.

    The license, with its two thresholds on revenue and MAU, isolates ultra-large-scale commercialization separately. For engineering teams, when evaluating this kind of model, look at the memory math, hardware/framework compatibility, and license terms first, then the leaderboard scores. The former determines whether it can enter your data center and your compliance process; the latter only determines how much better it is than the existing solution in the scenarios you have already gotten running.