Ultra-Large Open-Weight MoE Model: License Terms and Deployment Thresholds Explained
This article breaks down a 2.8-trillion-parameter open-weight multimodal MoE model release from four angles—architecture, benchmarks, licensing, and deployment—to answer three practical questions: what is inside the model, what the license allows, and whether a regular user can run it. It explains the 2.8T total / 104B activated parameter split, why vendor-reported benchmarks must be read with methodology in mind, how the custom MIT-style license differs from standard open source, and what it actually takes to serve 1.56TB of weights locally (starting from 8 high-end GPUs). It also covers a cloud-based alternative path and guidance on choosing between self-hosting and a hosted API.
On November 4, a Chinese AI team published the full weights of its new flagship multimodal model on a major model-hosting platform, following through on a commitment made when the API and web versions went live. The weights can be downloaded without an application. The repository contains 112 files, 96 of which are weight shards totaling roughly 1.56TB.
These numbers carry three immediate implications. First, it is one of the largest openly downloadable open-weight models currently available. Second, “open weights” does not mean “easy to run”—an 8-GPU node is only the starting point. Third, the repository ships under a custom license whose terms must be checked clause by clause.
The sections below cover the model architecture, benchmark results, license, and deployment practice, focused on three questions:
- What is inside?
- What does the license allow?
- Can a regular user run it?
1. Model Architecture: Where the 2.8 Trillion Parameters Come From
According to the model card, this is a native multimodal agent model continued from the previous generation. It ships with a proprietary vision encoder and can read images directly, and it is optimized for multi-step tool calling and task execution.
Core specifications
- Total parameters: ~2.8 trillion
- Activated parameters per token: ~104 billion
- Architecture: sparse Mixture-of-Experts (MoE), 896 routed experts with 16 selected per token, plus 2 shared experts
- Layers: 93
- Context length: 1 million tokens
- Vision encoder: proprietary vision module with image input support
- Parameter precision: 4-bit weights, 8-bit activations—the output of quantization-aware training
“Total parameters” and “activated parameters”: two numbers not to confuse
The “2.8T total, 104B activated” combination is the main path of modern large-model scaling: the Mixture-of-Experts architecture.
You can think of each expert as a feed-forward subnetwork. At inference time, a router selects a few experts to compute each token, while the shared experts stay active across all tokens and carry general-purpose capability. Total parameters determine the model’s knowledge capacity; activated parameters determine the compute per token. Once these two are decoupled, the model can be scaled up without increasing per-token compute.
# Simplified sketch of an MoE FFN layer to illustrate the compute structuredef moe_ffn(x, router, experts, shared_experts):# Select 16 of the 896 experts per tokentopk_ids, topk_w = router(x) # shape: [tokens, 16]y = sum(topk_w[..., i][..., None] * experts[topk_ids[..., i]](x)for i in range(topk_ids.shape[-1]))# 2 shared experts run on every tokeny = y + shared_experts(x)return y
There are costs, too:
- Parameter count and VRAM: weight size grows close to linearly, which directly raises the VRAM floor.
- Communication overhead: when experts are spread across devices, each layer requires two rounds of cross-device data exchange—first dispatching tokens along the token dimension to the devices holding their experts, then gathering the results afterward. This is the most common communication bottleneck in MoE inference.
Long context and sparse attention
A 1-million-token context window means the model can hold an entire codebase, a whole book, or long-span multimodal material in a single pass. Compared with the previous generation, there is only one structural change: sparse attention (DSA) replaces full attention in 24 layers.
Attention compute grows quadratically with sequence length, so beyond 128K the compute cost of long-context inference climbs quickly. Sparse attention lets each token attend only to a filtered subset of the history, pushing compute back to near-linear at long sequence lengths. As a trade-off, it needs extra infrastructure to maintain the selected content, shifting some of the compute into managing the attention mechanism.
Quantization-aware training: 4-bit out of the box
4-bit weights and 8-bit activations are a direct product of quantization-aware training. The precision loss from low precision is folded into the optimization objective during training and post-training, so the released weights are natively suited to low-precision inference without a separate “train, then compress” step.
Two things to be clear about:
- The team released only this single version and provided no higher-precision weights such as BF16 or FP8. Researchers who want to do precision comparisons or fine-grained fine-tuning have no ready-made high-precision baseline.
- 4-bit weights are a necessary condition, not a sufficient condition for “everyone can run it.” Weights on the order of 1.4TB still do not fit on a single machine.
2. Reading the Benchmarks: Vendor-Reported Numbers Need a Methodology Lens
All figures on the model card are vendor-reported:
- Coding tasks: 65.7% on a terminal-operation benchmark, slightly below a flagship closed-source model at 66.4% and above another closed-source model at 58.2%.
- Code repair: 51.7% on a multi-file codebase repair benchmark, above the competitor’s 46.3%.
- Scientific reasoning: only 38.1% on a science-task benchmark, versus the competitor’s 63.3%.
- General reasoning: 39.4% on a high-difficulty general knowledge and reasoning benchmark, versus the competitor’s 60.9%.
- Third-party evaluation: an independent evaluation body gave an intelligence index of 55, ranking it fourth globally.
The takeaway: of 37 benchmarks, 16 beat the reference model—a mixed record. Its strengths are concentrated in coding and agent-execution tasks; its weaknesses are in science and some reasoning tasks.
Three common traps with this kind of data:
- Undisclosed methodology: few-shot settings, whether tools are used, inference budget, and pass@1 vs. pass@k all significantly affect results. Confirm the methodology is consistent before comparing across models.
- Benchmark contamination: public test items may already be in the training data. Vendor-run evaluations lack third-party reproduction, so their credibility should be discounted.
- Averages hide structural differences: when a top-1 result and a last-place result coexist, the average is meaningless. Choose by comparing item by item against your own workload distribution, not the total score.
3. License: “Open Weights” Does Not Mean “Open Source”
This is the point most worth scrutinizing on its own in this release.
The repository ships under the vendor’s custom Kimi K3.1 Pro License, tagged license:other on the hosting platform, which is not a standard open-source license like MIT or Apache. It is word-for-word identical to the previous generation’s license, with only the name changed.
On the positive side: the core terms are an MIT-style grant, allowing free use, copying, modification, distribution, fine-tuning, and commercial use. Compared with closed-source models that expose only an API, enterprises get full self-hosting and customization capability, data can stay in their own data center, and long-term costs are controllable.
What needs checking: a vendor’s custom license usually attaches specific clauses on top of an MIT-style core. Common types include:
- Large-scale commercial use requires separate authorization;
- Production use requires prominent attribution of the model source;
- Derivative models must carry this license or a specified license;
- Restrictions on competing with the entity named in the license;
- Disclaimers and limitation of liability.
The “no competing with the entity named in the license” clause happens to land squarely on the many cases where a product builds a service directly on top of a flagship model.
A suggested self-check order:
- Read the full License text and confirm whether there are revenue or active-user thresholds;
- Determine whether your business model falls under the competition clause;
- Clarify the naming and redistribution obligations for derivative models;
- Have legal review before commercial use.
In addition, “open weights” only means the model files are downloadable. Training data, data pipelines, distributed training code, and evaluation scripts typically are not released with the weights, so outsiders cannot reproduce the full training process and can only run inference and fine-tuning on the given weights. The correct way to understand open weights, then, is “self-hostable, modifiable, distributable,” not “open-source reproducible.”
4. Local Deployment: How the 8-GPU Figure Is Derived
Once the weights are open, the first question is “Can my computer run it?” The answer is no, and the threshold is higher than 8 GPUs.
Step 1: VRAM. A rough estimating formula:
VRAM per card ≈ (total weights + KV cache + framework overhead) / number of cards
- 1.56TB divided by 8 is about 195GB.
- At 2.8 trillion × 4-bit, the raw weights are about 1.4TB, which divided by 8 is about 175GB.
- The two differ by about 160GB, likely mainly due to precision differences in the non-MoE parts. The text backbone is 4-bit, while attention, embeddings, the vision encoder, and shared experts may be kept at 8-bit or higher. Take the actual configuration as authoritative.
Put 175GB into the inequality “number of cards × per-card VRAM ≥ weights + KV cache + overhead” and the conclusion is direct: 8 GPUs only work when built on high-end cards with hundreds of GB of VRAM. For example, a machine with 141GB-class cards and about 1.1TB of VRAM per node cannot even hold the weights. Ordinary accelerators with 96GB or 80GB of VRAM need 16 or more cards to reach 1.4TB. Even at 16 cards, the inter-card interconnect bandwidth of these cards is typically lower than that of dedicated high-speed interconnects, and the all-to-all communication of MoE becomes the main bottleneck.
Step 2: Bandwidth determines decode speed.
MoE decoding is a memory-access-intensive workload. Each token must read all activated expert parameters, so the theoretical ceiling is roughly:
decode speed ≈ memory bandwidth / bytes read per token
- Activated 104 billion parameters at 4-bit means reading at least about 52GB per token.
- With 8 cards’ combined memory bandwidth on the order of 20TB/s, the theoretical ceiling is about 384 tokens/s.
- After subtracting communication, scheduling, and kernel overhead, real-world throughput is typically a fraction of the theoretical value.
Step 3: Long context drives up the KV cache.
A 1-million-token context means the KV cache can reach hundreds of GB:
KV cache bytes ≈ 2 (K/V) × layers × attention heads × head_dim × sequence length × bytes per element
In extreme cases, the KV cache can consume all available VRAM.
Conclusion: the sensible way to deploy this model is enterprise self-build or renting a high-end GPU cluster; running it locally is nearly infeasible for individuals.
Deployment checklist: from download to stable serving
1. Resource preparation
- Download with 1.5TB+ of free space, leaving margin.
- Use multi-threaded resumable downloads and verify each file’s sha256.
- Confirm the weight shards, config, tokenizer, and vision encoder files are complete. For high-compression-ratio weights, any data error can cause precision loss or even garbled output.
2. Parallelism strategy
- Large-scale tensor parallelism (TP) shards each layer’s weights across all cards and must sit within a high-speed interconnect domain.
- Expert parallelism (EP) splits by expert and can significantly reduce all-to-all communication.
- Pipeline parallelism (PP) splits by layer and suits cross-node scaling.
- The number of attention heads must be divisible by the TP degree; otherwise the sharding scheme needs adjustment.
3. Precision and KV cache
- Use 4-bit inference that matches the weights to avoid re-quantization.
- The KV cache can be 8-bit or 4-bit, with small quality loss and large VRAM savings.
- Use paged KV cache for long contexts to reduce fragmentation.
4. Batching and observability
- Enable continuous batching and prefix caching.
- Monitor the inter-card communication share; if it exceeds 30% of total time, communication has become the bottleneck.
- Watch VRAM fragmentation, throughput, P99 latency, and GPU utilization.
5. Compliance
- Configure content safety filtering, access auditing, and log retention per your business requirements.
- Within compliance constraints, inference requests can stay inside your own environment.
Example configuration (illustrative, not the real parameters of any tool)
# Single-node multi-GPU MoE inference launch parameters (illustrative; follow your framework's docs)TP=8 # Tensor parallel: shard weights across 8 GPUsEP=8 # Expert parallel: split by expert to reduce all-to-all trafficQUANT=mxfp4 # Match the weight precision; avoid re-quantizingMAX_LEN=262144 # Truncate as needed; don't max out 1M blindlyKV_CACHE_DTYPE=fp8GPU_MEM_UTIL=0.90
Fine-tuning notes
- With 1.56TB of weights, full fine-tuning is nearly infeasible.
- Prefer low-rank adaptation (LoRA)-style methods: freeze the backbone and train only a small number of adapter parameters.
- A quantization-aware-trained base is sensitive to quantization. A 4-bit base plus 4-bit adapters must first be validated on small samples to see whether precision is acceptable.
5. Deployment on Baidu AI Cloud: A Cloud Alternative Path
If your own hardware is insufficient, a cloud path is an option. Taking Baidu AI Cloud as an example, a common combination is:
- Object storage: 1.5TB of weights can be accelerated with multi-threaded concurrency and resumable uploads, and sharded uploads make verification easier.
- Container platform: GPU instances can be scheduled elastically as containers, with the parallelism strategy configured uniformly in the launch parameters.
- Monitoring, alerting, and log services: collect GPU utilization, VRAM usage, throughput, and latency, and provide unified search and retention for training and inference logs.
For elastic inference scenarios, model loading takes a long time. You can pre-warm the weights into a resident image, or use shared storage to speed up loading, avoiding re-pulling 1.5TB of data on every scale-out.
Deploying an open-weight model like this also means objectively weighing the cost of self-hosting. When high-end GPU resources are tight and operations capability is limited, a cloud API is often the more pragmatic choice. The real advantages of open-weight models lie in data control, deep customization, and offline operation.
Conclusion: Open Weights Are the Starting Point, Not the Finish Line
For developers, three judgments matter most:
- The license is a hard constraint. Under the default MIT-style core, the most easily overlooked parts are the large-scale commercial threshold and the competition clause—read the full text before deploying.
- Open weights are only the starting point. The communication complexity of the MoE architecture, the precision sensitivity of 4-bit quantization, and the KV cache pressure of long context all mean the difficulty of deployment is concentrated on the engineering side.
- Benchmarks should be checked against your needs. It performs strongly on coding and agent tasks and still trails on science and reasoning tasks, so validate item by item against your business workload distribution rather than looking at the total score.
If you want to validate the model but don’t yet have an 8-GPU high-end cluster, run a set of business-relevant benchmarks through the API first, confirm the payoff, and then decide whether to self-host.
