Trillion-Parameter Open Weights Released: Technical Breakdown and Deployment Feasibility
A flagship 2.8T-parameter model has been released as open weights, with roughly 1.56TB of files available for direct download. This article breaks down the model's architecture — MoE sparse activation, a shift to sparse attention, and MXFP4 weight quantization — and explains what those choices mean for deployment. It interprets the vendor-reported benchmarks, which show near parity with international flagship commercial models on coding and agentic tasks but a clear gap on science and some reasoning tasks. It clarifies the custom "open weights" license (an MIT-style core with added conditions) and walks through the real VRAM math: a high-capacity 8-GPU node is only the starting point for short-context, low-concurrency serving, while long-context or high-concurrency workloads typically require 16 or more GPUs or a hosted service. It closes with a comparison of three deployment paths and action checklists for individual developers, SMB teams, and infrastructure teams.
Introduction: An “On-Schedule” Open-Weight Release
On November 4, a major-model developer released the full weights of a 2.8T-parameter flagship model, as it had previously promised: about 1.56TB of files made publicly downloadable with no application required. The developer called it “the strongest open model you can currently download.” At first the repository was closed; it opened that evening, and discussion heated up soon after.
Downloading is not the same as being able to run it, and open weights are not the same as open source. This article follows a “capability — license — deployment” throughline and offers judgments on three levels:
- Capability: coding and agentic tasks approach international flagship commercial models, but science and some reasoning tasks still lag;
- License: a custom license that is essentially MIT-style with added conditions — “open weights,” not standard open source;
- Deployment: a single 8-GPU node can barely handle loading the weights; long-context scenarios usually need 16 or more GPUs, or a hosted service.
1. Event Recap: Timeline and Key Numbers
- October 24: The model launched first via API and a web version. The team promised the full weights would be released before November 4, and this phase focused on demonstrating capability.
- November 2: A public weight repository was created but remained closed.
- November 4: The repository was still closed in the morning and opened that evening. The weights landed within the promised window, shifting control from hosting platforms back to the community.
- Weight scale: The repository holds 112 files, including 96 weight shards (safetensors format), totaling about 1.56TB. The weights could also be pulled through community mirrors.
- Historical position: This is the largest open-weight model released publicly to date — an order of magnitude larger than comparable prior models.
One note up front: the architecture parameters, benchmark scores, license terms, and download scale discussed here all come from the official model card and public reporting. Anything that is the author’s own calculation is labeled “estimated” in the text.
2. Architecture Breakdown: How 2.8T Parameters Are Built
2.1 MoE Sparse Activation: What a 3.7% Activation Rate Means
According to the official model card, the architecture is:
- 2.8T total parameters, with only 104B activated per token;
- An MoE structure with 896 routed experts, 16 selected per token, plus 2 shared experts;
- 93 layers;
- A context window of 1 million tokens;
- A MoonViT-V2 vision encoder that supports image understanding;
- Positioned as a natively multimodal agentic model, continued-trained from the previous generation.
104B ÷ 2.8T ≈ 3.7%, meaning that for each token generated, fewer than 4% of the parameters take part in the computation. This is the key to understanding the deployment cost of such models, because VRAM usage is determined by total parameters, while compute consumption is determined by activated parameters. These two metrics differ by roughly 27×, so the traditional “parameters × bytes” heuristic for estimating VRAM is no longer sufficient:
- Using the dense-model rule of thumb “about 2 GFLOPs per 1B parameters per token,” the compute per token is about 208 GFLOPs;
- Using the total 2.8T parameters gives 5.6 TFLOPs, an overestimate by about 27×.
The cost of MoE lies in the uncertainty introduced by routing. A single expert carries only a small fraction of tokens, so inference engines struggle to batch matrix multiplications into large sizes, and GPU compute utilization tends to drop. At the same time, experts are spread across multiple cards, and every routing step involves cross-card communication. So the actual throughput of an MoE model often depends on communication efficiency and batching strategy, not just on how few parameters are activated.
2.2 Differences From the Previous Generation: One Structural Change
In terms of parameter scale, 93 layers with 896 experts means each layer is not large. The developer says only one thing changed versus the previous generation: the full attention used in the original 24 layers was replaced with sparse attention (DSA), while the rest of the design carried over.
The benefits of sparse attention come mainly from two areas:
- Reducing how much compute grows quadratically with context length: full attention’s compute grows quadratically with context length; after sparsification, the prefill stage for long text is noticeably faster;
- Lowering the marginal cost of long context: combined with the 1-million-token window, tasks like document-level analysis and cross-file refactoring become more feasible.
Note that sparse attention mainly reduces the compute of attention; it does not necessarily reduce the VRAM usage of the KV cache. Whether the KV cache is compressed, and by how much, depends on the specific implementation and the inference engine’s support. The fact that dozens of layers keep full attention also shows the team’s tradeoff between long-context capability and accuracy: sparsification was applied only to the less sensitive middle layers.
For engineering teams, this “limited change” architecture has a hidden benefit: much of the previous generation’s deployment experience, parallelism strategies, and monitoring configuration can be reused, keeping migration costs relatively low.
2.3 Quantization Design: MXFP4 Weights With MXFP8 Activations
The model uses MXFP4 weights and MXFP8 activations. MX (Microscaling) is a block floating-point format specification: data shares a single scaling factor in small blocks, balancing 4-bit storage with dynamic range. The weights are the product of quantization-aware training (QAT), meaning low precision was factored in during training rather than applied as post-hoc compression.
The developer released only this one version — no BF16 or FP8 version. This has two implications:
- Positive: The weight size is about a quarter of a same-scale BF16 model, greatly lowering the VRAM threshold; when the hardware natively supports the format, the weights need not be dequantized during loading, saving a full memory read-write pass.
- Constraint: The inference framework used for deployment must support this format. If it doesn’t, you must dequantize first and then requantize, adding substantial temporary storage and conversion time, and possibly precision loss. The absence of a BF16 version also means the fine-tuning ecosystem must be filled in by the community.
3. Capability Assessment: How to Read Vendor Self-Test Data
3.1 Benchmarks: Strong at Coding and Agents, Weak at Science and Some Reasoning
All officially published data is vendor self-tested:
| Benchmark | This model | Intl. flagship A | Intl. flagship B |
|---|---|---|---|
| Terminal task benchmark | 65.7% | 66.4% | 58.2% |
| Long-horizon software engineering benchmark | 51.7% | 46.3% | — |
| Terminal science task benchmark | 38.1% | 63.3% | — |
| Hard academic Q&A benchmark | 39.4% | 60.9% | — |
Here “international flagship models A/B” refer to the two international commercial flagship models the team cited in its official comparisons. Overall:
- Coding and agentic tasks: only about 0.7 percentage points behind A on the terminal task benchmark — near parity — and about 5.4 points ahead of A on long-horizon software engineering.
- Science and reasoning tasks: about 25 points behind A on the terminal science task benchmark; about 21.5 points behind on hard academic Q&A — less than two-thirds of A.
- Overall distribution: the developer says that of 37 benchmarks, 16 beat A, with advantages concentrated in coding and tool calling, and weaknesses concentrated in tasks requiring deep domain knowledge or multi-step rigorous reasoning.
This matches the capability distribution pattern of MoE models. Sparse activation is highly efficient on pattern-based, verifiable tasks (writing code, running commands, calling tools). But on tasks requiring broad world knowledge and long reasoning chains, the effective parameters each token “activates” are fewer, so weaknesses surface more easily.
3.2 Key Points for Interpreting Self-Test Data
Vendor self-test data is only suitable for reading trends, not for direct decision-making:
- Scaffolding differences: the same model can score noticeably differently with a different prompt template, toolset, or sampling parameters;
- Evaluation methodology: pass@1 (single-attempt pass rate), pass@k (at least one pass within k attempts), and multi-run averages are not directly comparable;
- Data contamination: public benchmarks may already be in the training data;
- Aggregate rankings: a third-party composite intelligence index is just one reference view and cannot replace validation on your specific scenario.
3.3 Build an Internal Evaluation Set
A sensible approach is to build a small evaluation set close to your own business, run candidate models side by side with the same scaffolding and the same set of prompts, and record task completion rate, average latency, output token count, and cost per call at the same time. This mirrors the user-feedback-driven iteration approach in application development. The evaluation set need not be large — 100 to 300 highly discriminative samples are usually enough to reveal clear gaps.
4. License Interpretation: “Open Weights” ≠ “Open Source”
This is the part of the release most worth discussing on its own. The repository ships a custom Kimi K3.1 Pro License, marked as license:other on the model hosting platform — not a standard open source license like MIT or Apache. According to public reporting, its body text is identical, word for word, to the previous generation’s license, with only the name changed.
4.1 Core Grant: MIT-Style License
The core grant is an MIT-style license, permitting free use, copying, modification, distribution, fine-tuning, and commercial use. For the vast majority of individual developers and small-to-medium teams, this means they can use it for secondary development, build vertical applications, even embed it in commercial products, without paying a licensing fee.
4.2 Additional Conditions: Thresholds for Large-Scale Commercial Use
Custom licenses typically add constraints on top of the core grant. Based on public reporting, this license at least requires attention to the following dimensions:
- Redistribution obligations: derivative models may need to be renamed and carry a license notice, and the original copyright and license text must not be removed;
- User-scale thresholds: once a product reaches a certain monthly-active-user scale, separate written authorization may be required;
- Usage-scenario restrictions: a few licenses restrict certain industries or uses.
Note here: the exact wording of the additional conditions must follow the license text in the repository. Any second-hand interpretation, including this article, cannot replace legal judgment.
4.3 Open Weights vs. Open Source
The open-source definition the industry commonly cites typically requires that the software can be freely used for any purpose, without restrictions on how it is used. Licenses with user-scale thresholds or scenario restrictions do not meet this standard. The industry generally calls such licenses “open weights” or “source-available” to distinguish them.
| Dimension | Typical open source (MIT/Apache, etc.) | Open-weight custom license (this model) |
|---|---|---|
| Weight access | Public | Public, no application required |
| Commercial use | Allowed | Allowed, but may be subject to user-scale constraints |
| Modification & fine-tuning | Allowed | Allowed |
| Redistribution | Keep copyright and license notice | Additional naming and attribution requirements |
| Training code & data | Usually released together or described | Generally not included |
| Future version risk | Downstream use protected under same license | New versions may change terms |
The last row is especially worth noting. Such custom licenses typically carry clauses like “this model or its outputs may not be used to improve the capabilities of other models.” If a team plans large-scale distillation or continued pre-training on the weights, it needs to assess compliance risk in advance.
4.4 Enterprise Compliance Checklist
- Retain all license and copyright files in the repository, and archive and distribute them together with the model;
- Check the license text to confirm whether your own product’s monthly-active-user scale triggers additional authorization;
- If doing secondary distribution, name and attribute derivative models as the license requires;
- Verify that the license text matches the model card’s description, treating the actual text in the repository as authoritative;
- Third-party components such as the vision encoder may carry their own licenses and should be checked as well.
5. Deployment Feasibility: What “8-Card Minimum” Really Means
5.1 Hardware Threshold: VRAM Is the First Hard Constraint
The weight files total about 1.56TB (decimal), which converts to about 1.42TiB (binary). Whatever parallelism scheme is used, this data must reside in GPU VRAM (or be coordinated between unified memory and host memory). As long as per-card capacity doesn’t increase, parallelism can only spread the weights across more cards — it can’t reduce the total out of thin air.
Estimating against current mainstream per-card VRAM capacities:
| Node config | Total VRAM (approx.) | Remaining after weights (approx.) | Feasibility |
|---|---|---|---|
| 8×80GB | 586 GiB | -870 GiB | Not feasible, weights don’t fit |
| 8×144GB | 1050 GiB | -402 GiB | Not feasible |
| 8×192GB | 1394 GiB | -58 GiB | Not feasible, weights already overflow |
| 16×144GB | 2101 GiB | ~650 GiB | Feasible, short context |
| 8×240GB | 1788 GiB | ~330 GiB | Feasible, short context, low concurrency |
Per-card capacity converts roughly as 128GB ≈ 119 GiB, 80GB ≈ 74 GiB, and so on. As the table shows, “8-card minimum” presupposes at least 240GB-class VRAM per card. Common single-node 8×80GB or 8×144GB configurations cannot hold these weights.
5.2 The Truly Scarce VRAM: The KV Cache
After subtracting the weights, the remaining VRAM must serve all of the following at once:
- The KV cache;
- Runtime activations;
- CUDA context and memory fragmentation;
- Communication buffers.
Assuming a 128-core CPU, a GPU VRAM utilization ceiling of 0.9, and NVLink interconnect, an estimation script can quickly give the remaining headroom:
# VRAM budget estimate (illustrative); all units in GiBdef estimate(weight_gib=1452, cards=8, per_card_gib=224,util=0.90, kv_gib=0, reserve_gib=20):total = cards * per_card_gib * utilreturn total - weight_gib - kv_gib - reserve_gib# 8 cards x 240GB: ~310 GiB with no KVprint(estimate())# after reserving 200 GiB for KV: ~110 GiBprint(estimate(kv_gib=200))
For 8×240GB:
- Runtime-available VRAM is about 1612 GiB;
- After deducting weights and system reserve, about 140 GiB is available for the KV cache;
- If concurrency is deliberately kept low, leaving only 40 GiB for the KV cache, the overall headroom is about 240 GiB.
This headroom isn’t generous for a single long sequence. By the general principle of attention, the bytes of KV cache per token can be approximated as:
2 × KV heads × head dim × precision bytes × layers
where the factor of 2 accounts for one copy each of keys and values. Taking “8 KV heads, head dim 128, KV precision FP8, all 93 layers keeping a full KV” as an upper-bound estimate: each token is about 174 KiB, and a 1-million-token context needs about 166 GiB. Whether grouped-query attention (GQA) is actually used, and whether some layers use sparse or compressed attention, will change this number significantly.
This yields a practical rule of thumb:
With short context (about 30K tokens) and low concurrency (1–4 streams), a high-capacity 8-card configuration is viable; once you target long context or multiple concurrent streams, you need 16 or more cards, or optimizations such as KV quantization, paged caching, and prefix reuse.
5.3 Throughput: The Decode Stage Is Memory-Bandwidth Bound
An MoE model is memory-access-intensive during autoregressive decoding: every token generated requires reading the activated parameters in full.
- Single-stream speed: with 8 cards, each token requires moving at least 104B parameters from VRAM. Assuming about 8 TB/s bandwidth per card and a 50% read-efficiency factor, single-stream decode speed is roughly 10 to 20 tokens/s.
- Batch scaling: as the concurrent batch grows, the weights only need to be read once yet can be shared across multiple requests, so throughput can multiply until compute becomes the bottleneck.
- Prefill stage: compute per token is about 208 GFLOPs. Assuming about 1 PFLOPS of dense compute per card and a 30% effective utilization, long-text prefill runs at roughly a few thousand tokens per second.
For agentic applications, multi-turn tool calls repeatedly trigger prefill, so prefix-cache reuse often matters more than peak compute.
5.4 Parallelism Strategy: Expert Parallelism Within a Node, Data Parallelism Across Nodes
The core of parallel design for MoE models is keeping the all-to-all communication produced by routing within the high-speed interconnect domain as much as possible.
- Within a single node: use tensor parallelism (TP) to split non-expert layers, and expert parallelism (EP) to spread routed experts across the NVLink domain. NVLink bandwidth can reach around 900 GB/s, enough to handle all-to-all communication.
- Cross-node scenario: prefer data parallelism (DP) plus tensor parallelism, so each node holds a full model copy and each copy handles requests independently. Only inputs and outputs cross nodes, so the communication volume is small.
- Avoid cross-node expert parallelism where possible: tokens have to be routed back and forth across nodes, and cross-node network bandwidth is typically more than an order of magnitude lower than NVLink, easily becoming a throughput bottleneck.
Rule of thumb: for MoE deployment, ask “how are the cards interconnected” before asking “how many cards do I need.”
An illustrative deployment configuration is as follows (parameter meanings are general and do not represent any specific framework):
model:path: /models/k3_1_pro_mxfp4max_model_len: 32768 # start at 30K tokens, raise as neededparallel:tensor_parallel_size: 8 # TP within a node, don't span nodesdata_parallel_size: 2 # two nodes each hold a full copycache:kv_cache_dtype: fp8 # trade precision for KV capacityenable_prefix_caching: true # reuse system prompt and tool definitions
5.5 An Often-Overlooked Path: CPU+GPU Hybrid Inference
The -ngl parameter in the llama.cpp ecosystem controls “how many layers go on the GPU”: -ngl 99 means offload as many layers as possible to the GPU, and -ngl 0 means run everything on the CPU.
For this roughly 1.42TiB of weights:
- On machines with clearly insufficient total VRAM, keeping some expert layers in CPU memory lets you get the model running first;
- But every token generated requires moving weights over the capacity-limited, higher-latency PCIe link, dropping speed to seconds per token or even slower.
Its role is “runs and validates,” not production serving.
5.6 Downloading, Storage, and Integrity Checks
A 1.56TB download is an engineering problem in itself:
- Bandwidth and time: gigabit bandwidth takes about 3.5 hours in theory, plus source-side rate limiting, network latency, and retries;
- Optimization: enable chunked parallel downloads and resumable downloads; prefer a fast mirror when available;
- Integrity checks: the safetensors format carries header metadata and is well suited to per-shard verification; after downloading, compare each shard against the officially published checksums;
- Storage: reserve about 1.6× the weight volume. If you later convert the format or export GGUF, reserve another 1.5TB+ of temporary space.
6. Choosing a Deployment Approach: Three Paths Compared
| Approach | Initial investment | Unit cost | Elasticity | Best for |
|---|---|---|---|---|
| Hosted API | Low | Grows linearly with usage | Naturally elastic | Early validation, spiky traffic |
| Self-hosted cloud GPU instances | Medium | Near-fixed | Self-managed scaling | Stable traffic; deep customization or data isolation |
| On-premises cluster | High | Low after fixed costs are amortized | Long expansion cycle | Data that can’t leave the intranet; sustained high load |
As a rough calculation for a typical on-demand cloud GPU instance: an 8-card 240GB-class instance costs roughly RMB 20–30 per hour, or about RMB 15,000–22,000 per month. Such prices vary by region, instance type, and billing model. This is roughly on par with the flagship API output prices of leading models — meaning self-hosting isn’t inherently cheaper. Only when average daily usage reaches the hundreds-of-millions-of-tokens level does the amortized unit-cost advantage appear.
If a company chooses the cloud route, it can build its deployment environment using general capabilities from cloud providers like Baidu AI Cloud — GPU instances, container services, object storage, and monitoring/alerting. For teams already using other Baidu AI Cloud products, completing model deployment and data processing within the same cloud environment can also reduce the complexity of cross-cloud data transfer and network links.
7. Best Practices: Action Lists for Three Types of Users
Individual Developers and Researchers
- Download: confirm the machine has at least 1.6× the weight volume in free disk space;
- Hardware: a single 80GB-class card cannot load it; an 8-card workstation with 192GB-class accelerators is a more realistic starting point;
- CPU hybrid inference: suitable for validating logical feasibility — don’t have high expectations for speed;
- Insufficient resources: a hosted API is the most cost-effective way to try it out;
- License: review the license and the code separately, and watch for attachment clauses.
Small and Medium Business Teams
- Positioning needs: land first in scenarios where the model excels — code generation, command execution, tool calling; for deep industry-knowledge Q&A and complex scientific computing, run small-scale comparison tests first;
- Approach selection: teams with fewer users that don’t hit the license’s scale threshold can use it commercially with confidence; as scale approaches the threshold, start the authorization conversation early;
- Cost control: use “total cost per task” rather than “price per million tokens” as the decision metric.
Platform and Infrastructure Teams
- Interconnect: prioritize NVLink between cards and reserve bandwidth for expert parallelism;
- Architecture: insist on data parallelism across nodes, and avoid cross-node expert routing;
- Performance optimization: enable KV quantization, paged caching, and prefix reuse;
- Observability: monitor TTFT, TPOT, KV cache utilization, and expert-routing balance;
- Security: add input/output content-safety filtering, and enforce permission controls on tool calls to prevent agents from being exploited via prompt injection;
- High availability: deploy multiple replicas to avoid a single point of failure taking down the whole service.
Conclusion
The full release of a 2.8T-parameter model with 1.56TB of weights marks the arrival of open-weight models in the trillion-parameter era. For the industry, this means the capability ceiling keeps approaching — and partly surpassing — international flagship commercial models, and the way model capability is obtained is shifting from “API calls only” to “bringing it back into your own hands.”
But this “freedom” has boundaries:
- License: the scaling thresholds and redistribution requirements in the custom license determine who can use it and how;
- Hardware: a high-capacity 8-card configuration is only a starting condition; long-context and high-concurrency scenarios demand multiples of that investment;
- Capability: being near parity on coding and agentic tasks does not mean the science and reasoning gaps can be ignored.
The rational approach is to run a low-cost trial with a hosted service first, complete evaluation in real business scenarios, and then decide whether to self-host based on traffic scale and compliance requirements. The model can be downloaded, but engineering judgment cannot be outsourced.
