Trillion-Parameter Open-Weight Model: Weights, License, and Hardware Reality
An analysis of a newly released 2.8T-parameter open-weight multimodal model that moves past the launch headlines to answer three practical questions: what is actually inside the safetensors weights, what the vendor's custom license permits, and whether ordinary teams can realistically run a model that needs roughly 1.56TB of VRAM and at least 8 high-memory GPUs. It covers the sparse MoE architecture, the switch to sparse attention, 4-bit microscaling quantization, a worked KV-cache memory estimate, how to read vendor self-reported benchmarks, and a cost model for deciding between self-hosting and managed inference.
Introduction: The engineering work starts after the weights land
In early November, a large-model vendor made good on an earlier promise and publicly released the full weights of a 2.8T-parameter multimodal model on a mainstream model hosting platform. The weight package is roughly 1.56TB and can be downloaded without an application, making it the largest open-weight model available at the time. But “downloadable” is not the same as “usable”: by a rough VRAM calculation, local inference needs at least 8 high-memory GPUs.
This article skips the launch-day talking points and answers three concrete questions: what is actually inside the weights, what the license allows you to do, and whether an ordinary team can realistically run it.
1. Recap: API first, weights second
- Late October: The model went live first as an API and a web interface. The vendor promised the full weights would be released before November 4. Shipping the service first and opening the weights later let the vendor collect feedback from real usage and avoid the concentrated self-hosting support pressure that tends to follow a weight release.
- November 4: The weights were released on schedule. The repository on the hosting platform had been created on November 2 and was still closed that morning; it opened only in the evening, and outsiders could only wait for the official announcement.
- Delivery format: The weights are stored in sharded safetensors format—96 shards, 112 files, about 1.56TB in total.
safetensors is a common tensor-storage format in the industry: files store raw tensors directly and do not need to be deserialized into Python objects the way pickle does. This brings two practical benefits:
- Faster loading, with less risk of hitting memory-mapping or deserialization pitfalls during multi-process sharded loading.
- It contains only data and essential metadata and cannot execute arbitrary code on load, which lowers supply-chain risk.
2. Under the hood: how 2.8T parameters are composed
2.1 Separating total and activated parameters
According to the model card, this is a natively multimodal model trained for agent scenarios and continued from pre-training on the previous generation. Its most important structural feature is sparse mixture-of-experts (MoE):
| Dimension | Configuration | Engineering implication |
|---|---|---|
| Total parameters | 2.8T | Sets the VRAM floor; all experts must stay resident |
| Activated parameters per token | 104B | Sets per-step compute, roughly the compute of a 104B dense model |
| Routed experts | 896, 16 selected per token | Each token uses only about 1.8% of the experts |
| Shared experts | 2 | Every token passes through; carries general capability |
| Layers | 93 | Affects memory-bandwidth access depth and KV cache size |
| Context length | 1M tokens | Long documents, large codebases, long-horizon agent tasks |
Activated parameters are about 3.7% of the total, meaning roughly 27 units of “reserve knowledge” sit behind every unit of compute. This is exactly the MoE trade-off: trading VRAM for capacity, and decoupling “how much the model knows” from “how much each token computes.”
2.2 Attention redesign: 24 global-attention layers switched to sparse attention
Compared with the previous generation, only one structural change was made: 24 global-attention layers were replaced with sparse attention.
In standard global attention, the nth token must compute relevance against each of the previous n−1 tokens, so both compute and memory grow quadratically with sequence length. Sparse attention lets each token attend to only a subset of positions; common approaches include a sliding window plus a few global tokens, block sparsity, and learnable token selection. Complexity drops from O(n²) to O(n·k), where k is a constant tied to the window size.
At a 1M-token context, the significance of this change is direct: about a quarter of the layers no longer do a global pairwise comparison. But note that what is saved here is compute, not necessarily memory. Whether these 24 layers keep a full KV cache depends on the specific sparse-attention implementation. You must measure before deployment—don’t judge by the word “sparse” alone.
2.3 Native multimodality: vision encoder and agent capabilities
The vision side uses a ViT-style visual encoder trained together with the language model, forming native multimodal capability rather than a bolted-on image-text concatenation. Engineering-wise:
- Images also consume the token budget: Images are patch-encoded into the sequence, and high-resolution images can consume thousands of visual tokens. The window left for text must subtract what images and the system prompt occupy.
- Long-horizon agent tasks: The point of a 1M-token window is to let the model “remember” the entire task’s tool-call history, intermediate artifacts, and error logs in a single inference, reducing information loss from mid-course summarization.
2.4 Low-precision weights: memory savings and compatibility cost
The weights use a 4-bit microscaling floating-point format and activations use 8-bit microscaling floating-point; both are products of quantization-aware training (QAT), meaning the model adapted to low precision during training rather than being compressed afterward. Microscaling is intuitive: every 32 elements form a group that shares an 8-bit exponent scaling factor, and each element in the group stores only 4 significant bits. The cost is the overhead of the scaling factors themselves; the benefit is roughly one-quarter of the BF16 memory footprint and reduced error accumulation from per-channel quantization.
Note that the vendor released only this one quantized version, with no BF16 or FP8 variant. This means:
- Benefits are tied to hardware: The inference framework and GPUs must natively support this kind of block-level microscaling format. Otherwise you must dequantize before running, and the memory and compute advantages disappear.
- No precision fallback: If your business has numerical-precision requirements, there is no ready-made higher-precision version to switch to; you can only evaluate on your own data whether the error is acceptable.
# Model parameter sketch (illustrative summary; defer to the official model card)model:total_params: 2.8e12 # Total parameters: sets the VRAM flooractive_params: 1.04e11 # Activated parameters per token: sets compute demandnum_layers: 93context_length: 1048576 # 1M tokensmoe:num_experts: 896experts_per_token: 16 # Routed experts: only 16/896 activated per tokenshared_experts: 2 # Shared experts: every token passes throughattention:full_attention_layers: 69 # Global attentionsparse_attention_layers: 24 # Sparse attention, reduces long-sequence computequantization:weight: 4bit_microscaling # Group shares an 8-bit exponent scaling factoractivation: 8bit_microscalingtraining: qat # Quantization-aware training
3. The memory ledger: how 1.56TB adds up to 8 cards
3.1 Volume check: where 1.56TB comes from
A rough check: at 2.8T parameters and 4 bits per parameter, the weights themselves come to about 2.8×10¹² × 0.5 B ≈ 1.4TB. The microscaling format adds one scaling factor per 32 elements, plus metadata and configuration across 96 shards, so a total in the 1.56TB range is entirely reasonable. This also shows that 1.56TB is essentially the true in-memory footprint of the weights themselves, not a compressed-archive size.
3.2 Per-card load: 8 cards means about 195GB each
Weights don’t shrink just because they are spread across 8 cards. A rough per-card load is 1.56TB ÷ 8 ≈ 195GB, and the weights alone already exceed the ceiling of current mainstream single-card capacity. Comparing across single-card capacities:
| Single-card VRAM | Cards needed for weights only (est.) | Notes |
|---|---|---|
| 80GB | 20 cards | Spans at least 3 8-card machines, cards connected over network, latency-sensitive |
| 141GB | 12 cards | Still requires cross-machine deployment |
| 192GB | 8 cards | 8 cards total about 1.54TB, comparable to the weight size |
So “8 cards to start” really means you need at least one 8-card machine with about 192GB per card to hold the weights. The numbers above count only the weights, not the KV cache and runtime overhead. When one card simultaneously serves dozens or even hundreds of sequences, those two also consume substantial VRAM.
Fitting inside the machine is only half of it; how the cards are connected matters too. Each MoE inference layer has two kinds of communication:
- Tensor parallelism: Splitting a single expert’s matrix computation across cards.
- Expert parallelism: Dispatching tokens to the cards hosting experts via all-to-all, then gathering the results.
Both depend on high-bandwidth, low-latency inter-card interconnect; ordinary 10-gigabit Ethernet-class networks struggle to keep up. In practice, a single 8-card machine with high-speed interconnect is basically the minimum bar for this class of model.
4. KV cache: the real bottleneck for long-context experience
4.1 Estimation formula
KV cache size can be broken into three parts:
KV cache bytes ≈ 2 (K and V) × layers × KV heads × head dim × sequence length × bytes per element × concurrency
Most of the 2.8T parameters are “sleeping” experts that occupy memory bandwidth only when routed to. The KV cache is the opposite: every new token the sequence generates, all layers read and write the full history cache. In long-context scenarios, inference easily shifts from “compute-bound” to “memory-bandwidth-bound.”
def estimate_memory(weight_bytes, # Total weight sizelayers=93, # Number of layerskv_heads=8, # KV heads: illustrative, defer to model confighead_dim=128, # Per-head dimension: illustrativekv_bytes=1, # KV cache precision: 1 byte, e.g., FP8ctx_len=1_048_576, # Context lengthconcurrency=1, # Number of concurrent sequencesoverhead=1.15, # Runtime overhead: activations, comm buffers, VRAM fragmentation, etc.):bytes_per_token = 2 * layers * kv_heads * head_dim * kv_byteskv_bytes_total = bytes_per_token * ctx_len * concurrencyreturn {"weights_GB": round(weight_bytes / 1e9, 1),"kv_cache_GB": round(kv_bytes_total / 1e9, 1),"total_GB": round((weight_bytes + kv_bytes_total) / 1e9 * overhead, 1),}print(estimate_memory(1.56e12))# {'weights_GB': 1560.0, 'kv_cache_GB': 190.5, 'total_GB': 2013.2}
With the illustrative parameters above, a single 1M-token sequence needs an extra ~190GB of KV cache, pushing the whole-machine requirement from about 1.56TB to about 2TB. That is before concurrency: 8 concurrent sequences need nearly 1.5TB, and 10 concurrent sequences approach 1.9TB.
For comparison:
| KV configuration | Cache for one 1M-token sequence |
|---|---|
| Standard cache, FP8 | ~190GB |
| KV cache quantized to 4-bit | ~95GB |
| Cache only some history tokens (eviction policy) | Drops linearly with retention ratio |
4.2 Why long context easily degrades into “fake capability”
Real inference frameworks generally don’t let every request occupy the full window; instead, through paged management and preemptive reclamation, they allocate the KV cache on demand to active requests. As the context is stretched, three phenomena can appear:
- Throughput collapse: As concurrency rises, there isn’t enough memory to allocate, and requests queue up or get preempted and recomputed.
- Ineffective long context: Nominal 1M tokens, but stable serving might be only tens of thousands of tokens. Beyond that, you either queue or are forced to truncate.
- Diluted attention: With very long sequences, the model’s efficiency in using mid-sequence information can drop noticeably.
So the only reliable way to judge whether this class of model works for long documents or long-horizon agent tasks is load testing: fix concurrency, gradually increase input length, and watch time-to-first-token, output throughput, and the VRAM-usage curve.
5. License: open weights are not open source
This is the point most worth calling out separately in this release.
| Comparison dimension | Standard open-source license (e.g., MIT, Apache family) | Custom model license (used here) |
|---|---|---|
| Core grant | Use, copy, modify, distribute | Main terms close to an MIT-style grant, plus additional terms |
| Additional conditions | Mainly copyright and disclaimer | May involve a usage policy, commercial-scale thresholds, attribution or naming requirements |
| Platform label | Usually directly identifiable as the corresponding open-source license | Labeled “other” (license: other) |
The hosting platform labels it license: other, not a standard open-source license like MIT or Apache. The repository carries the vendor’s custom Kimi K3.1 Pro License, whose text is word-for-word identical to the previous generation’s, with only the name changed.
Breaking it down:
- The main terms are permissive: Free use, copy, modification, distribution, fine-tuning, and commercial use are allowed—important for downstream development such as continued pre-training or domain fine-tuning on your own data.
- Additional terms need line-by-line review: Custom licenses usually attach an acceptable-use policy, extra obligations once a certain commercial scale is reached, and model naming or attribution requirements. Don’t treat it as standard open source just because it “looks like MIT.”
- Open weights ≠ open source: By the prevailing definition in the open-source governance community, a fully open-source model should also include training code, training methodology, and even data documentation, so third parties can reproduce the training process. Open weights publish only the final parameters; the training recipe and data are not included. Continued training and continued pre-training can’t be done from weights alone.
Before deploying in an enterprise, confirm at least four things:
- Whether commercial use has scale thresholds: For example, whether reaching a certain DAU, revenue, or distribution volume requires a separate license or additional obligations.
- Whether redistribution is restricted: What terms apply to offering the model as SaaS, or embedding it in hardware or commercial products.
- Ownership of output content: Whether model-generated content can be freely commercialized, and whether attribution is required.
- Compliance obligations: Generated-content labeling, data export, content safety, and similar requirements need joint confirmation with legal and security teams.
6. Benchmarks: how to read vendor self-reported numbers
The model card gives only vendor self-reported results:
- Strong on coding and agent tasks: Terminal-Bench 4.0 scores 65.7%, slightly below a closed-source flagship’s 66.4% and above another’s 58.2%; the long-horizon software engineering task SWE Marathon scores 51.7%, beating the other’s 46.3%.
- Weaker on science and some reasoning tasks: The scientific terminal task Terminal-Bench-Science is only 38.1% vs. the other’s 63.3%; the hard composite-knowledge test HLE-Diamond is 39.4% vs. 60.9%.
- Overall a mixed bag: The vendor says it beats the comparison model on 16 of 37 benchmarks, and going through them one by one confirms the “strong on coding and agents, weak on science and some reasoning” judgment.
- Third-party reference: An evaluation organization’s intelligence index gives it 55 points, ranking fourth globally.
Three principles for reading these leaderboards:
- Self-reported isn’t reproducible: Vendors usually use their own evaluation frameworks, prompt templates, and sampling parameters, and different frameworks handle tool calls, error retries, and context concatenation very differently.
- A single score can’t replace business validation: General benchmarks differ greatly from real business distributions. Technology selection should rely on a self-built eval set, judged by business metrics like pass rate, average interaction turns, and human adoption rate.
- Open weights provide the condition for self-verification: This is a practical advantage of open-weight models over pure-API models. But the 8-card starting bar means verification costs aren’t low for most teams.
7. Deployment decisions: which scenarios fit and how to calculate cost
7.1 Scenarios to prioritize
- Data can’t leave the domain: Code, contracts, or medical and government documents that can’t be uploaded to an external API.
- Ultra-long context: A single task needs a whole code repository, long documents, or complete tool-call history.
- Complex agent workflows: The model needs to plan autonomously, call tools, and repeatedly fix errors.
- Existing high-memory compute: You already have an 8-card high-memory machine or equivalent cloud resources.
7.2 Scenarios to approach cautiously
- High-concurrency tasks that are mainly text classification, extraction, or rewriting: Using a model with 104B activated parameters is usually less cost-effective than a small model.
- Teams without GPU-ops experience: The tuning cost of distributed inference often exceeds the model’s license cost.
- Strong constraints on output format: You need to first verify structured-output stability under low-precision quantization.
7.3 Cost and deployment options
The core of self-hosting is fixed cost: GPU-machine depreciation or annual cloud fees, rack and power, and ops staff. Calling an API is a variable cost billed per token. There is a break-even point between the two, expressed as a formula:
V* = F ÷ (p_api − p_self) × 10⁶
where F is the monthly fixed cost of self-hosting, and p_api and p_self are the per-1K-token costs under API and self-hosting modes respectively. When monthly call volume (in millions of tokens) exceeds V*, self-hosting is usually more economical. This is only a rough estimate; p_self on the self-hosting side also needs to account for waste from insufficient GPU utilization.
If self-hosting isn’t economical, you can also use managed inference services on cloud platforms, which offer general capabilities like GPU compute clusters, object storage, and model serving while saving you the cluster-ops effort. The choice between self-hosting and managed should be based on a combined estimate of your own call volume, data-compliance requirements, and ops capability—don’t judge by per-call price alone.
7.4 Fine-tuning and distillation
The license allows fine-tuning and distribution, so you can do domain adaptation on this weight base, such as continued pre-training, supervised fine-tuning, and preference alignment. A common cost-reduction idea is tiered deployment:
- Route simple requests to a small model.
- Escalate complex requests to the trillion-parameter model.
- Collect model outputs and user feedback, distill a small model, and progressively replace high-frequency calls.
This covers most request volume with less compute.
Conclusion
The value of this weight release isn’t just a downloadable model—it puts the engineering problems of deploying ultra-large open-weight models on the table. Four takeaways:
- Scale: 2.8T parameters and about 1.56TB of weights make it the largest open-weight model to date, and that capacity underpins agent and long-context capabilities.
- Architecture: Sparse MoE with shared experts separates “capacity” from “compute,” sparse attention lightens the load for a 1M-token context, and low-precision quantization compresses the memory requirement to around 1.5TB—but also leaves the compatibility bar to the inference framework and hardware.
- License: Open weights aren’t open source; the main terms are permissive, the additional terms need line-by-line review, and legal must confirm before commercial use.
- Deployment: An 8-card high-memory machine is only the starting point; the KV cache is what determines long-context experience.
Benchmark leaderboards are only a reference. What really determines whether it can be deployed comes down to three things: performance on your own eval set, throughput and latency under load testing, and how well the license terms match your business.
