# 2.8T Open-Weights Models: License Boundaries and Deployment Costs
A model team has released the full weights of its flagship Kimi K3.1 Pro, an open-weight model with 2.8T total parameters and a 1.56 TB download. This article explains what the weights reveal about its architecture (sparse MoE, sparse attention, native MXFP4 quantization), how to interpret its self-reported benchmarks, where the limits of its custom license lie, and how to budget VRAM for self-hosting. It also compares self-hosting with cloud hosting and ends with a five-step validation checklist to complete before committing hardware.
1. The weights are out: three things that actually matter
A major model team has released the full weights of its flagship model, Kimi K3.1 Pro, on the date it had committed to. With 2.8T total parameters and a repository size of about 1.56 TB, it is now the largest open-weight model available for download. The weights are hosted in sharded form on mainstream open model repositories and can be downloaded without any application.
The model first launched as an API and a web app, with the weights following two weeks later, right on schedule. What was once “using someone else’s service” is now “having the model in hand.”
Once the weights land, the question shifts from “can we use it” to three others:
- Architecture: how the 2.8T parameters are organized. The sparse activation ratio determines compute cost, and the quantization format sets the hardware bar.
- License: downloadable does not mean commercially usable. The additional terms in the custom license are the real compliance boundary.
- Deployment: 1.56 TB of weights divided by the number of GPUs is a hard threshold. “8 GPUs to start” still requires a VRAM budget that has to be worked out carefully.
Each of these is examined below.
2. Model architecture: how the 2.8T parameters are organized
2.1 Sparse MoE: total compute is decoupled from per-token compute
Kimi K3.1 Pro is a natively multimodal agent model. It was trained on from the previous-generation base model. Its core architectural features are:
| Dimension | Configuration |
|---|---|
| Total parameters | 2.8T |
| Activated parameters per token | 104B (about 3.7% of the total) |
| Expert structure | 896 routed experts, 16 selected per token, plus 2 shared experts |
| Layers | 93 |
| Context length | 1 million tokens |
| Visual encoding | Proprietary multimodal vision encoder |
The key implication: VRAM cost scales with 2.8T, while compute cost scales with 104B.
All weights must stay resident in VRAM (or be swapped in and out between layers). However, only about 3.7% of the parameters take part in matrix multiplications for each token generated. This explains two seemingly contradictory facts:
- The deployment barrier is extremely high.
- Per-GPU compute utilization is not low. Throughput often holds a competitive edge over dense models with comparable activation levels.
The shared experts also provide stable capacity for general capabilities, so those capabilities do not depend on whether the router happens to hit the right experts.
2.2 The one structural change from the previous generation: sparse attention
The only notable structural change is in the attention mechanism. In 24 of the 93 layers, full attention has been replaced with DeepSeek Sparse Attention (DSA). The goal is clear: under a 1M-token context, the complexity of attention computation grows quadratically with sequence length. Making it sparse is necessary to save compute at long context lengths.
For deployers, the implications cut both ways:
- Benefits: the compute and wall-clock time of the prefill stage drop significantly at long context lengths. Tasks such as long-document and large-codebase understanding become more practical.
- Costs: sparse attention depends on dedicated kernels in the inference framework. If the framework does not support these kernels, real-world performance may fall far short of the levels claimed on the model card.
2.3 Quantization-native design: what an MXFP4-only release means
The weights use MXFP4 (4-bit microscaling floating point) and the activations use MXFP8. This comes from quantization-aware training (QAT), not post-training quantization. Two trade-offs need to be understood:
- Advantages: QAT lets the model adapt to low precision during training. As a result, accuracy loss is far smaller than with PTQ (post-training quantization). The 1.56 TB download size is itself a direct result of the 4-bit format.
- Limitations: the team has released only this one precision version, with no BF16 or FP8 release. This means the hardware must natively support dequantization and compute in the MX family of microscaling formats. If an accelerator does not support them, the framework has to emulate the dequantization. VRAM usage could then double and throughput could drop sharply, which would completely change what “8 GPUs” actually means.
3. Reading the capability benchmarks: how to interpret the vendor’s comparison table
The model card’s self-reported results are listed below. All figures come from the vendor’s own testing and have not been independently reproduced.
| Benchmark | This model | Closed-source flagship A | Closed-source flagship B |
|---|---|---|---|
| Terminal command coding | 65.7% | 66.4% | 58.2% |
| Long-horizon software engineering tasks | 51.7% | 46.3% | — |
| Scientific reasoning | 38.1% | 63.3% | — |
| General academic knowledge | 39.4% | 60.9% | — |
By the vendor’s account, the model outperforms flagship A on 16 of 37 benchmarks. A third-party benchmark aggregator gives it an intelligence index of around 55, which ranks it fourth globally.
The conclusions should be read cautiously. The capability profile is clearly uneven:
- Coding and agent tasks are close to, or above, the closed-source flagships.
- Scientific reasoning, advanced academic knowledge and some reasoning benchmarks trail by more than 20 percentage points.
The right way to make a selection is as follows:
- Match against your workload distribution, not an overall leaderboard rank. If your workloads are code generation, tool calling and multi-step agent orchestration, these weights may offer better cost-performance than a closed-source API. If they are scientific reasoning or complex mathematical derivation, the gap will directly affect delivery quality.
- Watch for systematic bias in self-reported data. The evaluation framework version, prompt templates, number of few-shot examples, sampling parameters and verifier strictness can each cause swings of several percentage points. Before launch, be sure to run an A/B comparison on your own business datasets.
- Validate agent capabilities separately. Failures in long-horizon tasks often come not from individual step correctness, but from error accumulation and context pollution. Only an end-to-end run of the complete workflow can assess them properly.
4. License analysis: open weights are not open source
This is the most overlooked part, yet it carries the highest commercial risk. The repository uses a custom license, and its metadata labels it as “Other” rather than a standard open-source license such as MIT or Apache. The license text is word-for-word identical to the previous generation’s, with only the name changed.
4.1 The main terms are a permissive, open-source-style grant
The main terms already grant free use, copying, modification, distribution, fine-tuning and commercial use. For the vast majority of teams, this means internal use, embedding in a product, fine-tuning and closed-source distribution are all unrestricted.
4.2 Two additional constraints target large-scale commercial use
- Model-as-a-service (MaaS) revenue threshold: an entity that offers the model as a service to external customers and has cumulative revenue of more than roughly USD 20 million over 12 months must sign a separate agreement with the model team before commercial use.
- Product scale threshold: an entity whose related commercial products have monthly active users (MAUs) above an agreed threshold likewise needs separate authorization.
4.3 Compliance recommendations
Whether you trigger these clauses depends not on whether you “use” the model, but on who owns the revenue and how user scale is measured:
- Revenue attribution: does the revenue threshold apply to the entire company or only to that product line? Are you operating your own API, or wrapping and reselling the model? These two questions determine how the threshold is calculated. It is advisable to have legal counsel issue a formal determination.
- Measurement window: the threshold is usually measured over a rolling 12-month period. Business growth can push you over the line in a given quarter, so an early-warning mechanism is needed.
- Derived models: whether a model that has been fine-tuned remains bound by the original license depends on how the license defines a “derivative work.” Do not assume either way.
- Overseas distribution: this adds further layers of requirements, including export controls, cross-border data transfers and generated-content compliance.
In one sentence: this is a hybrid model of “open weights plus commercial conditions.” Small and medium-sized businesses can use it commercially with freedom, while players scaling up should set aside time for legal and business negotiations.
5. Self-hosting: turning “8 GPUs” into a VRAM budget
5.1 The static calculation: weights cannot be compressed
The 1.56 TB of weights is a hard constraint. Start with the basic arithmetic:
Available VRAM = number of GPUs × per-GPU VRAMStatic headroom = available VRAM - weight size
Suppose each GPU has around 141 GB of VRAM. Eight GPUs then provide roughly 1,128 GB (about 1.1 TB), which cannot hold the 1.56 TB of weights. An “8-GPU minimum” therefore usually implies one of the following:
- Accelerators with around 192 GB of VRAM per card (8 × 192 GB ≈ 1.5 TB, which is still extremely tight).
- Offloading to host memory or storage. In this case, token generation speed drops sharply because PCIe bandwidth becomes the bottleneck.
Also pay attention to how sizes are reported:
- Repository sizes are usually counted in binary units.
- Actual VRAM usage also includes framework metadata, alignment padding and quantization scale factors. In practice, it is often 3–8% higher than the stated download size.
Conclusion: 8 GPUs are only a theoretical floor. For production, plan for 12–16 GPUs to leave room for the KV cache and memory fragmentation.
5.2 The dynamic calculation: the KV cache grows linearly with context length
Long context is a core selling point of this class of model, and also its biggest memory drain. The KV cache can be estimated as follows:
KV_bytes ≈ 2 × L × n_kv_heads × d_head × bytes_per_elem × seq_len
Here, L is the number of layers, and the factor of 2 accounts for the separate key and value caches. Using sample parameters (L = 93, n_kv_heads = 8, d_head = 128, BF16 storage):
- 32K context: about 6 GB
- 128K context: about 24 GB
- 1M context: about 190 GB
At a 1M-token context, the actual VRAM requirement is on the order of a second copy of the weights.
In practice, several techniques need to be combined:
- KV cache quantization (FP8/INT8): compress the cache by more than half.
- PagedAttention-style paging: reduce fragmentation.
- Prefix caching: reuse system prompts and long-document prefixes.
- Sparse attention: it lowers compute, but has only a limited effect on KV storage.
5.3 Choosing a parallelism strategy
Two typical routes exist for configuring parallelism with this kind of extra-large MoE model:
- Tensor parallelism (TP): split each layer’s weights across multiple GPUs. TP is communication-intensive but mature in implementation. It is well suited to high-bandwidth NVLink interconnect within a single node.
- Expert parallelism (EP): distribute the 896 experts across different GPUs, with only 16 activated per token. EP moves less data and scales well, but it is sensitive to all-to-all communication and routing load balance. Hotspot experts can overload individual GPUs.
In practice, teams often combine TP × EP with data-parallel replicas. A sample configuration:
# Sample inference service parallelism config (illustrative; tune to your GPUs and framework)model:tensor_parallel_size: 8 # tensor parallelism within a nodeexpert_parallel_size: 8 # expert parallelism dimensiondtype: mxfp4 # requires native hardware supportmax_model_len: 131072 # derive from available VRAM; don't max out at 1Mkv_cache_dtype: fp8 # compress the KV cacheenable_chunked_prefill: true # chunk long prompts so prefill doesn't block decodeenable_prefix_caching: true # reuse shared prefixesgpu_memory_utilization: 0.90scheduler:max_num_seqs: 64 # concurrency limit; balance throughput vs. VRAM for MoEmax_num_batched_tokens: 32768
The key trade-off: context length is not free. Raising max_model_len from 128K to 1M makes KV VRAM grow 8×. That alone can cut serving concurrency to single digits. Before launch, be sure to set this value based on the distribution of real prompt lengths, rather than configuring it to the model’s advertised maximum.
5.4 Minimum viable validation process
Before committing to a hardware budget, run small-scale validation in this order:
- Format and kernel validation: confirm that the framework and accelerators natively support the MX family of 4-bit formats. After loading the weights, check whether actual VRAM usage matches the theoretical value.
- Accuracy alignment validation: compare the outputs of your self-hosted deployment against the API on your own evaluation set. Focus on whether quantization degrades quality on long-text tasks.
- Long-context stress test: run prefill at 1.2× your target context length. Measure time to first token and peak VRAM usage.
- Concurrency knee-point test: increase concurrency step by step until throughput stops rising and latency climbs sharply. Use that turning point to size capacity.
- Final license review: enter production only after legal counsel issues a written conclusion.
5.5 Self-hosting vs. cloud hosting
| Dimension | Self-hosted cluster | Cloud hosting / managed service |
|---|---|---|
| Initial investment | High: purchase a multi-GPU system up front | Low: pay as you go |
| Hardware requirements | Must match MX-family 4-bit support | Adaptation handled by the platform |
| Operational burden | Drivers, frameworks and failed-GPU replacement are all on you | Handled on the platform side |
| Elasticity | Weak: long lead times to scale out | Strong: scales with traffic |
| Data sovereignty | Fully under your control | Depends on the deployment model |
| Long-term unit cost | Better at high utilization | Better with fluctuating workloads |
The decision rule is simple:
- Self-hosting is more cost-effective when utilization is stable and stays above 50–60% over the long term, and when keeping data on the internal network is a hard requirement.
- Cloud hosting has a lower total cost when workloads have pronounced peaks and valleys, or when the team lacks experience operating large-scale distributed inference.
On the cloud path, object storage is typically used for weight distribution, container platforms for orchestrating inference services, and monitoring, alerting and logging services for observability. All of these are standard cloud capabilities rather than vendor-specific solutions.
6. Summary: an actionable decision checklist
- Prioritize capability fit over parameter count. 2.8T does not mean an across-the-board lead. Coding and agent workloads are worth evaluating closely, while scientific reasoning and advanced knowledge tasks need careful validation.
- Address license risk up front. The custom license is friendly to small and medium-sized businesses. However, the dual revenue-and-MAU thresholds mean that any path to scale needs planning in advance, and legal involvement cannot wait until after launch.
- Do the VRAM math twice. The static weights take about 1.56 TB, and “8 GPUs” is an idealized floor. Plan for 12–16 GPUs in production. Budget separately for the linear growth of the KV cache at your target context length.
- Hardware support is a hidden barrier. Only a 4-bit quantized version has been released. If your accelerators do not support the corresponding format, performance and VRAM usage will deviate sharply from the vendor’s figures.
- Context length is a cost switch. Do not configure the service to the advertised 1M-token context. Set it from the distribution of real prompt lengths, then combine KV quantization, paging and prefix caching to control costs.
- Validate at small scale before purchasing. Complete the five steps before signing any hardware contract: format support, accuracy alignment, long-context stress testing, concurrency knee point and final license review.
For a team, the real value of these weights lies not in being “the largest.” The release pushes open-source capability into the top tier for agent and coding workloads. At the same time, it puts two long-undervalued issues, license compliance and VRAM cost, squarely on the table.
