# Deploying the 2.8 Trillion Parameter Kimi K3.1 Pro MoE Model Locally: What's in the 1.56 TB Weights, How the License Works, and What Hardware It Takes

    This article examines the release of the open weights for Kimi K3.1 Pro, a 2.8 trillion parameter MoE model with 104B activated parameters, published as 112 files and 96 shards totaling about 1.56 TB with a single MXFP4/MXFP8 quantized build. It explains why open weights are not the same as open source and walks through the custom license's permissive core and its large-scale commercial thresholds, then breaks down the architecture, the sparse activation and sparse attention design, the KV cache and parallelism math behind the "8 high-memory GPUs minimum" rule of thumb, three practical deployment routes, and a decision checklist for teams weighing self-hosting against API use.

    1. What Actually Happened: After the Weights Landed, the Real Questions Begin

    With the model race now entering the “trillion parameter” phase, a model development team launched its new flagship model, Kimi K3.1 Pro (hereafter K3.1 Pro), in late October, first through an API and a web interface, and promised to publish the full weights before November 4. On November 4, the weights appeared as promised on a major open-weight model hosting platform, publicly downloadable without an application process.

    From an engineering standpoint, a few details of this release are worth noting:

    • The weight repository was created in advance and kept closed until release day, then opened that evening, which suggests the release cadence was deliberately controlled.
    • 112 files, 96 weight shards, roughly 1.56 TB in total — the largest publicly downloadable open-weight model to date, far larger than any comparable predecessor.
    • Only one quantized build was published, with no higher-precision alternative.

    The model’s capabilities and benchmark scores at launch in late October have already been covered extensively, so this article does not repeat them. Instead, it focuses on the three questions developers care about once the weights actually land: what is inside, how the license works, and whether ordinary setups can run it.

    2. First, Clear Up a Concept: Open Weights Is Not Open Source

    This is the most easily misread aspect of the release.

    On the hosting platform, the model’s license tag is license:other — “other,” not a standard open source license such as MIT or Apache-2.0. What the team published is a custom Kimi K3.1 Pro License, word-for-word identical to the license of the previous-generation Kimi K3 apart from the name.

    2.1 Core Clauses: Close to a Permissive License

    The main body of the license follows a grant style similar to MIT: anyone may use, copy, modify, distribute, fine-tune, and commercially use the model and its outputs and derivative models, free of charge. For the vast majority of individual developers and small-to-medium teams, these terms impose almost no barrier.

    2.2 Additional Clauses: Only Constraining Large-Scale External Business

    On top of that permissive core, the license adds two restrictions, and both apply only to scenarios involving large-scale external operations:

    1. Model-as-a-service (MaaS) revenue clause: a company offering the model as a service must sign a separate agreement with the model provider before commercial use if its cumulative revenue over any 12 months exceeds USD 20 million.
    2. Monthly active user clause for commercial products: once the monthly active users of a commercial product exceed a certain scale, separate authorization is required as well. (The source material breaks off while describing this clause; the specific threshold should be taken from the license text itself.)

    Example code (Python, for self-checking license thresholds; revenue in USD, MAU in units of 100 million users):

    1. MAAS_REVENUE_LIMIT_USD = 20_000_000 # 12-month cumulative revenue threshold
    2. def needs_separate_agreement(is_maas: bool, revenue_12m_usd: float,
    3. mau_yi: float, mau_limit_yi: float) -> bool:
    4. """Read mau_limit_yi from the license text. Do not fill it in from memory."""
    5. if is_maas and revenue_12m_usd > MAAS_REVENUE_LIMIT_USD:
    6. return True
    7. return mau_yi > mau_limit_yi

    2.3 Is the License “Supported”? Look at the Source, Not Just the Label

    This kind of “permissive core plus commercial thresholds” licensing is not unusual when very large parameter models are opened up. Model providers are generally willing to let research and small-to-medium commercial use benefit fully, while keeping room to negotiate around large-scale commercialization. Developers should note three things:

    • The license text itself contains no consequences for breach and no termination clause. If a hosting platform provides only the license file without additional terms, most compliance scanning tools will flag “unsupported license” and list warnings such as “license not provided” and “patent grant possibly undefined.” The safer approach is to read the license text clause by clause.
    • “Open weights” and “open source” are two different levels of openness. The former publishes trained parameters; the latter typically also requires publishing training code, data processing pipelines, and the complete training methodology, measured against widely recognized standards such as the OSI open source definition. K3.1 Pro is a typical open-weights release, not open source in the strict sense.
    • Check the counting unit. When determining whether the second restriction is triggered, confirm whether the original text counts monthly or daily active users, and whether it refers to a single product or an entire product portfolio, and re-verify periodically.

    3. Model Architecture: How the 2.8 Trillion Parameters Add Up

    3.1 Key Specifications at a Glance

    Dimension Specification
    Positioning Native multimodal agent model, continued training from Kimi K3
    Total / activated parameters per token 2.8T / 104B
    Expert configuration 896 routed experts, 16 activated per token; plus 2 shared experts
    Layers 93
    Context length 1 million tokens
    Vision encoder MoonViT-V2
    Structural change vs. K3 24 of the full-attention layers replaced with sparse attention (DSA)
    Numerical precision Weights MXFP4, activations MXFP8, produced by quantization-aware training

    3.2 Sparse Activation: Why Local Deployment Is “Bottlenecked on VRAM, Not Compute”

    K3.1 Pro uses a classic sparse mixture-of-experts (MoE) architecture. Dense parts such as attention and the shared experts apply to all tokens, while routed experts are selected per token by a gating network — 16 of them per token here.

    The “sparsity” applies only at the compute level. At the storage level, every parameter still has to be held somewhere. As a result:

    • Forward-pass compute scales roughly with the 104B activated parameters, comparable to a dense model in the tens-of-billions to low-hundreds-of-billions range, which a single GPU can handle in terms of raw throughput.
    • VRAM consumption depends on all 2.8 trillion parameters.

    This is the fundamental reason for the “8 GPUs minimum” rule of thumb: for this class of model, the first bottleneck is VRAM capacity, and compute comes second.

    3.3 Sparse Attention: Saving Compute for Million-Token Context

    Relative to K3, the only architectural change in K3.1 Pro is replacing full attention with sparse attention (DSA) in 24 of the 93 layers. In standard full attention, every token attends to all preceding tokens pairwise, and the cost grows quadratically with sequence length. Scaling a sequence from 32K to 1 million tokens multiplies the length by roughly 32, which theoretically increases attention compute by about 1000x.

    The idea behind sparse attention is to let each token attend only to a small number of carefully selected tokens or token blocks, bringing the cost down to roughly linear in sequence length. For context windows at the million-token scale, this kind of change can substantially reduce the compute pressure of long-sequence inference, though at the cost of greater implementation complexity and possible precision loss in fine details.

    3.4 Quantization: MXFP4 Weights Plus MXFP8 Activations

    The model’s weights use MXFP4 and its activations MXFP8, the product of quantization-aware training (QAT). MX (Microscaling) formats are a family of low-precision numeric formats that split data into small blocks (for example, groups of 32 elements) that share a common exponent. They preserve dynamic range as far as possible at extremely low bit widths, and are now widely supported by mainstream AI accelerator hardware.

    Accounting for quantization error during training usually loses less precision than a “train first, quantize later” approach. But it must be emphasized: only this single build was released — there is no BF16 or FP8 version. This means developers cannot choose a higher-precision deployment path and are limited to inference, and to limited fine-tuning, on top of 4-bit weights.

    3.5 Capability Benchmarks: Vendor-Reported, Read With a Third-Party Lens

    All figures in the table below come from the model vendor. The comparison targets are two current mainstream closed-source flagships, labeled A and B; missing cells indicate the public information did not cover that entry.

    Benchmark (proper nouns removed from the source data) K3.1 Pro Closed-source flagship A Closed-source flagship B
    Coding (terminal-task benchmark) 65.7% 66.4% 58.2%
    Coding (long-horizon software engineering benchmark) 51.7% 46.3% —
    Science (terminal experiment benchmark) 38.1% 63.3% —
    Knowledge and reasoning (hard multidisciplinary QA benchmark) 39.4% 60.9% —

    The vendor claims K3.1 Pro outperforms closed-source flagship A on 16 of 37 benchmarks. A separate independent evaluation gave it a composite intelligence index of 55, ranking fourth globally.

    Three conclusions follow from the table:

    1. Coding and agent capabilities are first-tier. On standard coding tests it trails closed-source flagship A only slightly, and on the long-horizon software engineering test it leads by about 5.4 percentage points.
    2. Science and parts of reasoning are clear weak spots. It trails by more than 25 percentage points on the science experiment test and by more than 20 points on the hard knowledge-reasoning test.
    3. The overall picture is mixed. It suits workloads centered on code generation, tool calling, and long-running automation; for business with heavy demands on scientific computing and complex reasoning, validate against your own evaluation set before committing.

    4. Where the 1.56 TB Comes From: The VRAM Math for Local Deployment

    4.1 Estimating and Sanity-Checking the Weight Size

    In MXFP4 format, each parameter takes roughly 0.5 bytes. A rough calculation puts 2.8 trillion parameters at about 1.4 TB; the gap versus the actual file size comes from two sources:

    • Router, normalization layers, the vision encoder, and other parts unsuitable for 4-bit quantization, which are still stored at higher precision;
    • Sharding, alignment, and metadata overhead.

    After downloading, you can run a magnitude check with the script below (parameter-count based, independent of the specific directory layout):

    Example code (Python, order-of-magnitude check):

    1. def expected_weight_size_tib(total_params: float, bits: int = 4,
    2. dense_ratio: float = 0.03,
    3. dense_bytes: int = 2) -> float:
    4. """dense_ratio: share of all parameters still stored in BF16. Sample assumption only."""
    5. q = total_params * (1 - dense_ratio) * bits / 8
    6. dense = total_params * dense_ratio * dense_bytes
    7. return (q + dense) / 1024**4 # 1 TiB = 1024^4 bytes
    8. # ~1.47 TiB, in the same range as the official ~1.56 TB (decimal, about 1.42 TiB)
    9. print(round(expected_weight_size_tib(2.8e12), 2))

    4.2 The VRAM Budget: Spreading 1.56 TB Across GPUs

    Component Nature Deployment strategy
    Model weights (~1.56 TB) Static, can be pre-sharded Tensor parallelism plus expert parallelism, spread evenly across GPUs
    KV cache Dynamic, grows linearly with sequence length and concurrency Cap maximum context and concurrency; quantize or compress if needed
    Activations and temporary buffers Dynamic, depends on batch size Reduce batch size, enable chunked prefill
    Communication and framework overhead Fixed plus variable Reserve headroom, typically an extra 10%–20% of VRAM

    In practice, a common approach is to keep the weight footprint at 70%–80% of per-GPU VRAM, leaving the rest for KV cache, activations, communication buffers, and framework overhead, so the process does not crash from VRAM exhaustion at runtime.

    On a single 8-GPU machine, that works out to roughly 195 GB of weights per card — more than the combined VRAM of two 80 GB-class GPUs on its own. So the “8 GPUs minimum” premise holds only if:

    • You use 8 high-memory accelerators, for example single cards at the 192 GB level, totaling more than 1.5 TB of VRAM.
    • If you only have 8 cards of the 80 GB class (640 GB total), you must offload weights to host memory or even local disk, and inference speed drops sharply, no longer suitable for interactive use.

    4.3 Million-Token Context: KV Cache Is the Hidden VRAM Hog

    Example code (Python, KV cache estimate):

    1. def kv_cache_gib(layers: int, kv_heads: int, head_dim: int,
    2. seq_len: int, bytes_per_elem: int) -> float:
    3. # One copy each for K and V
    4. n = 2 * layers * kv_heads * head_dim * seq_len * bytes_per_elem
    5. return n / 1024**3
    6. # KV head counts and head dims below are assumptions, to demonstrate the calculation
    7. for heads in (1, 4, 8):
    8. print(heads, round(kv_cache_gib(93, heads, 128, 1_048_576, 2), 1), "GiB")
    9. # Output: 1 44.5 GiB / 4 178.1 GiB / 8 356.2 GiB

    Taking 93 layers as an example and assuming grouped-query attention, the KV cache alone for a single 1-million-token sequence at FP16 precision can consume anywhere from 45 GB to 356 GB. Even if sparse attention reduces how much KV data is actually read, this memory still has to stay resident. So during deployment you should:

    • Explicitly cap the maximum context per request, rather than exposing the full 1 million tokens by default.
    • Enable prefix caching so requests sharing a system prompt can reuse KV.
    • Enable chunked prefill to avoid a long prompt monopolizing VRAM in one shot.

    4.4 Parallelism Strategy: Combining TP, PP, and EP

    Deploying this class of model usually means combining three forms of parallelism:

    • Tensor parallelism (TP): splits each layer’s matrix operations across GPUs. Communication-heavy, so it suits intra-node environments with high inter-GPU bandwidth.
    • Pipeline parallelism (PP): cuts the model into stages by layer. Less communication, but pipeline bubbles appear and single-request latency is higher.
    • Expert parallelism (EP): distributes the 896 experts across GPUs, with each token’s results aggregated via All-to-All communication, whose volume grows with sequence length.

    On a single 8-GPU node, one common approach is a “TP × EP” combination, running dense parts under tensor parallelism and routed experts under expert parallelism, while avoiding cross-node pipeline bubbles. Also watch for expert load imbalance: if a few hot experts concentrate on one card, that card’s throughput drags down the whole node, so routing load-balancing mechanisms are needed.

    5. Three Practical Deployment Routes

    Route Hardware requirements Suitable for Main trade-off
    Single-node 8 high-memory GPUs 8 GPUs, roughly 1.6 TB of VRAM in total Interactive inference, agent services Large hardware investment, relatively simple deployment
    CPU/GPU hybrid offloading High-memory host plus a few GPUs Validation, low-frequency batch jobs Low hardware barrier, low throughput and high latency
    Elastic cloud compute Multi-GPU cloud instances provisioned on demand Elastic workloads, teams without datacenter facilities No ops burden, watch long-term cost

    Routes one and two emphasize full local deployment under your own control; route three suits teams without large-scale GPU infrastructure. On cloud platforms such as Baidu AI Cloud, you can obtain single-node multi-GPU compute through GPU cloud servers, pre-stage the 1.56 TB of weights in object storage, mount them quickly onto compute nodes, and handle deployment and operations with container services, log services, and monitoring alerts.

    Whichever route you pick, the download step deserves attention. For 1.56 TB in total, use a download tool that supports resumable transfers and concurrent verification, and leave at least twice the weight volume in disk space (download cache plus extracted copies). The weights are stored as 96 shards, and all shards must be present before loading will work at all — a single missing shard prevents startup.

    6. A Decision Checklist for Developers

    1. Ask about requirements first, hardware second. Open weights are usually wanted for keeping data in-house, deep fine-tuning, or cost control. If you only need to call an API, open weights add limited value.
    2. Do the full accounting. An 8-GPU high-memory server is a large one-time investment, on top of datacenter, power, and operations costs. Self-hosting only beats API calls when utilization is high enough.
    3. Know your limits. Teams without enough VRAM should first try smaller open models or lower quantization, or simply use cloud compute. Do not force an “8 GPUs into 2” setup.
    4. Take compliance seriously. Personal use, research, and small-to-medium commercial use are essentially unrestricted; large MaaS providers and products with very high monthly active users should discuss authorization with the model provider in advance.
    5. Build your own evaluation. Vendor benchmarks are only a reference point. Assemble an evaluation set from real business data, with particular attention to the weak areas.
    6. Watch the ecosystem. Only the MXFP4 build is available, and the model’s sheer size means fine-tuning requires infrastructure at nearly a trillion-parameter scale. For most teams, the sensible approach is to focus on inference services, complemented by prompt engineering and parameter-efficient fine-tuning.

    7. Conclusion

    Opening the K3.1 Pro weights turns “can trillion-parameter models be deployed locally?” from a theoretical question into an engineering one. The answer is: yes, it can run, but the bar is explicit — at least 8 high-memory accelerators, roughly 1.6 TB of VRAM, and the matching high-speed interconnect, plus a license that carries conditions aimed at large-scale commercialization.

    For developers, this release offers at least three takeaways:

    • See where “open” actually ends. Open weights are not open source; read the license clause by clause before commercial use.
    • Understand sparse model characteristics. Compute scales with activated parameters, VRAM with total parameters, so deployment planning starts from the VRAM math.
    • Treat vendor benchmarks rationally. Use open weights as an opportunity to base selection decisions on your own evaluations and real cost.