2.8T-Parameter Open-Weights Model Kimi-Omni: A Full Deployment Analysis

    Open weights finally reach the trillion-parameter scale. This article breaks down Kimi-Omni, a 2.8T open-weight model, across five dimensions—architecture, weight size, hardware cost, performance interpretation, and licensing—to answer three practical questions: what the model actually is, whether you can run it locally, and whether commercial use is restricted. If you are evaluating a trillion-parameter model for real workloads, this is what you need to know before you download.

    Introduction: What a Trillion-Parameter Open-Weight Release Means

    For application developers, “trillion parameters” can sound distant. But the value of an open-weight model is exactly that it is downloadable, deployable, and fine-tunable. A vendor recently released the complete weights of a very large model, Kimi-Omni: 2.8T total parameters, a repository of roughly 1.56TB, and—according to the official notes—at least 8 accelerators to run locally. It sets a new record for the scale of open-weight models and is the first case to push the complexity of “open weights” this high.

    Two caveats are worth flagging up front: the “open” release carries a custom license with commercial-use conditions, and the hardware barrier is steep. Using Kimi-Omni as a sample, this article examines five dimensions—model composition, weight size, the deployment ledger, performance interpretation, and license boundaries—to answer three questions: what it actually is, whether you can run it, and whether commercial use is restricted.

    1. Background: A Weight Release Delivered as Promised

    The timeline is straightforward:

    1. Service launched first: In late October, the model became available through an API and a web interface, and the vendor promised to release the complete weights within about two weeks.
    2. Released on schedule: In early November, the weights appeared as promised in the public repository of a mainstream model-hosting platform, downloadable without any application.
    3. Went live that night: The repository was created a few days earlier, remained closed through the morning of release day, and officially opened that evening.

    This release contains 112 files, 96 of which are sharded weight files totaling roughly 1.56TB. Split evenly by file count, each shard averages about 16GB. This is the common way to organize open-weight models: weights are split into fixed-size shards and loaded shard by shard with an index file. This makes verification and resumable downloads easy, and it enables parallel reads across multiple cards during deployment.

    At a time when trillion parameters have become standard for leading closed models, open-weight models have long stayed in the hundreds-of-billions range. The Kimi-Omni release pushes that ceiling directly to the trillion level. But whether a model can actually be put to use depends on three things:

    • The architecture determines its capability profile.
    • The weight size and quantization format determine the hardware barrier.
    • The license terms determine where it can be used.

    Let’s break these down one by one.

    2. Model Composition: How 2.8T Parameters Are Organized

    The official model card describes it as a natively multimodal agent model, trained on top of the previous generation. The key specifications are:

    Dimension Specification
    Total parameters 2.8T
    Activated parameters per token 104B
    Expert structure 896 experts, 16 routed per token, plus 2 shared experts
    Layers 93
    Context length 1M tokens
    Vision Built-in vision encoder, supports image input
    Weight / activation precision MXFP4 / MXFP8 (quantization-aware training)

    2.1 Mixture of Experts: 2.8T Parameters, Only 104B Computed per Token

    The model uses a Mixture of Experts (MoE) architecture: the feed-forward layers are composed of many parallel “expert” subnetworks, and a router module selects a few experts to participate in computation for each token. Kimi-Omni has 896 experts, selects 16 per token, and additionally has 2 shared experts that are always active. A rough calculation shows the share of routed experts each token actually invokes:

    16 ÷ 896 ≈ 1.8% (routed experts only)

    The direct consequence is that forward-pass compute is roughly proportional to the 104B activated parameters—about 1/27 of a full 2.8T model. In other words, MoE makes “compute” sparse without making “storage” sparse. The total weight volume is still 2.8T, a fact that runs through every discussion in this article.

    2.2 Million-Token Context: Sparse Attention to Lighten Long Sequences

    Compared with the previous generation, the structure changed in only one place: 24 of the 93 layers replace full attention with sparse attention (DSA).

    Attention compute grows quadratically with sequence length, and this overhead balloons quickly when targeting million-level context. Sparse attention lets each query attend to only a subset of key-value positions rather than computing against every historical token, directly reducing the most expensive step under long sequences. The design chooses to modify only some layers rather than all, typically to balance long-sequence efficiency against global information capture.

    It is worth being precise: sparse attention reduces compute and memory access, but the cached key-value (KV) tensors still grow with layer count and sequence length. The memory pressure from ultra-long context does not disappear (see Section 3).

    2.3 Native Multimodality: Vision Built into the Main Model

    The model has a built-in vision encoder, and image and text inputs are handled by the same model. This “native multimodal” approach differs from the bolt-on scheme of attaching a separate vision module to a language model, and it is better suited to mixed image-text long-context tasks, such as analyzing a full document together with its charts. But vision tokens also consume the context window and the KV cache, so processing large images or video frames requires a dedicated budget estimate.

    2.4 Low-Bit Quantization: Official Only Ships One MXFP4 Version

    Weights use MXFP4 (4-bit microscaling float) and activations use MXFP8 (8-bit microscaling float), both produced by quantization-aware training (QAT). “Microscaling” means one scale factor is shared per 32 elements, preserving as much dynamic range as possible within a 4-bit representation. QAT simulates quantization error during training and generally holds precision better than “train first, quantize later.”

    Note that the vendor did not release a BF16 or FP8 version—only this single MXFP4 build. This has two implications:

    • Less memory pressure, but the precision ceiling is capped by 4-bit. Teams that need full-precision fine-tuning can only work on the quantized base with low-rank adaptation or continued training, or dequantize on their own—and dequantization cannot recover the information that was compressed away.
    • The inference toolchain must natively support MXFP4/MXFP8, otherwise it must dequantize before inference, and memory and bandwidth overhead rise several-fold.

    3. The Deployment Ledger: The Arithmetic Behind 1.56TB and “8 Cards to Start”

    3.1 Weight Size: How the 1.56TB Adds Up

    A simple estimate: 2.8T parameters × 4 bits ÷ 8 = 1.4TB. Add scale factors, indexes, configs, and other metadata, and the 1.56TB repository size checks out. Conversely, this size is also a direct way to verify that the weights are indeed stored at 4-bit. The 96 shards of roughly 16GB each also make multi-card parallel loading and shard-level integrity checks convenient.

    3.2 The VRAM Ledger: 1.56TB Fits into No Single Card

    • Even at 192GB per card on a high-end accelerator, holding all the weights requires roughly 9 cards.
    • At a common 80GB of VRAM, it takes about 20 cards.

    So where does “8 cards to start” come from?

    “8 cards to start” most likely corresponds to a tiered deployment of VRAM plus host memory. Weights stay resident in host memory and are loaded to the GPU on demand during computation. Each card’s VRAM only needs to hold the portion of weights required for compute, plus the KV cache and runtime overhead. MoE’s sparse activation makes this tiering more feasible: each token touches only about 2% of the routed experts, so in theory more weights go “unused” in any given unit of time. The corresponding requirement is that the single machine be equipped with host memory comparable to, or larger than, the weight volume itself.

    3.3 The Cost: Bandwidth Becomes the New Bottleneck

    Tiered deployment saves VRAM but pays for it in bandwidth. Moving data from host memory to VRAM goes over PCIe, whose bandwidth is typically an order of magnitude lower than VRAM bandwidth. The real impact depends on the workload:

    • Long-document summarization, code-base analysis: long inputs, short outputs, most compute spent in the prefill phase, high arithmetic intensity, and relatively insensitive to data movement.
    • Agent tasks: multi-round calls that invoke tools while reasoning—again “read-heavy, write-light.”
    • High-concurrency online service: token-by-token decoding moves the relevant weights again for every token generated, and bandwidth becomes a hard constraint.

    The conclusion: 8 cards to start is suited to “capability validation” and “heavy-input, light-concurrency” scenarios, not to carrying high-concurrency online business. How much concurrency it can actually handle must come from real testing, not from extrapolating a spec sheet.

    3.4 Don’t Forget the KV Cache: Million-Token Context Is a Memory Hog

    Even if the weights fit, a 1M-token context is itself a memory hog. By a quantized example: K and V together come to 128 bytes per token, per layer, per KV head; across 93 layers that is roughly 12KB/token, and 1M tokens need about 11.4GB. This is only an order-of-magnitude example—the actual figure depends on the number of KV heads and the cache precision, and it scales linearly with the number of concurrent requests.

    The VRAM budget can be broken into four parts:

    VRAM requirement ≈ resident weights + KV cache + activations and temporary tensors + inference framework overhead

    The following illustrative code quickly estimates the weight file size and the lower bound of the KV cache for 1M tokens:

    1. TOTAL_PARAMS = 2.8e12 # Total parameter count
    2. BITS_PER_WEIGHT = 4 # Weight quantization bit width (bits)
    3. LAYERS = 93 # Number of layers
    4. KV_DIM_PER_HEAD = 128 # K + V dimensions per KV head (example)
    5. KV_BYTES_PER_ELEM = 1 # Bytes per KV cache element (example: 8-bit)
    6. def weight_size_bytes() -> float:
    7. """Theoretical weight size, excluding metadata such as scale factors; actual value is slightly higher"""
    8. return TOTAL_PARAMS * BITS_PER_WEIGHT / 8
    9. def kv_cache_bytes(seq_len: int) -> float:
    10. """KV cache estimate; defaults to a single KV head, for example magnitude only"""
    11. per_token = KV_DIM_PER_HEAD * KV_BYTES_PER_ELEM * LAYERS
    12. return per_token * seq_len
    13. if __name__ == "__main__":
    14. GiB = 1024 ** 3
    15. w = weight_size_bytes() / GiB
    16. kv = kv_cache_bytes(1_000_000) / GiB
    17. print(f"Theoretical weight size: {w:.1f} GiB") # ~1304 GiB (~1.4 TB)
    18. print(f"1M-token KV cache: {kv:.1f} GiB") # ~11.1 GiB
    19. for gpu_mem in (80, 141, 192): # Single-card VRAM (GB)
    20. print(f"At {gpu_mem} GB per card, weights alone need ~{w * GiB / 1e9 / gpu_mem:.1f} cards")

    4. Performance Data: How to Read the Vendor Leaderboard

    The vendor’s comparison data references two codenamed models, here called Model A and Model B (both closed flagship models). The specific results:

    • Terminal-Bench 4.0: Kimi-Omni scores 65.7%, essentially tied with Model A’s 66.4% and higher than Model B’s 58.2%.
    • SWE Marathon: 51.7%, leading Model A’s 46.3% by about 5.4 points.
    • Terminal-Bench-Science: only 38.1%, behind Model A’s 63.3% by 25.2 points.
    • HLE-Diamond: 39.4%, behind Model A’s 60.9% by about 21.5 points.

    The vendor’s claim is that it beats Model A on 16 of 37 benchmarks—about 40%. A separate report cites a third-party evaluation agency giving an intelligence index of 55, ranked fourth worldwide. Putting both pieces of information together, three judgments emerge:

    1. Strong at “doing,” weak at “thinking.” Coding and long-horizon agent tasks are already close to leading closed models, while scientific reasoning and some hard reasoning still lag noticeably. Match the model to your task profile rather than looking at a total score.
    2. The advantage does not exist across most benchmarks. 16/37 means it does not lead on most benchmarks, so the claim of “comprehensively surpassing closed flagships” does not hold.
    3. A high third-party index ranking does not mean it is best on your task. Aggregate metrics smooth over single-item differences, so in the end you should rely on measurements from your own test set.

    It is also worth noting that all of the above results are vendor-reported, and leaderboard fairness is affected by three factors:

    • Baseline choice: different closed-model versions differ among themselves.
    • Evaluation scaffolding: prompt templates, tool-calling methods, and retry strategies all significantly affect scores.
    • Differences between the hosted service and the weight version: the two may differ in system prompts or configurations.

    The rational approach, then, is to treat the vendor leaderboard as a “candidate list” and do a second round of validation with real business samples.

    5. License Boundaries: Open Weights Is Not Open Source

    This is the point most worth calling out separately. The repository carries a custom model license, labeled as an “other” license type on the hosting platform, not a standard open-source license like MIT or Apache.

    From the structure of the terms:

    • The core is an MIT-style license. It permits free use, copying, modification, distribution, fine-tuning, and commercial use, requires retaining the copyright and license notice, and provides no warranty.
    • Two constraints are added, and they target only large-scale external business:
      • Entities that offer the model as a service (MaaS) externally and whose cumulative revenue over 12 months exceeds $20 million must sign a separate agreement with the rights holder before commercial use.
      • Entities whose commercial product has a monthly active user base above a certain scale (on the order of hundreds of millions) must likewise reach a separate agreement.
    • It is word-for-word identical to the previous generation, with only the name changed. This can be understood as the rights holder continuing its consistent licensing strategy.

    By prevailing standards in open-source governance (such as the Open Source Definition’s requirement of “no restriction on the field of use”), a license with revenue and MAU thresholds does not qualify as an OSI-approved open-source license. A more accurate description is “open weights + custom license.” Users should perform three compliance checks:

    1. Run a checklist: go through your own scenario against both constraints, including business form (whether you offer the model as a service externally), 12-month cumulative revenue, and product MAU.
    2. Keep records: retain the weight source, the download time, and the license text for the corresponding version. If you redistribute or provide a hosted service, attach the full license text.
    3. Track continuously: license terms may change as the model is iterated, so refer to the official latest text.

    6. Who Should Deploy Locally: A Decision Checklist and Recommendations

    Combining architecture, hardware, and licensing, a clear decision guide emerges:

    Team profile Recommended path Key reason
    Has a GPU cluster and ops capability, and data cannot leave the domain Local deployment Data control; suits long-input, agent-heavy tasks
    SMB team wanting to validate quickly Prefer API or hosted inference The investment threshold of 1.56TB weights and 8 cards to start is high
    Offers MaaS externally with revenue above $20M Complete a license compliance assessment first Must sign a separate agreement with the rights holder before commercial use
    Needs full-precision fine-tuning Evaluate carefully The vendor only offers the MXFP4 version

    If you have decided to go the local-deployment route, proceed in this order:

    1. Benchmark: On the target machine, measure first-token latency and decode speed across different PCIe links, and confirm the user experience is acceptable.
    2. Full-chain stress test: Stress at the target concurrency, focusing on KV cache usage, GPU utilization, and memory bandwidth.
    3. Cost accounting: Compare “card-hour cost × actual throughput” against the API unit price, rather than looking only at the hardware purchase price.
    4. Compliance review: Complete the license check from Section 5 before commercial use.

    For teams choosing to host in the cloud, general cloud-platform capabilities can reduce operational complexity:

    • Use elastic GPU compute to scale up and down on demand.
    • Use object storage to hold TB-level weights and inference artifacts.
    • Use container services to standardize deployment and upgrades.
    • Use monitoring and alerting to track VRAM, throughput, and latency in real time.

    Baidu AI Cloud, for example, offers elastic compute, storage, and container products that can serve as a general foundation for such workloads, and teams can evaluate their options against their own needs.

    Conclusion

    Kimi-Omni’s significance goes beyond the label of “largest open-weight model.” It sends three signals at once:

    • On capability: a trillion-level model’s coding and agent capabilities are close to those of leading closed models.
    • On cost: 1.56TB weights and tiered deployment with 8 cards to start mean trillion-level models have entered the stage of “can run locally”—but they are still far from “run freely.”
    • On governance: open weights carrying revenue and MAU thresholds are a reminder that “can be downloaded” does not mean “usable without constraints.”

    For developers, the rational path is to look at the license first, then account the deployment ledger, and finally run an evaluation with your own real dataset. The endpoint of model selection is not a leaderboard ranking, but measured performance in your actual business scenario.