2.8T Open-Weight MoE Model Released: Deployment and Licensing Guide

    A team has released the full weights of K3.1 Pro, a 2.8-trillion-parameter Mixture-of-Experts (MoE) model, with the files totaling about 1.56 TB. This guide explains what the model is made of—its MoE routing, MoonViT-V2 vision encoder, sparse attention, and MXFP4 quantization—and how to estimate the minimum VRAM and bandwidth needed for local deployment. It also covers how to read the vendor's self-reported benchmarks, why open weights are not the same as open source under a custom license, and how to choose among four deployment options. The bottom line: 8 high-VRAM cards is only the theoretical floor, and large-scale commercial use calls for a license review first.

    1. Weight Release: Cadence and Usage Notes

    A team has publicly released the full weights of a very large Mixture-of-Experts (MoE) model, K3.1 Pro. According to the official release information, the files total roughly 1.56 TB. The rollout followed three broad steps:

    1. Service first: The model went live through an API and a web interface, with the team committing to open the full weights within a set period.
    2. Repository prepared: After the weight repository was created, downloads were not enabled immediately.
    3. Downloads opened: Once officially open, downloads required no application. The repository contains 112 files, 96 of which are weight shards averaging about 16 GB each—convenient for resumable and parallel downloads.

    Based on currently available information and the model card, this is one of the largest open-weight MoE models by parameter count. Users can follow the official channels and community mirrors. One point deserves special attention: prefer the official repository. If you use a mirror, verify the official hash or signature, and watch for tampered files, bundled installers, or supply-chain poisoning.

    2. Inside the Model: What Makes Up 2.8 Trillion Parameters

    1. MoE structure: total vs. active parameters

    K3.1 Pro is a natively multimodal Agent model with visual understanding and task-execution capabilities. Its total parameter count is 2.8T, but only 104B are activated per token—about 3.7% of the total. The routing configuration is:

    • Routed experts: 896, of which 16 are selected per token—about 1.8% of all experts;
    • Shared experts: 2 additional, which every token passes through;
    • Network depth: 93 layers;
    • Context length: 1 million tokens.

    It helps to keep two sets of numbers separate: total parameters determine VRAM usage, while active parameters determine the compute per token. During inference, each generated token only needs to load the selected expert weights. As a result, the 2.8T total parameter count significantly raises the VRAM threshold, but does not raise the compute requirement proportionally.

    2. Vision encoder and sparse attention

    The model includes a built-in vision encoder, MoonViT-V2, which processes image inputs directly—images do not need to be converted to text before being handed to the language model.

    Compared with the previous generation, the main architectural change is that 24 of the 93 layers switch from full attention to sparse attention (DSA), while the other 69 remain full attention. This change mainly targets long-context scenarios and reduces the computational complexity of attention. Note, however, that sparsification only saves attention compute; during inference the KV cache must still be kept in full.

    3. Quantization route: MXFP4 weights and precision

    The officially released weights are MXFP4; activation or compute precision may use MXFP8, depending on the inference framework implementation. The MX (Microscaling) format shares a scaling factor across small data blocks, preserving numerical precision as much as possible at a 4-bit width. This version is likely the product of quantization-aware training (QAT), which simulates low-precision errors during the training phase.

    Key implications include:

    • The precision loss from 4-bit is baked into the weights and cannot be avoided by loading a different-precision version;
    • The inference framework must fully support the MX format, or it cannot load the model;
    • Fine-tuning must be done within a quantization framework, which is harder than with a standard BF16 model;
    • How much expert weight is read per token mainly depends on the weight quantization bit-width. Estimating at 0.5 bytes per parameter for MXFP4, the 104B active parameters come to about 52 GB; if the inference framework represents expert weights internally at 8-bit, the read volume is about 104 GB. Verify against the actual implementation and load testing.

    4. How to calculate the minimum VRAM for local MXFP4 MoE deployment

    A three-part formula works:

    1. minimum VRAM ≈ weight VRAM + KV cache + runtime overhead
    2. weight VRAM ≈ total parameters × bytes per parameter
    3. KV cache ≈ 2 × layers × KV heads × head dim × bytes per element × context length
    4. runtime overhead ≈ framework, communication buffers, activations, VRAM fragmentation, and other reserves

    Estimating 2.8T parameters at 4-bit:
    2800 × 0.5GB ≈ 1400GB, i.e., about 1.4 TB. The 1.56 TB download also includes the router, the vision encoder, quantization scaling factors, and other extra content. Eight 192GB high-VRAM cards come to about 1.536 TB, which roughly matches the weight size but leaves almost no headroom. Adding the KV cache, activations, and framework overhead, a safer production floor is 9–10 cards of the same spec, or approaches such as KV quantization, multi-node deployment, or CPU offloading.

    3. Reading the Results: How to Judge Vendor-Reported Data

    The following are self-reported benchmark figures from the vendor’s release materials, compared against two contemporaneous flagship closed-source models. Exact scores and evaluation setups should be verified against the official model card and evaluation report.

    Benchmark dimension K3.1 Pro Comparison Model A Comparison Model B
    Terminal-command coding tasks 65.7% 66.4% 58.2%
    Long-horizon software engineering tasks 51.7% 46.3% —
    Hard science problems 38.1% 63.3% —
    High-difficulty reasoning / knowledge 39.4% 60.9% —

    The vendor states that it surpassed Model A on 16 of 37 benchmarks. Overall, coding and Agent tasks are roughly on par with or ahead of the comparison, while science and some reasoning tasks still lag.

    Three principles are worth following when interpreting such data:

    1. Distinguish self-reported from re-tested: Vendors tend to pick favorable metrics, so conclusions should rest on your own evaluation.
    2. Watch for benchmark saturation and contamination: Public test questions may have been included in training data, which can inflate scores.
    3. Return to your business scenario: For selection, prioritize small-sample A/B comparisons on a private test set that is close to real business use, rather than chasing aggregate rankings.

    4. Licensing: Open Weights Are Not Open Source

    This is the point most worth calling out separately in this release. The repository carries a custom-named model license, whose label in mainstream model communities is typically “Other (custom),” not a standard open-source license like MIT or Apache. Its terms can be summarized as “MIT-style grant + additional restrictions.”

    Main grant: Free to use, copy, modify, distribute, fine-tune, and use commercially—consistent with the core of the MIT license.

    Additional restrictions: These mainly target larger external-facing businesses. For example, if the model is offered as an external service and cumulative revenue over the trailing 12 months reaches a certain threshold, or if a commercial product’s monthly active users reach the hundred-million level, a separate authorization may be required before commercial use.

    Note: the above thresholds are only illustrative summaries; the exact terms are governed by the full official license text, and legal review is recommended. The license file is usually found in the LICENSE file of the official weight repository, the model card, or the release notes. Archive the full license text and the weight version number before adoption.

    Use case Affected? Notes
    Personal research and learning Essentially no Ready to use directly
    Most enterprise on-premises / private deployments Essentially no Not offering the model as a service externally
    Startups offering a model API externally Usually not No separate agreement needed below the threshold
    Large platforms with large-scale commercial use Legal review needed Confirm the licensing path in advance

    Three more points are worth noting:

    • Attached obligations still apply. MIT-style terms usually require retaining the copyright and license notice, and the additional restrictions are themselves part of the license.
    • Distribution of modified versions is also covered. When re-distributing derivative weights that have been fine-tuned, quantized, or distilled, the original license generally still applies; specifics are governed by the full license text.
    • This is strictly open weights. The industry’s strict definition of “open source” usually also requires opening training code, data documentation, and even the training data itself. A release that opens only the weights and attaches commercial thresholds is more accurately described as open weights.

    5. Can You Run It Locally: 8 Cards Is Only the Theoretical Floor

    The conclusion up front: 8 high-VRAM cards is the theoretical starting configuration; production environments are advised to reserve 9–10 cards of the same spec. Long-context scenarios also require KV quantization, tiered caching, or multi-node deployment.

    1. VRAM: 8 cards just barely fit

    Estimating at 4-bit, the weights are about 1.4 TB. Eight 192GB cards total about 1.536 TB, theoretically enough to hold the weights, but with insufficient headroom for the KV cache, activations, and framework overhead.

    1. def estimate_moe_memory(total_params_b, active_params_b, bytes_per_param,
    2. context_len, layers=93, kv_heads=8, head_dim=128,
    3. kv_bytes=2):
    4. weights_gb = total_params_b * bytes_per_param
    5. kv_per_token_kb = 2 * layers * kv_heads * head_dim * kv_bytes / 1024
    6. kv_gb = kv_per_token_kb * context_len / 1024 / 1024
    7. active_gb = active_params_b * bytes_per_param
    8. return {
    9. "weights (GB)": round(weights_gb, 1),
    10. "KV cache (GB)": round(kv_gb, 1),
    11. "total (GB)": round(weights_gb + kv_gb, 1),
    12. "expert weight read per token (GB)": round(active_gb, 1),
    13. }
    14. for ctx in (4096, 128 * 1024, 1024 * 1024):
    15. print(ctx, estimate_moe_memory(2800, 104, 0.5, ctx))

    Illustrative results:

    Context length Weights KV cache Total
    4K 1400 GB ~1.5 GB ~1.4 TB
    128K 1400 GB ~46–50 GB ~1.45 TB
    1M 1400 GB ~370–400 GB ~1.8 TB

    The KV head count in the code is an assumed value; the real value should follow the model configuration, but the order of magnitude will not fundamentally change. Conclusions:

    • The official 4-bit version differs from the BF16 version by about 3.6×, so quantization is a prerequisite for single-machine serving;
    • In short-context scenarios, 8 cards can only just fit, with no headroom;
    • To run a full 1 million-token context, VRAM requirements may approach 1.8 TB, requiring at least 10 high-VRAM cards, or techniques such as KV quantization and tiered caching.

    2. Bandwidth: more easily overlooked than capacity

    During MoE decoding, reading the active expert weights is required for every generated token. To stay consistent with the official MXFP4 weights, the bandwidth estimate in this article uses a 4-bit weight-read basis: the 104B active parameters are about 52 GB. If the inference framework represents expert weights internally at 8-bit, the read volume is about 104 GB; verify against actual load testing.

    1. bandwidth_per_card = 8.0 # per-card VRAM bandwidth in TB/s, illustrative value
    2. cards = 8
    3. active_gb = 104 * 0.5 # 4-bit weight read, ~52 GB per token
    4. ideal_tps = bandwidth_per_card * cards * 1000 / active_gb
    5. print(f"Theoretical ceiling: {ideal_tps:.0f} token/s")
    6. for eff in (0.2, 0.3, 0.4):
    7. print(f"Bandwidth utilization {eff:.0%}: ~{ideal_tps * eff:.0f} token/s")

    Actual throughput is usually only 20–40% of the theoretical value. Unbalanced expert routing, insufficient VRAM bandwidth utilization, inter-card communication overhead, the synchronization cost of tensor parallelism and expert parallelism (TP/EP), and the irregular memory access caused by sparse attention all lower effective throughput. The theoretical ceiling cannot be used directly for SLA commitments; capacity planning must be based on business load testing.

    3. Where single-machine ends and multi-machine begins

    • CPU offloading: Using host memory to temporarily hold expert weights can run, but the bandwidth between VRAM and host memory is usually one to two orders of magnitude lower than VRAM bandwidth, so it will be very slow.
    • Multi-node deployment: When VRAM requirements exceed what a single machine can hold, multi-node deployment is needed, and inter-card interconnect and cross-node networking become the new bottleneck.
    • Object storage + on-demand inference: If the weights are placed in object storage and the inference service is called on demand, you can avoid building your own inference environment, but cold-start and scheduling costs need to be evaluated.

    6. Choosing Among Four Deployment Options

    Dimension Vendor API / web Self-hosted inference CPU offloading Object storage + on-demand inference
    VRAM requirement None 8 high-VRAM cards to start Host memory above 1.5 TB Determined by inference nodes
    Response speed Fast, depends on provider Fastest, dedicated resources Slow, limited by memory bandwidth Slower on cold start
    Ops cost Low High Medium Medium
    Suitable scenarios Quick validation, elastic calls High data-compliance needs, deep customization Functionality validation only Weight distribution, on-demand multi-model scheduling

    Follow a three-step decision logic:

    1. Look at requirements first: Consider this kind of very large MoE model only when you need very long context, complex tool calling, or multimodal input; ordinary text tasks can use medium-scale models.
    2. Then look at compliance: Self-hosted deployment has value only when business data cannot leave the internal network or domain fine-tuning is needed.
    3. Finally, calculate cost: Compare self-hosted fixed costs and API variable costs over the same period.

    7. FAQ: Three Common Questions

    Can open weights be used commercially?

    In most cases yes, but it depends on the additional license terms. If it involves large-scale external MaaS services or a high-MAU commercial product, defer to the full official license and get legal review.

    Should ordinary developers self-host?

    Individual developers are generally advised not to self-host directly. 2.8T parameters, 1.4 TB of weights, and an 8-card minimum mean this is an enterprise-grade deployment, so a more suitable approach is to validate with an API first and then decide whether to go private.

    What is the common mistake?

    Looking only at file size while ignoring the KV cache and bandwidth. Being able to fit the weights does not mean being able to serve stably; long context, concurrency, and expert-routing load all significantly affect real-world availability.

    Conclusion

    This weight release offers references at three levels: capability—coding and Agent tasks approach flagship closed-source models, while science and some reasoning tasks still lag; engineering—2.8T parameters, 1.56 TB of weights, and an 8-card minimum mean local deployment is an enterprise-grade endeavor from the start; compliance—open weights are not open source, and a license review is required before large-scale commercial use.

    For developers, more important than “can it run” is “is it worth running.” Clarify the business scenario and compliance boundaries first, then use small-sample evaluation to validate the payoff—this is the most reliable path to adopting this kind of very large MoE model.