2.8T Open-Weight Model: 1.56TB Weights, Licensing, and the 8-GPU Deployment Threshold
On November 4, an AI team released the full weights of K3.1 Pro, a natively multimodal Agent model with about 2.8T total parameters and a 1.56TB download—described as the strongest openly downloadable model available today. This article examines what happens after the weights land: how the 2.8T parameters and roughly 3.7% activation rate are structured, what the fine-grained MoE and sparse attention mean for compute and memory, why 4-bit quantization-aware training shapes quality, how to read the vendor's coding-strong but science-weak benchmarks, where the custom "open weights" license diverges from open source, and why deployment realistically starts at 8 high-memory GPUs. It includes memory and KV-cache estimation scripts, parallelism configs, and a cloud deployment workflow to help teams judge whether the model is worth running.
Introduction: Trillion-Parameter Models Are Now Downloadable
On November 4, an AI team released the full weights of its latest-generation model, K3.1 Pro, as previously promised—roughly 2.8T total parameters and a 1.56TB download. It has been described as the strongest openly downloadable model available today. This “openness” comes with two caveats: it uses a custom license rather than a standard open-source one, and even for local deployment it requires at least 8 high-memory GPUs.
Open-weight models have mostly stayed in the tens-of-billions range; this release crosses the trillion-parameter threshold directly. This article focuses only on what happens once the weights are downloaded: what’s inside them, how the license works, and whether an ordinary user can actually run them.
1. Event Recap: API First, Weights Later
- Oct 24: K3.1 Pro launched first as an API and web interface; the team promised to release the full weights before November 4.
- Nov 2: The weight repository was created but kept closed.
- Nov 4: The repository was still closed in the morning and opened officially in the evening, downloadable without any application. It contains 112 files, 96 of which are tensor shards totaling about 1.56TB. The weights are also mirrored on major model-sharing hubs.
This “API-first, weights-later” cadence is now common for top open-model releases. Shipping the API early lets developers and evaluation labs verify capabilities sooner, while delaying the weights protects the commercial window. For users, three things matter:
- Release completeness: Open models depend on support from inference frameworks and community tooling. Whether the weights load smoothly matters more than the parameter count itself.
- License boundaries: What determines whether you “can use” it is the license text, not the moment openness is announced.
- Deployment cost: What determines whether you “can afford to use” it is memory, bandwidth, and interconnect conditions.
2. How the 2.8T Parameters Are Composed
According to the model card, K3.1 Pro is a natively multimodal Agent model, continued-trained from the previous generation K3. Key specs:
| Item | Spec | Engineering meaning |
|---|---|---|
| Total params | 2.8T | Determines weight size and memory floor |
| Activated params per token | ~104B | Determines per-token compute and decode speed |
| Expert structure | 896 routed experts, 16 selected per token, plus 2 shared experts | Sparse activation, fine-grained MoE |
| Layers | 93 | Added depth, fewer params per layer |
| Attention | 24 layers switched from full to sparse attention | Reduces long-context compute |
| Context | 1M tokens | High demand on KV cache and memory bandwidth |
| Vision encoding | Paired vision encoder, supports image input | Native multimodal, not bolted on |
| Numerical precision | 4-bit weights, 8-bit activations | Product of quantization-aware training |
1. Activation rate ~3.7%: trading memory for compute
104B/2.8T ≈ 3.7%. Ignoring the quadratic term in attention, each token’s forward pass is roughly 2×104B ≈ 0.21 TFLOPs, equivalent to a 20-billion-parameter dense model. By contrast, a dense model of similar scale needs about 5.6 TFLOPs per token. MoE yields two direct consequences:
- Compute-friendly: Per-token compute is low, so you can get meaningful throughput with fewer cards.
- Memory-expensive: No matter how many experts a token activates, all weights must stay resident in memory.
This is the root reason “local runs need at least 8 cards.”
2. Fine-grained experts: expressiveness vs. routing cost
896 experts with 16 selected per token is the fine-grained MoE the industry often talks about. The benefit is that different tokens are routed to more specialized parameter subspaces, with shared experts backstopping general capability, so compute stays controlled even as parameter count explodes. The cost is mainly communication: every token in every layer routes to 16 experts. In a parallel deployment, cards holding the relevant experts constantly exchange activation vectors; the sparser the compute, the higher the communication share.
3. Cross-checking the weight size
4-bit floats are about 0.5 bytes per parameter, so 2.8T parameters have a theoretical floor of about 1.4TB. The actual 1.56TB indicates additional overhead from the vision encoder, routing gates, and some unquantized tensors (e.g., embedding and normalization parameters). This also confirms the “4-bit weights, 8-bit activations” precision setting. For deployers, the safest approach is to re-total using the shard manifest and the tensor sizes in the model config, rather than relying only on the download page’s total size.
3. Sparse Attention: Saves Compute, Not Necessarily Memory
69 of the 93 layers are still standard full attention. Full attention’s compute and memory footprint both grow quadratically with context length, becoming a major cost in million-token scenarios. K3.1 Pro replaces 24 of those layers with sparse attention. The general approach:
- Lightweight indexing: A tiny scoring network evaluates the relevance of all historical tokens for the current token.
- Top-k selection: Keep only the k highest-scoring tokens.
- Sparse compute: Run exact attention only on the selected keys and values.
For million-level context, if k is fixed at a few thousand, per-token attention compute drops to a few thousandths of the original. The cost is that indexing still scans all history, and that overhead grows linearly with context; whether the benefit materializes depends on how cheap the indexing network is.
One point needs clarification: sparse attention saves compute, not necessarily memory. A naive top-k implementation still needs the full KV cache, which can reach hundreds of GB at a million tokens. Only with sliding windows, per-layer eviction, or cache compression does storage drop in tandem. For capacity planning, estimate conservatively under a “no KV compression” assumption.
4. 4-Bit Weights Only: What Quantization-Aware Training Means
The team released only this one precision version, with no BF16 or FP8 weights. K3.1 Pro uses the industry-standard microscaling 4-bit float format: each value is represented in 4 bits, with a shared scaling factor for quantization blocks of 16 or 32 elements.
Why quantize during training: Post-training quantization (PTQ) faces two problems at the trillion-scale MoE level:
- The calibration set can’t cover all experts, so routing and each expert’s activation distribution are hard to capture fully.
- Errors accumulate layer by layer.
Quantization-aware training (QAT) simulates quantization error during training, teaching the model to be insensitive to rounding. For low-bit weights, this is a common and reliable approach.
Concrete impact for users:
- Memory demand drops to about a quarter of BF16, which is the premise that makes the 8-card floor achievable.
- Quality varies by task: Coding and instruction-following are affected relatively little; multi-step math and scientific reasoning are more sensitive to numerical precision.
- With no BF16 weights, precision loss can’t be recovered afterward. To continue training at higher precision, you can only do quantization-aware fine-tuning on the quantized model, or first dequantize the weights to a higher-precision format.
- Higher toolchain barrier: The inference framework must support this format and its companion attention implementation; writing your own loading logic is basically infeasible.
5. How to Read the Official Scores: Strong at Coding, Weak at Science
All data below is vendor-reported, and the comparison models are also based on officially published figures.
| Benchmark | Focus | K3.1 Pro | Comparison model |
|---|---|---|---|
| Terminal operations benchmark 4.0 | Command-line task execution | 65.7% | 66.4% |
| Long-horizon software engineering benchmark | Multi-file, long-chain engineering tasks | 51.7% | 46.3% |
| Science task benchmark | Domain operations and reasoning | 38.1% | 63.3% |
| Hard reasoning benchmark (curated set) | Cross-disciplinary hard reasoning | 39.4% | 60.9% |
The vendor claims it beats the comparison model on 16 of 37 benchmarks. Going through them, it’s a mixed picture:
- Strengths: Terminal execution and long-horizon software engineering, which fit the Agent scenario well.
- Weaknesses: Scientific reasoning and the hardest reasoning problems, with gaps over 20 percentage points.
An independent evaluation agency’s composite intelligence index also places it at 55 points, fourth globally. This isn’t a coincidence. Agent tasks can call tools, execute, and verify intermediate results, so they’re relatively tolerant of single-step numerical precision; multi-step reasoning requires every step to be accurate, and low-bit quantization error plus attention approximation accumulate more easily over long chains.
Four points for reading self-reported data:
- Check the definition: Benchmarks with the same name often have multiple implementations; terminal benchmarks especially depend on tool-call permissions, execution sandboxes, and scaffolding design.
- Check the sampling: pass@1, majority voting, and best-of-multiple-sampling yield different conclusions.
- Check the context budget: Testing at 128K vs. 1M context often gives different results.
- Check the version: Whether the evaluated version matches the released version.
The most reliable approach is to re-test with your own business data while fixing inference parameters for reproducibility:
import json# Example: A/B evaluation of multiple models on business set A under uniform parametersdef evaluate(model_client, cases, n=1, temperature=0.0):"""Return {case_id: pass_count}, for later pass@1 / pass@n computation"""results = {}for case in cases:ok = 0for _ in range(n):resp = model_client.chat(messages=case["messages"],temperature=temperature,max_tokens=8192,)ok += int(run_checks(resp, case["expected"])) # business-defined checksresults[case["id"]] = okreturn resultsreport = {}for name, client in candidates.items(): # candidate model listreport[name] = evaluate(client, biz_cases, n=4, temperature=0.6)with open("eval_report.json", "w", encoding="utf-8") as f:json.dump(report, f, ensure_ascii=False, indent=2)
6. License: Open Weights ≠ Open Source
1. What the license text looks like
The repository carries a custom “K3.1 Pro License,” marked as an “other” license on the hosting platform, not a standard open-source license like MIT or Apache. A word-for-word comparison shows it is identical to K3’s license, with only the name changed.
The main body confirms an MIT-style grant: free to use, copy, modify, distribute, fine-tune, and commercialize. Custom licenses typically add clauses on top of this core grant, commonly including:
- Scope restrictions: e.g., requiring separate authorization or a signed agreement once you reach a certain user scale;
- Usage policy: e.g., content safety, prohibitions on specific high-risk scenarios;
- Derivative model rules: e.g., renaming or attribution requirements;
- Competitive-use restrictions: e.g., prohibiting training a competing model on its outputs.
The official full text governs in each case, and the fact that “you need to read the text” is itself the core difference between “open weights” and “open source.”
2. Why it isn’t called open source
Open source has an accepted definition: granting the rights to freely use, study, modify, and redistribute, without discriminatory restrictions on field or purpose. By this standard, the custom license deviates on at least two points:
- The license can be changed unilaterally by the publisher;
- It often carries scenario or scale restrictions.
The industry usually calls such weights “open weights”: you can download the weights, but their use is subject to additional constraints.
3. Enterprise compliance checklist
Before wiring this kind of model into production, at minimum complete the following checks:
- Commercial terms: Is there a user-scale threshold, and how do you apply for authorization above it;
- Redistribution: Can it be embedded in your own product for delivery to customers, or hosted on third-party platforms;
- Derivative models: Can fine-tuned models be published, and what are the naming and attribution requirements;
- Usage policy: Could your business scenario touch a prohibited clause;
- Output use: Can model outputs be used to improve your own model or train other models;
- Version locking: Archive the downloaded weights and license text to avoid the terms changing with upstream updates.
The safer practice is to fold license review into the model-selection process, treating it as a gating criterion on par with evaluation scores.
7. 8 Cards to Start: Where the Threshold Actually Lies
1. First, memory
Four parts need to stay resident in memory:
- The weights themselves, about 1.56TB;
- KV cache, growing linearly with context length;
- Activations and temporary buffers;
- Communication buffers and the inference framework’s own overhead.
A quick budget can be estimated with a small script:
def estimate(weights_tb, kv_cache_gb, buffer_ratio=0.10):"""Estimate in decimal TB/GB; weights_tb uses the measured value from the shard manifest"""need_gb = weights_tb * 1000 * (1 + buffer_ratio) + kv_cache_gbprint(f"Minimum memory requirement: {need_gb:.0f} GB")for gpus, per_card_gb in [(8, 200), (8, 144), (16, 144), (16, 96)]:total = gpus * per_card_gbprint(f"{gpus:>2} GPUs x {per_card_gb}GB = {total}GB:","OK" if total >= need_gb * 1.1 else "not OK") # reserve 10% headroomestimate(weights_tb=1.56, kv_cache_gb=50)
Sample output:
Minimum memory requirement: 1610 GB8 GPUs x 200GB = 1600GB: not OK8 GPUs x 144GB = 1152GB: not OK16 GPUs x 144GB = 2304GB: OK16 GPUs x 96GB = 1536GB: not OK
Implicit conditions of an 8-card config
Each card’s weight share is about 195GB; adding KV and buffers, an 8-card setup needs at least 1.7–1.8TB of total memory, i.e., over 200GB per card. Over the past two years, mainstream datacenter GPUs mostly have 80–192GB per card, so “8 cards to start” doesn’t mean “any 8 cards will do”:
- 8×80GB cards (640GB total) hold only about 40% of the weights;
- 8×192GB cards (1536GB total) can’t even hold the weights.
If the deployment floor is genuinely 8 cards, it most likely uses accelerators with over 200GB per card, or memory offloading.
KV cache can’t be ignored
KV cache size depends on layer count, KV heads, and head dimension. A sample calculation:
layers, kv_heads, head_dim, bytes_per_value = 93, 8, 128, 2 # example paramsdef kv_per_token(layers, kv_heads, head_dim, b):return 2 * layers * kv_heads * head_dim * b # one copy each for K and Vper_tok_kb = kv_per_token(layers, kv_heads, head_dim, bytes_per_value) / 1024for ctx in (128_000, 1_000_000):print(f"{ctx//1000}K context KV ≈ {per_tok_kb * ctx / 1024**2:.0f} GB")
128K context KV ≈ 44 GB1M context KV ≈ 347 GB
These are example parameters; the actual values should follow the layer count, KV heads, and head dimension in the model config. With a 16-card 200GB-class config, the weights take about half, and the remaining ~1.6TB can roughly support a few million tokens of concurrent cache. At an average of 160K tokens per request, that could serve dozens of concurrent requests in theory. The bottleneck lands on memory bandwidth and communication latency, not compute.
General KV compression strategies
| Strategy | Principle | Caveats |
|---|---|---|
| Grouped-query attention | Multiple query heads share one KV-head group | Changes model structure; must be fixed during pretraining |
| Multi-query attention | All query heads share one KV set | Large structural change; may affect quality |
| Sliding-window cache | Keep KV only for the most recent W tokens | Long-range dependency capability drops |
| Quantized KV cache | Store KV in 8-bit or 4-bit | Need to evaluate precision impact |
| Prefix reuse | Compute identical prefixes only once | Good for fixed system prompts or multi-turn dialogue |
2. How to split parallelism
Trillion-scale MoE usually combines three kinds of parallelism:
- Tensor parallelism: Split each layer’s matrix operations across cards; communication is frequent, so keep it within a single node’s high-speed interconnect;
- Expert parallelism: Assign the 896 experts across cards, each holding only a subset; the cost is all-to-all communication and load imbalance;
- Pipeline parallelism: Split by layer; low communication when scaling across nodes, but produces bubbles.
An example config (parameter names depend on the inference framework):
# Example config: 8-card local deploymentnum_gpus: 8gpu_memory_gb: 200tensor_parallel_size: 8 # keep within one node, leverage high-speed interconnectexpert_parallel_size: 8 # 896 experts assigned across cardsmax_model_len: 131072 # cap context length first for local usekv_cache_dtype: fp8 # enable KV quantization after precision evaluationgpu_memory_utilization: 0.85
Loading quantized weights and the sparse-attention backend also requires an inference framework version that supports the corresponding format. In large-scale online serving, the more common approach is to deploy multiple instances with data parallelism and load-balance between them.
3. Performance ceiling
The following are order-of-magnitude estimates, not measured results:
- Prefill: The longer the context, the more pronounced the sparse-attention benefit. At million-token scale, the compute saved by the 24 sparse-attention layers can account for a substantial share of the whole prefill; otherwise this cost grows quadratically with length.
- Decode: At decode time, generating each token reads all weights from memory once. At an aggregate memory bandwidth of 10TB/s, reading 1.56TB of weights has a floor of about 0.16s/token, i.e., single-digit to a dozen-plus tokens/s per request. Actual values are affected by communication overhead, batching strategy, and KV size.
For latency-sensitive online workloads, cap the context at roughly 100K–200K tokens, or reserve million-token context for offline batch jobs.
8. Common Cloud Deployment Practices
If your local setup is insufficient—say you only have 8×80GB cards—but you want to serve externally, the cloud is a more realistic option. Three points matter:
- Compute specs: Choose GPU instances with large per-card memory or full 8-card instances on demand; first filter models by the memory threshold, then evaluate interconnect bandwidth.
- Weight distribution: Put the ~1.56TB weights in object storage, keep a copy in each of multiple availability zones, and have instances pull them in parallel over the same-region intranet at startup, so downloads don’t become an expansion bottleneck.
- Serving: Deploy the inference service on a container platform, configure autoscaling by throughput and latency, and continuously track memory usage, queue length, first-token latency, and error rate via logging and monitoring.
Baidu AI Cloud, for example, offers general capabilities—GPU cloud servers, object storage, container services, and monitoring/alerting—that can be combined into a complete “storage, compute, orchestration, observability” chain.
Example flow:
[Object storage: weight replicas] --intranet parallel pull--> [Containerized inference service: multiple replicas]|+-------------------+-------------------+| |[Load balancer entry] [Logging + monitoring/alerting]| |[Business app] memory / latency / error rate
Conclusion: Who It’s For
K3.1 Pro pushes the scale ceiling of open weights to a trillion parameters, showing on coding and Agent tasks that it can go toe-to-toe with top closed models. But it’s not a model “anyone can run.”
- Worth trying: Although a single instance’s memory floor is high, once priced per token the activated parameters are only in the tens of billions, so per-token cost may actually be low—suited to well-funded teams targeting coding and automation tasks.
- Not ideal for: Teams wanting to deploy locally on 8 ordinary GPU cards, and teams needing BF16 precision for continued pretraining.
For final judgment in model selection, consider three questions:
- Can your business accept the constraints of a custom license?
- Can your memory and interconnect conditions support the target concurrency?
- Do the re-test results on real business data justify the time cost of downloading it?
