Trillion-Parameter MoE Open Weights Released: Deployment and Evaluation for Long-Context Agents

1. Background: Industry Significance of Open Weights

On November 4, 2024, an AI research institute officially uploaded the complete weights of its flagship MoE model to mainstream model hosting platforms, fulfilling its public commitment to open weights. This move has garnered significant attention within the technical community. The attention stems not only from the model’s sheer scale—2.8 trillion total parameters with 104B activated parameters per token—but also from its representation of a critical step for the open-weight ecosystem toward trillion-parameter MoE architectures.

For teams looking to deploy, fine-tune, or audit the model independently, the release of complete weights signifies three key advantages:

• Reproducibility: Researchers can reproduce officially claimed capability metrics in local environments without relying on closed-source APIs.
• Customizability: Enterprises can perform continuous pre-training or instruction fine-tuning based on proprietary domain data to build private agents.
• Auditability: Security teams can perform static and dynamic analyses on the weights to assess potential risks.

Notably, in the rankings published by Arena on the same day, the model was listed as No. 1 among open-weight models and No. 13 overall. This performance is derived from real user long-horizon Agent session data rather than laboratory benchmarks, providing stronger practical reference value.

2. Model Architecture: Synergy Between Sparse Attention and MoE Experts

2.1 Overall Layers and Attention Hybridization

According to the specifications disclosed in the model card, the model consists of 93 layers in total, wherein:

• 69 layers adopt KDA (Kernelized Dynamic Attention-like architecture)
• 24 layers adopt Gated DSA (Gated DeepSeek Sparse Attention)

Compared to its predecessor, the core change is replacing full-attention Gated MLA layers with Gated DeepSeek Sparse Attention. The direct purpose of this modification is to reduce the computational overhead of long-context modeling.

Traditional full attention has a computational complexity of O(L²) for a sequence length of L. When the context expands to the million-token level, the memory and compute consumption of the attention matrix increases drastically. Sparse attention significantly reduces the effective complexity by restricting each token to attend only to a subset of historical positions, while preserving critical information pathways through gating mechanisms.

2.2 MoE Expert Configuration

The expert layers employ a large-scale sparse activation design:

Metric	Value
Total Experts	896
Activated Experts per Token	16
Shared Experts	2

This “896 choose 16 + 2 shared” configuration ensures that the actual computation per token is far less than what the total parameter scale suggests. The inclusion of shared experts guarantees the stable transfer of general knowledge, preventing all experts from over-specializing.

2.3 Vision Encoder

The vision component is handled by MoonViT-V2, which has 401M parameters, and supports image-text mixed inputs. This means the model can directly process multimodal content such as screenshots, charts, and document images, which is particularly crucial for “screen-viewing operations” in agent scenarios.

3. Quantization Strategy and Storage: MXFP4 vs. MXFP8 Trade-offs

3.1 Weight Storage Format

The weights are stored in MXFP4 format, while activations use MXFP8. The official documentation explicitly states the use of Quantization-Aware Training (QAT) rather than Post-Training Quantization (PTQ).

The technical implications of this choice are:

• MXFP4 Storage: Compresses weights to 4-bit floating-point, drastically reducing memory footprint and disk size.
• MXFP8 Activations: Maintains 8-bit precision for activation values during inference, balancing speed and numerical stability.
• QAT Training: Simulates quantization errors during the training phase, making the model weights more robust to low-bit representations and typically resulting in lower accuracy loss compared to PTQ.

3.2 Actual Storage Footprint

The model hosting platform API indicates that the repository occupies approximately 1.56TB; the parameter count derived from the weight files is approximately 2.78 trillion. This figure closely aligns with the officially claimed 2.8T, with minor discrepancies arising from statistical methods and actual quantized storage.

For deployers, a 1.56TB weight volume implies:

• In single-node multi-GPU scenarios, sufficient VRAM or unified memory is required to load the weights.
• If multi-node inference is used, high-bandwidth interconnects are necessary to mitigate communication bottlenecks.
• Disk I/O becomes the critical path for loading speed during cold starts.

4. Deployment Paths: Mainstream Inference Framework Compatibility

Several mainstream inference frameworks have already provided deployment recipes, covering scenarios ranging from single-node multi-GPU to multi-node distributed setups. When selecting a framework, developers should focus on the following dimensions:

1. Support for MXFP4/MXFP8 mixed-precision loading
2. Efficient implementation of sparse attention kernels
3. KV Cache management for million-token contexts
4. Parallel strategies for multi-node multi-GPU setups (tensor parallelism, pipeline parallelism, expert parallelism)

A typical deployment workflow can be abstracted into the following steps:

# 1. Pull weights (neutralized example)
# Download safetensors shards from the model hosting platform
# 2. Configure inference framework parallel parameters
# Example: tensor_parallel_degree=8, expert_parallel_degree=8
# 3. Start inference service
# Specify model path, precision mode, and max context length
# 4. Health check and stress testing
# Verify first token latency, throughput, and long-context stability

It should be emphasized that there is no official unified hardware recommendation; third-party estimates are for reference only. Community users have estimated that a viable configuration could involve 8 high-end datacenter GPUs, roughly 2.3TB of VRAM, with hardware costs ranging from $550,000 to $650,000. This scale indicates that local deployment of a trillion-parameter MoE remains a high-barrier scenario, best suited for teams with explicit privatization requirements and adequate compute budgets.

5. API Usage: Compatible with Mainstream Interface Standards

For teams preferring not to self-deploy, an API access method is available. The model name is kimi-k3.1-pro, which is compatible with both OpenAI and Anthropic interface styles. Additionally, a kimi-k3.1-pro-highspeed version is offered, which the official documentation states is 2 to 3 times faster than the standard version.

For inference control, reasoning_effort can be set to low, high, or max, defaulting to max. This means that by default, the model will dedicate significant inference compute, making it suitable for complex tasks; if latency sensitivity is a concern, this parameter can be lowered.

Critical detail for multi-turn tool calls: In continuous conversations, the reasoning_content and tool_calls from the previous turn must be passed back. Otherwise, the model will lose its previous reasoning chain and tool execution context, leading to inconsistent behavior. This is especially important when building Agent loops.

6. Agent Arena Evaluation: Extracting Signals from Real Sessions

6.1 Evaluation Data Source

Agent Arena data is sourced from real user Agent sessions rather than manually constructed test sets. It extracts the following types of signals:

• Whether the user explicitly confirms task success
• User praise and complaints regarding the model
• The quality of the model’s response after user correction
• Whether shell commands are executed successfully

Based on these signals, Arena calculates a “net improvement“ metric. The design philosophy behind this metric is to evaluate not only what the model gets right but also whether it can effectively adjust after being corrected.

6.2 Ranking Results

Based on over 17,500 real long-horizon sessions, the model (in Max mode) achieved:

• Net improvement of +5.1%
• No. 1 among open models, No. 13 overall
• No. 1 among open models in the Work, Code, and Chat categories

The previous generation model had a net improvement of +0.91%, ranking No. 6 among open models. This comparison illustrates a significant leap in real-world task performance for the new generation.

6.3 Breakdown Performance

Among open models, this model achieved a Bash reliability of +7.9%, ranking No. 1; the previous generation recorded -4.5%, ranking No. 13. Bash reliability directly impacts an agent’s ability to execute shell commands correctly, serving as a core capability for automated operations and code execution scenarios.

7. Practical Recommendations for Developers

1. Prioritize API for capability boundary verification: Before investing in hardware, use the API to validate business scenarios, confirming the model’s performance in multi-turn tool calling, long contexts, and multimodal inputs.
2. Monitor the cost curve of reasoning_effort: The default max is suitable for complex reasoning, but for simpler tasks, lowering it can reduce latency and cost.
3. Always pass back complete context in multi-turn calls: Include reasoning_content and tool_calls, otherwise Agent behavior will degrade.
4. Evaluate Total Cost of Ownership (TCO) for local deployment: Consider not only GPU procurement costs but also power, cooling, operations, and model update frequency.
5. Leverage open weights for domain adaptation: If you have proprietary data, consider lightweight fine-tuning within a QAT framework, but be mindful of numerical stability in low-bit quantization.

8. Conclusion

The release of trillion-parameter MoE open weights signifies that the open ecosystem is approaching the frontier of closed-source models in both scale and capability. The combination of 2.8T total parameters, 104B activation, million-token context, image-text input, and the MXFP4/MXFP8 QAT scheme establishes a robust technical foundation tailored for long-horizon agent tasks. The real-session evaluation on Agent Arena further validates its leading position in dimensions such as Bash reliability, Work, Code, and Chat.

For developers, the core decision point currently is: rapid validation via API, or investing in hardware for private deployment. Both paths have their applicable scenarios, and the greatest value of open weights lies in returning the power of choice to the technical teams themselves.