Open-Source K3.1 Pro Released: A Deep Dive into 2.8T Parameters and Long-Horizon Agent Capabilities

As large language models evolve from pure text generation toward complex task execution and agentic workflows, model architectures and training methods face unprecedented challenges. Recently, the latest model in the K3 series, K3.1 Pro, officially released its model weights. With its massive parameter scale, innovative architecture design, and outstanding performance on long-horizon agent tasks, it has quickly become a focal point of the technical community. This article provides an in-depth technical analysis of the model across multiple dimensions, including architecture, training strategy, performance benchmarks, and deployment practices.

1. Core Architecture: MoE Design with 2.8T Total Parameters

The most striking feature of K3.1 Pro is its Mixture-of-Experts (MoE) architecture. The model has a total parameter count of 2.8T (trillion), yet through the MoE design, only 104B (billion) parameters need to be activated per inference. This design philosophy of “massive total parameters, refined activated parameters” is a key approach for balancing performance and compute cost in current large models.

The core idea of MoE architecture is to split the model into multiple “expert” sub-networks. During forward propagation, a gating router mechanism dynamically selects the most appropriate experts to process the current input. This sparse activation pattern means that although the model possesses an extremely large knowledge capacity, the computational overhead for processing a single token is only related to the number of activated experts, thereby significantly increasing model capacity while effectively controlling actual inference computation. Building on K3, K3.1 Pro replaces 24 layers of Gated MLA (Multi-head Latent Attention) with Gated DSA (Dense Sparse Attention), combined with 69 layers of Kimi Delta Attention. This hybrid attention mechanism design is the physical foundation for its breakthrough in long-context and complex reasoning tasks.

2. Technical Highlights: Sparse Attention and Context Window Extension

In long-horizon agent tasks, models often need to process ultra-long contexts of hundreds of thousands or even millions of tokens, covering historical records from multi-turn tool calls, codebase files, external retrieval results, and more. Traditional dense attention mechanisms exhibit quadratic growth in computational complexity in such scenarios, leading to excessive memory usage and sharply degraded inference speed.

To address this pain point, K3.1 Pro introduces the Gated DSA mechanism. The core innovation of this mechanism is that it does not let all historical tokens participate in attention computation. Instead, through a learnable gating strategy, it dynamically selects approximately 2,048 key historical tokens per layer to participate in attention scoring, working in conjunction with Kimi Delta Attention to capture long-range dependencies. This dynamically sparse attention computation greatly reduces the compute consumption and memory pressure of long-sequence inference. Meanwhile, the model supports a native 1-million-token context window and has native image-text input capability. This means developers can directly feed entire large code repositories, multiple long documents, or complex PDFs containing charts to the model without tedious splitting and preprocessing, enabling the model to perform global understanding and reasoning.

3. Training Strategy: Joint Reinforcement Learning Across Hundreds of Domains

The ceiling of a model’s capability depends not only on architecture but also on training methodology. Building on the K3 base model (which underwent large-scale pre-training), K3.1 Pro underwent an unprecedented “post-training” phase, namely joint reinforcement learning (RL).

According to the official technical report, the model’s training covers hundreds of downstream domains. This multi-task joint RL is not simply task concatenation; rather, it leverages a shared reward model or preference alignment strategy to collaboratively optimize across multiple dimensions including code generation, mathematical reasoning, instruction following, and safety alignment. This training approach effectively prevents the model from overfitting on a single task while promoting positive transfer between different skills.

For example, when performing software engineering benchmarks such as SWE Marathon, the model may need to invoke multiple tools and modify multiple files. In financial research scenarios, it needs to conduct multi-turn web retrieval and information verification. Joint RL training enables the model to learn to flexibly switch “thinking budgets” across different tasks. The official release provides three thinking budget settings for inference: low, high, and max. Developers can finely control the model’s actual reasoning depth and token consumption based on task complexity, balancing response speed and accuracy.

4. Performance Benchmarks: Hard Results and Scenario Coverage

To assess whether an open-source model has practical application value, standardized benchmarks are the most informative dimension. Based on officially published benchmark data, K3.1 Pro performs impressively across multiple high-difficulty task sets:

• Terminal Operations (Terminal-Bench 4.0): Score 65.7. This benchmark primarily evaluates the model’s operational capability in real command-line environments, including file management, environment configuration, and script execution. It is an important indicator of whether the model can handle automated operations and system administration tasks.
• Software Engineering (SWE Marathon v1.1): Score 51.7. This benchmark simulates real GitHub issue resolution workflows, where the model must understand problem descriptions, locate code defects, write patches, and submit them. This result indicates strong code repair and software maintenance capabilities.
• Deep Software Engineering (DeepSWE v1.1): Score 75.5. This further delves into complex codebase-level refactoring tasks, validating the model’s context modeling capability in large-scale projects.
• Web Browsing (BrowseComp): Score 92.1. In delegated tasks, the model must autonomously operate a browser for information retrieval and extraction. The high score confirms its reliability in agent tasks requiring real-time information acquisition.
• 3D Design (BenchCAD, with tools): Score 92.7. This score reveals the model’s potential in cross-modal generation tasks, driving CAD software to generate 3D models through natural language instructions, which is groundbreaking for engineering design automation.
• Office Document Processing (OfficeQA Pro v2): Score 64.4. This demonstrates strong productivity tool invocation capability in complex office ecosystems such as spreadsheet processing, PPT generation, and PDF summarization.

It should be emphasized that the above data are all official results obtained using the max thinking budget setting, with some scenarios assisted by agent frameworks such as Claude Code. While acknowledging these impressive scores, we should also rationally assess the specific deployment challenges in particular environments.

5. Open Ecosystem: Weight Licensing and Deployment Practices

As an open-source model, K3.1 Pro demonstrates a high degree of openness while maintaining rigorous compliance.

• Weight Release: The official release provides model files quantized with MXFP4 (a hybrid format based on FP4 precision). MXFP4 significantly reduces memory usage and I/O overhead while ensuring nearly lossless model precision, making it possible to deploy the model on a single A100/H100 or even consumer-grade GPUs with smaller memory.
• Inference Frameworks: The model natively supports mainstream high-performance inference engines such as vLLM and SGLang, and is compatible with inference acceleration technologies like TokenSpeed. It also provides standard API interfaces compatible with OpenAI and Anthropic specifications, facilitating seamless migration of existing agents or toolchains built on these interfaces.
• Usage Notes: Developers should note that in multi-turn tool-calling scenarios, the complete conversation history and tool-call message records must be preserved (rather than only text summaries) to ensure the model has sufficient context for accurate next-step decisions.
• License Agreement: This is one of the most critical legal considerations of this release. K3.1 Pro does not use the common Apache 2.0 or MIT license; instead, it adopts a custom license specific to this model. This means that while the weights can be downloaded and used commercially, modification rights, distribution rights, and commercial boundaries are subject to specific terms. Developers must carefully review and understand the constraints of this license before integrating it into commercial products.

6. Summary and Outlook

The release of K3.1 Pro marks a solid step forward for open-source large models in balancing “scale” and “efficiency.” Its 2.8T-parameter MoE architecture proves that “bigger is better” still holds, while the introduction of the Gated DSA sparse attention mechanism is an effective response to the compute bottleneck of long-horizon tasks. The model’s high scores in code, terminal operations, and multimodal document processing make it one of the ideal foundations for building complex AI agents today.

For developers, this is undoubtedly an open-source model worth investing time in for deep testing and adaptation. Whether using it as an underlying engine to build a private coding assistant or exploring its potential in vertical domains such as automated process handling and intelligent financial analysis, K3.1 Pro offers a highly competitive frontier technology option. In the future, whether the model can consistently reproduce the high-level performance seen in benchmarks across more real-world deployment scenarios will be the key litmus test of its true value.