Open-Source Long-Horizon Agent Model Explained: 2.8T Sparse Architecture and Tool-Calling in Practice

1. Model Overview and Positioning

An open-source model designed for long-horizon agent tasks has officially released its weights. Its design targets three core bottlenecks in current autonomous agent systems: context degradation during long-running tasks, reasoning stability in multi-step tool calling, and limited cross-domain generalization. The model uses a Mixture-of-Experts architecture with approximately 2.8T total parameters and around 104B activated parameters. This design raises model capacity to a level sufficient for complex long-horizon planning while maintaining inference efficiency.

At the architectural level, the model supports a context window of up to 1 million tokens and natively accepts image and text inputs. This means a single inference pass can accommodate the codebase of an entire software project, a technical document hundreds of pages long, or all intermediate results generated across multiple rounds of tool calling. For a long-horizon agent, context window size is not simply a matter of how much text can fit. It determines whether the agent can continuously track the global state of a task throughout the inference process.

The model is not positioned as a general-purpose conversational model. It explicitly targets three categories of long-horizon scenarios: code development and 3D creation, agent and knowledge workflows, and complex automation tasks requiring cross-tool collaboration. Understanding this positioning is essential for understanding all of its technical decisions. The context compression strategy, the choice of attention mechanisms, and the thinking budget tiers all revolve around one central question: how to keep an agent running stably over task cycles lasting hours or even days.

2. Sparse Attention: The Core Support for Long Context

The first major technical challenge for long-horizon agents is that attention computation complexity grows quadratically with sequence length. With a 1-million-token context, conventional full attention is not engineering-feasible, while forced truncation causes the agent to forget critical early information. The model addresses this with a layered sparse attention scheme that deserves a closer look.

Layer 1: Gated DSA (Gated Dynamic Sparse Attention). The core idea is that each token does not need to attend to the entire history. During inference, only a fixed number of relevant tokens—around 2,048—are dynamically selected from the historical sequence for attention computation. The selection process is controlled by a gating mechanism that scores the relevance between the current token’s semantic representation and historical tokens. Unlike traditional sliding-window attention, DSA selects from the entire context. Even early information thousands of steps away can be selected if it is highly relevant to the current reasoning step.

Layer 2: Combined use of Kimi Delta Attention. The attention system is not a single mechanism. Instead, Gated DSA is combined with the existing Delta Attention mechanism on a per-layer basis. Based on the parameter scale, the model uses a deep network structure of roughly 93 layers, of which 69 layers retain the Delta Attention design and 24 layers are replaced with Gated DSA. Delta Attention reduces repeated computation overhead through incremental updates to attention states. DSA solves the problem of precise recall from the global context. The combination forms a complementary structure: Delta layers efficiently maintain fine-grained information in adjacent regions of the sequence, while DSA layers dynamically retrieve relevant long-range information from a global view.

The practical effect of this dual-layer sparse attention design is that large-scale long-horizon task inference moves from theoretical possibility to engineering reality. For developers, understanding this mechanism helps with prompt structure planning. Because DSA layers perform relevance retrieval over the full token sequence, placing key constraints and task goals early in the reasoning chain makes them more likely to be reliably recalled by later steps than burying them in the middle.

3. Joint Reinforcement Learning and Thinking Budgets

The post-training stage uses large-scale joint reinforcement learning across hundreds of downstream domains. The defining characteristic of this training strategy is its “joint” nature—rather than training separate reward models or policy models for each domain. The advantage of joint reinforcement learning is that the model is simultaneously exposed to code repair, terminal operations, CAD design, financial research, document processing, and other task types during optimization. This allows it to learn reasoning patterns and tool-use strategies that generalize across domains, rather than overfitting to task-specific solution templates for a single category.

The inference stage provides three thinking budget tiers: low, high, and max. This design reflects an often-overlooked reality in long-horizon agent tasks: not all steps require the same depth of reasoning. In a terminal operation task, the reasoning depth required for “list the current directory contents” is fundamentally different from that required for “identify the root cause from an error message and design a fix.” A model with fixed thinking depth wastes compute on simple steps while potentially under-reasoning on critical ones.

The concrete differences among the three tiers lie in reasoning chain length and exploration breadth. The low tier suits steps with strong determinism, producing short, direct reasoning chains. The high tier fits subtasks requiring comparison among multiple approaches. The max tier enables the longest reasoning chains and is intended for high-difficulty decision points such as architecture design or complex bug root-cause analysis. When building an agent in practice, the recommended pattern is for an upper-level scheduler to dynamically raise or lower the tier based on subtask complexity, rather than applying the max tier uniformly across the entire run. This controls end-to-end latency while maintaining reasoning quality at critical junctures.

4. Interpreting the Official Benchmark Results

The officially published benchmark scores provide a reference for understanding the model’s practical capability boundaries. It is important to note that these results were measured under the max thinking tier, and some evaluations were conducted with external agent frameworks such as Claude Code. These numbers should not be interpreted as raw model scores. They should be viewed as the performance of an integrated system consisting of the model, the agent framework, and the tool environment.

Looking at the score distribution: Terminal-Bench 4.0 achieved 65.7, indicating that in real terminal environments with multi-step operations, roughly one-third of scenarios still cannot be completed independently. SWE Marathon v1.1 at 51.7 further confirms this—in long-duration software engineering tasks, more than half of the tasks still require human intervention or multiple retries. By contrast, DeepSWE v1.1 at 75.5 and BenchCAD at 92.7 show that the model performs significantly better in scenarios with relatively clear problem definitions and regular environment feedback. BrowseComp at 92.1 suggests the model is approaching practical usability for information retrieval and synthesis tasks with relatively linear steps.

These results reveal a key insight: the capability bottleneck of current long-horizon agent models is not evenly distributed across task types. It is concentrated in tasks with high environment uncertainty, sparse feedback signals, and a need to maintain long-term planning. When evaluating whether to deploy such models in production, developers should base their assessment on the characteristics of their own tasks—whether the environment changes dynamically, whether feedback is immediate, and whether subgoals can be clearly decomposed—rather than directly applying leaderboard scores.

5. Deployment, Quantization, and Tool-Calling Constraints

Weights and quantization format. The released weights use the MXFP4 quantization format. The direct implication is that deployment memory requirements are significantly lower than full precision, but there is an established trade-off between inference speed and precision. For teams with multi-GPU deployment capacity, it is advisable to verify the precision difference between higher-precision inference and MXFP4 inference on their own tasks to determine whether the trade-off is acceptable.

Inference framework compatibility. The model supports integration with mainstream open-source inference engines and provides API formats compatible with OpenAI and Anthropic. This means existing agent frameworks can adopt it by changing the endpoint and model name without rewriting tool-calling logic. One critical note: the model’s license is not a common permissive open-source license, but a customized proprietary license. Before commercial deployment, the license terms must be reviewed item by item, especially restrictions involving the use of model outputs for training other models, multi-tenant SaaS services, and weight redistribution.

Key constraint for multi-turn tool calling. The official documentation explicitly requires that complete historical thinking and tool-calling messages be retained during multi-turn tool calling. This is not an optional engineering choice. It is directly tied to how the model was trained. The model was optimized to make decisions based on complete reasoning trajectories, not compressed state summaries. If developers implement “history message trimming” or “keep only tool return results” optimizations in their agent frameworks, the model’s behavior may deviate significantly from its training distribution. This can manifest as broken reasoning chains, repeated calls to the same tool, or ignoring previously obtained key information.

Recommended message management strategy in practice: retain complete thinking and tool-calling messages, but control context growth through the following methods—structurally truncate long text outputs from tools while preserving key fields relevant to the task; summarize and compress old thinking steps while explicitly marking them as compressed regions; insert clear phase boundary markers when switching between task stages. These strategies keep effective context usage within a reasonable range while preserving information integrity.

6. Practical Recommendations for Developers

Based on the technical characteristics above, the following practical points can guide the construction of long-horizon agents:

First, task decomposition and budget allocation. Long-horizon tasks should be decomposed into a sequence of subtasks with clear acceptance criteria, with a thinking budget assigned to each subtask at the orchestration layer. For example, use the low tier for environment setup, the high tier for code logic implementation, and the max tier for architecture selection or complex defect localization.

Second, context structure design. Leverage DSA’s global retrieval behavior by placing core constraints, acceptance criteria, and prohibited actions in a fixed region such as the system prompt or the beginning of the task. During long tasks, periodically repeating key constraints improves the probability that they will be consistently selected by dynamic attention.

Third, tool-calling log completeness. Disable automatic history trimming in the agent framework and ensure that all thinking steps and tool-calling results are fully retained. If context length must be controlled, prioritize structural filtering of non-essential fields in tool returns rather than deleting thinking steps themselves.

Fourth, evaluation and fallback mechanisms. Before deploying the model in production, build a regression evaluation set covering your own tasks, including both normal and failure paths. The agent framework should include failure detection and fallback capability: when the model is detected falling into repeated tool calls or reasoning chain breaks, trigger task replanning or escalate to a higher thinking tier.

The common goal of these recommendations is to keep the model operating within its training distribution. Complete historical trajectories, clearly expressed constraints, and appropriate reasoning depth are the input forms the model was optimized for during training. Engineering optimizations that deviate from this distribution may introduce performance degradation that is hard to detect but real.