Technical Architecture and Reinforcement Learning Practices of Ultra-Long Context Agent Models

1. Background: What Model Capabilities Long-Horizon Agents Demand

As agent tasks evolve from single-turn Q&A to multi-step execution, models must not only understand user instructions but also maintain goal consistency, tool-call consistency, and contextual memory consistency over extended time spans. Representative scenarios include code repairs lasting tens of minutes, cross-document information extraction and summarization, multi-turn feedback-driven 3D parameter adjustment, and financial research tasks requiring repeated retrieval and comparison.

These tasks impose three core requirements on the model:

1. Long-context capacity: The ability to process hundreds of thousands or even millions of tokens without losing early information.
2. Stable tool-call behavior: The model must correctly generate call parameters, parse return results, and adjust subsequent actions across multiple interaction turns.
3. Controllable inference cost: Higher reasoning intensity improves success rates on complex tasks but also increases latency and compute consumption, so tiered control is necessary.

Traditional dense attention models face quadratic growth in memory and compute as context scales, making them unsuitable for large-scale long-horizon tasks. Sparse attention and Mixture-of-Experts (MoE) architectures have therefore become the dominant technical approach for ultra-long context models. This article uses a representative open-source ultra-long context agent model as a reference to analyze its architecture, training, and deployment methods. It does not point to any specific vendor or product and discusses only general technical solutions.

2. Overall Architecture: Mixture-of-Experts Meets Ultra-Long Context

The model adopts a Mixture-of-Experts architecture with approximately 2.8T total parameters and 104B activated parameters. The core idea behind MoE is to split the feed-forward network into multiple expert modules, activating only a subset during each inference pass, thereby expanding model capacity while controlling compute cost.

This design is particularly critical for long-horizon agents. Different steps in an agent task often involve different knowledge domains: some require code syntax understanding, others document semantic summarization, and still others numerical reasoning. The MoE structure allows different tokens to be routed to different experts, improving overall performance on multi-domain tasks.

On context length, the model supports roughly one million tokens of input and allows mixed image-and-text input. Native multimodal support means the model does not need an additional vision encoder as a bridge; it can process code screenshots, architecture diagrams, table images, and natural language instructions within a unified representation space. For agent tasks that require referencing UI screenshots for terminal operations or design review, this capability significantly reduces information loss.

From an engineering perspective, ultra-long context support is not simply a matter of extending positional encodings. It also requires supporting cache management, batching strategies, and inference engine adaptations. The next section focuses on the key technology that underpins this capability: sparse attention.

3. Sparse Attention: From Gated MLA to Gated DSA

In the previous generation architecture, certain layers used Gated Multi-Head Latent Attention (MLA). This mechanism compresses key-value caches through low-rank latent projections to reduce memory consumption during long-sequence inference. However, latent attention still requires traversing all historical tokens, so computation remains substantial when context approaches the million-token level.

The new version replaces those layers with Gated Dynamic Sparse Attention (DSA), combined with a separate class of Delta Attention layers. The specific changes are:

• Replacement scope: 24 layers of gated multi-head latent attention are replaced with gated dynamic sparse attention.
• Composition: These layers work together with the existing 69 Delta Attention layers to form the complete attention stack.
• Sparsity strategy: Each dynamic sparse attention layer dynamically selects approximately 2,048 tokens from the history for attention computation at inference time, rather than traversing the full context.

The advantage of dynamic sparse attention is compressing context computation from linear growth to near-constant. The challenge is equally clear: how to ensure the 2,048 selected tokens are precisely what the current step needs most. To address this, the model introduces a learnable sparse routing strategy during training, allowing attention selection to adapt dynamically based on task signals.

Delta Attention layers serve a different function: maintaining cross-layer state delta updates. For long-horizon tasks, the model needs to remember intermediate results from earlier steps without recomputing the full history for every step. The Delta mechanism lets the model store only the changes in key states and recover them quickly when needed.

The cooperation between the two attention mechanisms can be understood this way: dynamic sparse attention handles “what the current step needs to look at,” while Delta attention handles “what past steps have left behind.” This division of labor lets the model stay sharp on recent context in long-horizon tasks while retaining memory of distant information.

4. Training Strategy: Joint Reinforcement Learning and Quantization

The training difficulty for long-horizon agent models lies in the extremely broad task distribution. Single-domain reinforcement learning tends to cause overfitting on specific tasks and loss of cross-domain generalization. To address this, the model uses a joint reinforcement learning strategy covering hundreds of downstream domains.

Joint reinforcement learning is not simply mixing data; it requires balancing gradient signals across domains in every update. In practice, this typically involves:

1. Task sampling strategy: Dynamically adjusting sampling probabilities based on domain difficulty, data diversity, and task length to prevent simple tasks from dominating gradients.
2. Reward model design: Long-horizon tasks often cannot be characterized by a single scalar reward. A common approach is to combine process rewards with outcome rewards—for example, in code tasks, considering both intermediate step execution feedback and final test pass rates.
3. Multi-turn interaction training: Agent tasks are inherently multi-turn. Training must construct complete “instruction—reasoning—tool call—environment feedback—continued reasoning” trajectories rather than optimizing only single-turn outputs.

On weight precision, the model releases weights in MXFP4 quantization format. MXFP4 is a block-level floating-point format that preserves dynamic range better than traditional integer quantization. For models exceeding 2T parameters, quantization is a prerequisite for distribution and deployment. FP4 precision significantly compresses model size but requires corresponding dequantization operators and mixed-precision computation strategies in the inference framework.

5. Inference Tiers and Agent Scenarios

The model provides low, medium, and high reasoning budgets. The reasoning budget controls how many reasoning tokens the model may generate before producing a final answer. The low tier suits simple queries and fast responses, the medium tier suits routine tool calls, and the high tier is for complex planning and multi-step problem solving.

In official evaluations, the highest reasoning tier is used by default, partially combined with external agent frameworks for tool orchestration. Evaluations cover these typical scenarios:

• Terminal operations: Given a natural language instruction, the model executes commands in a simulated terminal, observes output, and corrects errors.
• Code repair: Locating defects and generating patches in large software engineering tasks, potentially spanning multiple files and multiple rounds of test feedback.
• CAD generation: Generating or adjusting 3D model parameters from design requirements, involving parsing and iteratively optimizing design tool output.
• Web retrieval: Extracting evidence from large numbers of web pages to answer complex questions, with emphasis on retrieval accuracy and evidence chain completeness.
• Financial research: Processing multi-source documents and numerical tables to perform summarization, calculation, and attribution analysis.
• Office document processing: Performing information extraction, summarization, and structured output across long document collections.

Different scenarios place vastly different demands on model capabilities, so reasoning tier selection should be part of agent application design. For example, interactive Q&A sensitive to user wait time can use the low tier, while background batch deep-research tasks can use the high tier with longer acceptable latency.

6. Deployment and Engineering Best Practices

Deploying ultra-long context sparse models requires attention to the following engineering issues:

6.1 Inference Engine Adaptation

Sparse attention introduces dynamic token selection logic, placing new demands on cache management in the inference framework. The traditional prefill-decode two-phase approach may suffer from cache fragmentation under ultra-long contexts. The inference engine needs to support cache updates in dynamic sparse mode and handle different sparse selection results across samples in batching scenarios.

Current mainstream open-source inference stacks have begun supporting such sparse architectures, but careful evaluation of the throughput-latency tradeoff is still needed under high-concurrency scenarios.

6.2 Message Management for Multi-Turn Tool Calls

Agent tasks typically use frameworks compatible with OpenAI or Anthropic-style interfaces for tool calls. A common mistake is retaining only the final text reply across multiple turns, discarding intermediate reasoning and tool-call messages. For long-horizon models trained with reinforcement learning, intermediate reasoning and tool feedback are critical bases for the model’s state inference. Truncating this information may cause the model to lose accurate judgment of task progress in subsequent turns.

Therefore, multi-turn tool calls should preserve the complete message chain:

- User's original instruction
- Model reasoning content
- Tool call request
- Tool return result
- Model's continued reasoning based on the result
- Final response

6.3 License Compliance Assessment

The model weights are released under a proprietary license rather than a permissive open-source license. Proprietary licenses typically impose additional terms on commercial use, redistribution, or modification rights. Before deploying in production, enterprises should complete a license compliance assessment to clarify whether internal deployment, external API services, and fine-tuning or distillation based on the weights are permitted.

This constraint has significant implications for developer selection. Permissive licenses favor rapid integration and secondary development, while proprietary licenses demand greater attention to long-term usage risk and compliance cost.

7. Conclusion

The core challenges of ultra-long context agent models are: maintaining inference efficiency at million-token context scales, ensuring behavioral stability across multi-step tasks, and controlling engineering complexity in complex deployment environments. The Mixture-of-Experts architecture provides the capacity foundation, dynamic sparse attention and incremental attention mechanisms solve long-sequence efficiency, joint reinforcement learning secures cross-domain generalization, and reasoning tiers and tool message management directly affect real-world application outcomes.

For developers and enterprise users, evaluation should go beyond benchmark scores. Attention must be paid to the model’s state-retention ability in real multi-turn tasks, the maturity of the inference framework, and the impact of license terms on business boundaries. Only by carefully assessing architecture, training, inference, and compliance across all four dimensions can truly deployable long-horizon agent applications be built.