2.8T Open-Source MoE Model for Long-Context Agents: Engineering Trade-offs and Deployment

1. From “Parameter Stacking” to “Computable”: Engineering Trade-offs of the 2.8T MoE

As large language models enter the era of long-horizon tasks, the core challenge is no longer simply “answering a single question correctly.” Instead, it is about “maintaining state and controlling costs across dozens or hundreds of interactive steps.” Recently, a technical team officially open-sourced the weights of a large-scale Mixture-of-Experts (MoE) model. Several figures from its public release deserve close attention from developers:

• Total parameters: 2.8T; Activated parameters: 104B
• Context length: 1 million Tokens
• Native multimodal input (text and image)
• Three inference-side “thinking budgets”: low / high / max
• Weights released in MXFP4 quantization format

This combination of parameters effectively answers a critical engineering question: how to maintain ultra-large knowledge capacity while keeping the computational cost of a single forward pass within acceptable limits. The fundamental approach of MoE is that “total parameters determine capacity, while activated parameters determine computational overhead.” A total of 2.8T parameters means the model can accommodate more finely divided expert specialization, while the 104B activation means the actual parameters involved in the computation per token are significantly smaller than the total.

For deployment teams, VRAM usage is primarily dictated by total parameters and quantization precision, whereas throughput and latency depend more on the activation paths. This is why the weights are released in MXFP4 format: 4-bit floating-point quantization significantly compresses the weight footprint. Paired with an inference engine that supports this format, the model can actually be run on a relatively limited cluster of accelerators.

It must be emphasized, however, that MoE’s “sparse activation” does not equate to “free scaling.” Expert parallelism introduces extra communication overhead, routing imbalances can cause certain experts to overload, and the growth of the KV Cache in long-context scenarios becomes a new bottleneck. Therefore, the sparse attention modifications discussed below are the true key to adapting this model for long-horizon agents.

2. Sparse Attention: Driving Down the Cost of Million-Token Context

Public information indicates that the new version introduces a crucial adjustment to its attention architecture: replacing 24 layers of Gated Multi-head Latent Attention (Gated MLA) with Gated DSA (Gated Sparse Attention), combined with 69 layers of Kimi Delta Attention. DSA selects 2,048 historical tokens to participate in each attention computation.

To understand the value of this modification, we must first clarify the cost structure of long-context inference. Under standard full attention, every new token must compute its relevance against all historical tokens, meaning computational complexity grows quadratically with sequence length. Even with KV Cache reuse, the memory footprint and bandwidth pressure at the million-token scale remain substantial. The philosophy behind sparse attention is to retain long-context capabilities but only allow a subset of historical information to participate in each computation.

The setting of “selecting 2,048 historical tokens each time” can be viewed as a dynamic, retrieval-based attention: the model does not treat all history uniformly but dynamically selects the most relevant fragments based on the current query. This yields two direct benefits:

1. Controllable computation: The overhead of the attention mechanism correlates with the “number of selected tokens” rather than linearly or quadratically with the total sequence length. This ensures a more stable step-by-step cost during long-horizon tasks.
2. Memory and bandwidth efficiency: The read scope of the KV Cache is narrowed, significantly reducing memory bandwidth pressure during the decoding phase.

There are trade-offs, however. The selection mechanism itself requires additional scoring and indexing overhead. If the selection strategy is inaccurate, it may miss crucial historical information, manifesting in long tasks as “forgetting earlier agreements” or “omitting intermediate step results.” Therefore, sparse attention typically requires joint optimization during the training phase, rather than simple inference-time pruning. The mentioned “joint reinforcement learning post-training” covers hundreds of downstream domains. From an engineering perspective, a key purpose of this post-training is precisely to enable the model to make robust decisions based on sparsely visible history.

A conceptual pseudocode snippet helps illustrate the “select-then-compute” logic (representing logic, not an actual API):

# Pseudocode: Logic of sparse attention
def sparse_attention(query, history_tokens, top_k=2048):
    # 1. Score relevance of historical tokens (lightweight scorer)
    scores = relevance_scorer(query, history_tokens)
    # 2. Select top-k historical tokens
    selected = top_k_indices(scores, k=top_k)
    # 3. Compute attention only over the selected subset
    return attention(query, history_tokens[selected])

For application teams, the practical impact of this architecture is that long context is no longer “usable but expensive,” but rather “affordable but approximate.” For tasks requiring precise verbatim recall of full-text details (such as comparing contract clauses line-by-line), developers should supplement with tool calls or Retrieval-Augmented Generation (RAG) rather than relying entirely on the model’s sparse memory.

3. Three-Tier Thinking Budgets: Making Inference Cost a Tunable Parameter

Providing low / high / max thinking budgets on the inference side is a highly practical design for production environments. A “thinking budget” typically refers to the length of internal reasoning or computational volume the model is allowed to perform before generating the final answer. The higher the tier, the more the model tends to expand on multi-step reasoning, self-checking, and tool planning; the lower the tier, the faster the response and lower the cost.

The value of this mechanism lies in explicitly handing the “performance vs. cost” trade-off to the caller:

• Low tier: Suited for high-frequency, low-risk requests like intent recognition, simple Q&A, or format conversion, prioritizing Time-to-First-Token (TTFT) and throughput.
• High tier: Ideal for moderately complex analytical tasks, such as summarizing document key points or routine code modifications.
• Max tier: Designed for long-horizon agents, complex bug localization, and multi-step tool orchestration. Officially published benchmark scores default to using this tier.

In engineering practice, it is important to note that thinking budgets interact directly with context length and tool call rounds. In a multi-turn tool-calling task, enabling the max tier for every single round will significantly amplify the total cost. A sound strategy is to adjust dynamically by round—use a high tier during the planning phase, and downgrade during execution and verification phases.

4. Agent-Oriented Capabilities: Code, CAD, and Knowledge Work

Based on the breadth of public benchmarks, the model’s capability design is clearly skewed towards “long-horizon agents” rather than single-turn Q&A, primarily targeting three categories of tasks:

1. Software Engineering and Code Development

This covers terminal operations and code repair tasks. Terminal operation benchmarks assess the model’s ability to execute multi-step commands in a real command-line environment, read outputs, and retry after errors. Code repair benchmarks are closer to real-world engineering scenarios: given a repository and a bug description, the model must locate the issue and submit a patch. These tasks demand rigorous context management—retaining sufficient code context while remembering failed paths across multiple retry attempts.

2. 3D Design and CAD Generation

High scores in tool-augmented CAD generation indicate the model is viable for combined tasks involving “calling geometric tools + generating modeling scripts.” A necessary caveat: such tasks heavily depend on toolchain stability and parameter validation. The model’s output should be treated as a “first draft” and must pass geometric validity checks.

3. Knowledge Work and Office Scenarios

This supports web search, financial research, office document processing, and tool calling. Web search benchmarks assess information localization and cross-validation skills. Office document processing involves long-document parsing and structured output. For these tasks, a “retrieve-summarize-cite” workflow is recommended, requiring the model to output traceable source snippets to mitigate hallucination risks.

5. How to Interpret Benchmark Data

Public benchmark scores include: Terminal Operations 65.7, Long-horizon Software Engineering 51.7, Code Repair 75.5, Web Search 92.1, Tool-augmented CAD Generation 92.7, and Office Document Q&A 64.4. The official notes clarify that these scores default to the max thinking tier, and some evaluations utilized agentic frameworks like AI coding tools.

When interpreting these numbers, developers should maintain three points of clarity:

1. Framework impact is significant: The same model can score quite differently across various agentic frameworks. The framework’s tool design, retry strategies, and context pruning all affect the final outcome. Benchmark scores cannot be directly equated to “out-of-the-box” capabilities.
2. Different tiers are incomparable: Scores achieved at the max tier do not represent production performance at the low tier. Capacity planning should be based on re-evaluating with your own business samples.
3. Task distribution bias: Benchmark datasets differ from real-world business distributions. Extra adaptation and evaluation are necessary, especially regarding proprietary enterprise codebases and specialized document formats.

6. Deployment and Integration Essentials

The weights are released in MXFP4 quantization format, with stated support for mainstream open-source inference engines and compatibility with OpenAI/Anthropic-style APIs. For teams planning self-deployment, several points require early planning:

• Quantization format matching: Confirm the level of MXFP4 support in your chosen inference engine and accelerators. A mismatch will cause a fallback to higher precision, altering your VRAM and throughput expectations.
• Communication overhead of expert parallelism: With 2.8T total parameters, multi-GPU deployment is essentially mandatory. Intra-node and inter-node bandwidth must be thoroughly evaluated.
• Context retention for multi-turn tool calls: The official documentation specifically notes that multi-turn tool calls must retain the complete history of thoughts and tool messages. This is critical—if the middleware strips out the thought process or tool returns when concatenating requests, the model loses its state, resulting in repeated calls or broken logic. It is recommended to explicitly distinguish among four message types in the session management layer: “system prompts, historical thoughts, tool calls and results, and user inputs,” retaining them strategically.
• License compliance: The weights use a proprietary license rather than a permissive open-source one. Commercial use requires a clause-by-clause review of usage scope, redistribution conditions, and attribution requirements to avoid compliance risks.

A simplified message retention strategy is illustrated below:

# Pseudocode: Message retention strategy for multi-turn tool calls
messages = []
messages.append({"role": "system", "content": system_prompt})
def on_tool_round(user_input, thought, tool_call, tool_result):
    # Fully retain thought and tool messages without pruning
    messages.append({"role": "assistant", "content": thought})
    messages.append({"role": "tool_call", "content": tool_call})
    messages.append({"role": "tool_result", "content": tool_result})
    # Only compress early "tool_results" by summarization if the budget is exceeded
    return compress_if_needed(messages, keep_recent_rounds=3)

7. Applicable Boundaries and Selection Advice

In summary, this model is well-suited for: agent tasks requiring long-horizon multi-step execution, bug localization and repair within large-scale codebases, tool-augmented geometric modeling, and long-document processing with RAG-based Q&A. It is less suited for: latency-sensitive, highly concurrent online requests (where lower tiers or smaller models should be evaluated), tasks requiring strict verbatim full-text recall (where retrieval should be paired), and single-GPU environments with limited VRAM.

For most teams, a more pragmatic three-step path is: first, use a hosted API to conduct small-scale evaluations with business samples to confirm the optimal tier and prompt strategy; second, solidify stable task patterns into tools and workflows; finally, evaluate the cost-benefit of self-deployment. The value of open-source weights lies in control and customizability, not free compute. Factoring deployment, tuning, and operational costs into the total equation is the only way to make a sound technological choice.