Open-Weights at Trillion-Parameter Scale: Engineering Realities of a 2.8T MoE Model

1. A New Milestone for Open Weights: From Parameter Count to Capability Validation

Recent years have seen a steady climb in large model parameter counts, but the industry has gradually converged on a key point: parameter count alone does not equal practical capability. What truly determines a model’s value is its performance on real tasks—especially long-horizon, multi-step agentic work.

A recent ultra-large model built on a Mixture-of-Experts (MoE) architecture has been released with open weights: 2.8 trillion total parameters, roughly 104 billion activated per inference, and a context window of about 1 million tokens. This marks open-weight models officially entering the “trillion-parameter” era. But more noteworthy than the parameter count is its performance on agent evaluations: on an agent leaderboard built from real user sessions, the model ranks first among open-weight models, thirteenth overall, and first among open models in all three categories of work, coding, and chat.

This reflects a broader trend: competition among open-weight models is shifting from leaderboard optimization toward reliability on real tasks.

2. Rethinking Evaluation: From Blind Pairwise Voting to Causal Tracing

Traditional model evaluation often relies on pairwise blind tests in which users vote on two responses. This is intuitive but has clear limits: user preference does not equal task success, and performance in short conversations does not equal long-horizon agentic capability.

A newer generation of agent evaluation takes a different approach. Instead of asking users to vote, it extracts signals from real users’ agent sessions, including:

• Explicit user confirmation of task completion;
• Praise and complaints;
• Corrective instructions;
• Shell command execution errors.

It then uses a randomized inference method called “causal tracing” to estimate each model’s net improvement rate relative to the average. The advantage of this approach: signals come from real usage scenarios rather than artificially constructed test sets, and capabilities such as error accumulation over long-horizon tasks, instruction following, and self-correction can all be captured.

The model’s ranking is based on more than 17,500 real long-horizon agent task sessions from users worldwide, with a net improvement rate of +5.1%—a 4.2 percentage point gain over the previous generation of the same model family, which scored +0.9%. That magnitude of improvement is substantial in real-task evaluation.

3. Reading the Sub-Metrics: Reliability, Satisfaction, and Steerability

Beyond the overall net improvement rate, the sub-metrics better illustrate the direction of capability evolution.

Shell command execution reliability: up from -4.5% (13th place) in the previous generation to +7.9%, ranking first among open models. Shell command execution is a core capability for terminal agents, involving command syntax, environment interaction, error handling, and more. A negative value means the model performs below average on this dimension, while +7.9% indicates a qualitative improvement in reliability on real terminal tasks.

User satisfaction (praise-to-complaint ratio): up from +1.3% to +7.7%. This metric directly reflects users’ subjective experience of model output; the gain suggests the model is more stable at understanding intent and generating useful responses.

Steerability: reaching +6.5%, first among open models. Steerability measures whether a model can adjust its execution direction based on user feedback. In long-horizon tasks, users often need to correct model behavior multiple times; a highly steerable model converges to the correct path faster.

Explicitly confirmed task completion rate: +3.9%, fourth among open models. This metric is relatively conservative, indicating there is still room for improvement under the strict standard of “user explicitly confirms task completion.”

4. Architecture and Training Strategy: Sparse Activation and Long-Context Optimization

The model uses a Mixture-of-Experts architecture with 2.8 trillion total parameters and roughly 104 billion activated per inference. The core idea of MoE is to activate only a subset of experts per inference through a gating mechanism, preserving large knowledge capacity while controlling compute overhead.

Compared with the previous generation, the model underwent large-scale joint reinforcement learning post-training on top of the original model as a base, covering hundreds of downstream domains. In addition, some full-attention layers were replaced with gated sparse attention layers to reduce the cost of long-context modeling. A 1-million-token context window means the model can handle very long documents, multi-turn complex conversations, and large codebases—but since the computational complexity of full attention grows quadratically with sequence length, sparse attention is a necessary engineering trade-off.

5. The Reality of Open Weights: Downloadable Does Not Mean Locally Runnable

“Open weights” is often misunderstood as “runnable locally however you like.” In practice, there is a significant gap between downloadable and locally runnable.

Weights for 2.8 trillion parameters stored in BF16 precision total about 5.6TB. Even with INT4 quantization, roughly 1.4TB is still required. Single-machine consumer GPUs are essentially not viable for deployment. For most developers and enterprises, the more realistic path is calling a cloud inference service rather than local deployment.

If local deployment is truly required, it typically demands a high-end multi-node, multi-GPU inference cluster, combined with quantization, distillation, expert pruning, and similar techniques. Below is a conceptual deployment assessment checklist:

1. Storage requirements: ~5.6TB for BF16, ~2.8TB for INT8, ~1.4TB for INT4
2. VRAM requirements: depends on the parallelism strategy; typically multi-GPU tensor parallelism + pipeline parallelism
3. Inference framework: must support MoE sparse activation and long-context attention optimization
4. Quantization scheme: INT4/INT8 require evaluating accuracy loss, especially for long-horizon tasks
5. Serving: consider batching, KV cache management, and request scheduling

6. Terminal Agent Benchmark Performance: The Gap to Top Models

On a terminal agent benchmark, the model scores 65.7%, above a mainstream closed model’s 58.2% and within 1 percentage point of another top closed model’s 66.4%.

This result is notable: open-weight models have come close to top closed models on terminal agent tasks. Terminal tasks typically involve complex command-line operations, filesystem interaction, error diagnosis and repair, and place extremely high demands on a model’s reasoning ability, tool use, and robustness.

7. How Developers Should Evaluate and Use It

Faced with ultra-large open-weight models like this, developers should evaluate rationally:

1. Clarify the task type: if the task involves long-horizon agents, terminal automation, or large-scale code understanding, the model’s capability gains are worth attention; for simple Q&A or short text generation, such a large model may not be necessary.
2. Assess deployment cost: the local deployment barrier is extremely high; prioritize cloud inference services. If localization is mandatory, carefully account for hardware, power, and operations costs.
3. Watch quantization accuracy: INT4 quantization reduces storage requirements but may affect reasoning stability in long-horizon tasks; validate on real tasks.
4. Use open weights for customization: the value of open weights lies in fine-tuning and distillation. Enterprises can use the model as a teacher to distill smaller models suited to their own scenarios.
5. Track evaluation continuously: agent evaluation methods are still evolving; pay attention to real-task signals rather than a single leaderboard score.

8. Conclusion

The open-weight release of an ultra-large Mixture-of-Experts model is an important step forward for the open ecosystem. It demonstrates that open models can approach top closed models on real agent tasks. But developers should also recognize clearly: the deployment barrier of 2.8 trillion parameters, the cost of long-context inference, and the accuracy loss from quantization are all problems that must be faced in practice.

The rational path is to understand the model’s capability boundaries, match them against your own task requirements, and choose an appropriate calling or deployment approach. The significance of open weights is not that everyone can run it locally, but that the entire ecosystem can innovate on top of it.