Open Weights for a 2.8T-Parameter MoE Model: Agent Evaluation and Deployment in Practice

In recent years, as large language models evolve from single-turn conversation tools into agents that execute complex tasks, instruction-following ability, tool-call reliability, and error recovery over long-horizon tasks have become central concerns for the industry. A research team has now released, as planned, the weights of a mixture-of-experts model with 2.8 trillion total parameters. On a well-known industry agent leaderboard, the model not only took first place among open-weight models, but also ranked first in its tier across three core scenarios: work, coding, and chat. This article analyzes the logic behind this technical breakthrough and its practical value for developers from four perspectives: evaluation methodology, architecture evolution, core capabilities, and engineering deployment.

1. Evaluation Methodology: From Blind Testing to Real Conversation Signal Extraction

Traditional LLM evaluation relies heavily on pairwise blind tests, in which users are shown two model responses and vote for the better one. While this reflects subjective preference to some extent, it struggles to accurately measure a model’s engineering reliability on real agent tasks. This leaderboard introduces a very different evaluation paradigm, centered on a “causal tracing” inference method.

1. Real conversation signal extraction: Evaluation no longer depends on subjective votes. Instead, multi-dimensional signals are extracted from more than 17,500 real long-horizon agent task sessions contributed by users worldwide. These signals include explicit user confirmation that a task was completed, spontaneous praise and complaints, corrective instructions issued during execution, and the frequency of errors in underlying Shell command execution.
2. Net improvement rate estimation: Using a randomized-inference “causal tracing” method, the system strips out environmental noise and estimates each model’s net improvement rate relative to the baseline average. This model reached +5.1% on this metric, a jump of 4.2 percentage points over the +0.9% of the previous-generation base model.
3. Detailed sub-metrics:
   • Bash Reliability: Rose sharply from -4.5% in the previous generation to +7.9%, indicating a significant improvement in the accuracy and safety of system-level operation commands and a reduction in destructive operations caused by hallucination.
   • Praise vs Complaint: Climbed from +1.3% to +7.7%, reflecting a qualitative change in user experience.
   • Steerability: Reached +6.5%, meaning the model can adjust its execution path more precisely based on mid-task user feedback—critical for multi-round Agent tasks.
   • Confirmed Success: Held steady at +3.9%, demonstrating solid end-to-end task completion capability.

2. Architecture Evolution: Mixture of Experts and Gated Sparse Attention

Supporting agent tasks at this scale requires fundamental optimization of the underlying architecture. The model adopts an advanced mixture-of-experts architecture and underwent deep restructuring during post-training.

1. Balancing massive parameters with sparse activation: The model has 2.8 trillion total parameters but activates only about 104 billion per inference. This sparse activation mechanism preserves large knowledge capacity and complex reasoning ability while keeping the computational complexity of a single inference comparable to that of dense models in the hundred-billion-parameter range, greatly optimizing inference latency.
2. Gated sparse attention: To support a context window of roughly one million tokens, the model replaces traditional full-attention layers. The computational cost of full attention grows quadratically with context length, whereas gated sparse attention uses a dynamic routing mechanism so that only the current query token interacts with the most relevant key-value pairs. This substantially reduces memory footprint and compute for long-context modeling and also mitigates the “lost in the middle” problem in long-text scenarios.
3. Large-scale joint reinforcement learning post-training: Starting from the base model, the team conducted large-scale joint reinforcement learning post-training spanning hundreds of downstream domains. This training approach enables the model to learn an action-space distribution better aligned with human preferences and engineering logic, which is the fundamental reason for its strong performance in coding and work scenarios.

3. Capability Leap: Real-World Performance on Terminal Agent Benchmarks

When evaluating a model’s ability to act as an autonomous terminal agent, the industry often uses demanding terminal agent benchmarks. On this test, the model scored 65.7%, significantly surpassing the 58.2% of a leading closed-source model and coming within less than 1 percentage point of the 66.4% achieved by a top international model.

These figures show that the model can already match or even exceed some first-tier closed-source models in understanding complex operating system environments, writing multi-step execution scripts, and autonomously debugging errors and rolling back. For developers building local automated operations or automated data-cleaning pipelines, an open-weight model with equivalent capability offers a more autonomous approach to data privacy protection.

4. Deployment in Practice: Overcoming the Engineering Bottlenecks of TB-Scale Weight Files

While open weights bring technical benefits to the industry, “downloadable” does not mean “locally runnable.” Deploying a model with 2.8 trillion parameters presents formidable hardware barriers.

1. The VRAM capacity wall: Stored in BF16 precision, the model’s weight files total roughly 5.6 TB. Even with aggressive INT4 quantization, the files still require about 1.4 TB. This means a single workstation—or even several equipped with consumer-grade GPUs—cannot load the model at all; enterprise-grade compute clusters are required.
2. Cloud-native distributed inference architecture: For deploying such large MoE models, developers need elastic compute and distributed frameworks provided by cloud services. A typical deployment architecture looks like this:
   • Compute layer: Build an inference cluster from multiple professional accelerator cards with large VRAM (e.g., 80 GB class).
   • Storage and network layer: Use high-performance object storage on the cloud platform to host TB-scale weight files, and distribute parameters to compute nodes at high speed over high-bandwidth networks (e.g., RoCE or InfiniBand).
   • Scheduling layer: Use a container platform (e.g., Kubernetes) to manage inference instances and achieve dynamic load balancing of expert routing. As activated parameters move between nodes, message queues and monitoring/alerting components ensure low-latency coordination of tensor parallelism and pipeline parallelism.

Below is a conceptual example of distributed inference service startup logic (based on generic container orchestration syntax):

# Conceptual example of distributed inference deployment for a large MoE model
apiVersion: apps/v1
kind: Deployment
metadata:
  name: moe-inference-cluster
spec:
  replicas: 4 # Configure multi-node coordination based on VRAM requirements
  template:
    spec:
      containers:
      - name: inference-engine
        image: custom-llm-runtime:latest
        resources:
          limits:
            nvidia.com/gpu: 8 # Mount multiple high-VRAM accelerators per node
        env:
        - name: MODEL_PATH
          value: "s3://enterprise-bucket/models/2.8t-moe-int4/" # Path to weights in object storage
        - name: TENSOR_PARALLEL_SIZE
          value: "8"
        - name: EXEPERT_PARALLEL_SIZE
          value: "4"
        volumeMounts:
        - name: dshm
          mountPath: /dev/shm # Shared memory to accelerate cross-node expert communication
      volumes:
      - name: dshm
        emptyDir:
          medium: Memory

5. Conclusion and Outlook

The open-weight release of the 2.8-trillion-parameter mixture-of-experts model and its top placement on the agent leaderboard mark a new stage in open models’ ability to execute complex tasks. By combining gated sparse attention with reinforcement learning post-training, the model achieves a qualitative leap in long-context processing, Shell command reliability, and steerability.

However, the deployment barrier created by massive parameter scale remains. While enjoying the technical autonomy that open weights provide, developers must build a complete engineering stack based on professional cloud-native compute, object storage, and distributed scheduling. As model quantization techniques and distributed inference frameworks continue to mature, there is good reason to believe that such top-tier agent models will empower automated business pipelines across industries at lower cost.