Trillion-Parameter MoE Open Weights: Agent Evaluation and Deployment Explained

1. Background: Open weights enter the trillion-parameter era

A large model research team recently released the complete weights of its flagship model on an open model hosting community, following through on an earlier commitment. The model uses a mixture-of-experts (MoE) architecture with roughly 2.8 trillion total parameters and about 104 billion activated parameters per inference step. Its context window is approximately 1 million tokens. On an agent evaluation benchmark built from real usage signals, the model ranks first among open-weight models and 13th among all evaluated models, including closed-source ones. It also ranks first among open models in the work, coding, and chat categories.

This matters not simply because another large model has open weights. It sits at the intersection of three broader trends:

• Capability: The gap between open-weight models and leading closed-source models keeps narrowing. The competitive focus is shifting from “how well it chats” to “whether it can finish agent tasks.”
• Evaluation: The limits of traditional pairwise blind testing are becoming increasingly clear. Evaluation methods based on real conversation signals are turning into a more useful benchmark.
• Engineering: Once parameter counts reach the trillion scale, the gap between “downloadable” and “runnable locally” widens dramatically. Deployment decisions now require a different kind of cost accounting.

The rest of this article examines these three threads.

2. MoE architecture: Total parameters and activated parameters are two separate accounts

The core idea of the mixture-of-experts architecture is to replace the feed-forward network layers in a Transformer with multiple parallel expert sub-networks. A gating router then dynamically selects a small number of experts for each token. As a result, the model’s knowledge capacity is determined by its total parameter count, while per-step inference cost depends only on the activated parameters. This is precisely why trillion-scale models are feasible in engineering terms. If a dense architecture activated all 2.8 trillion parameters on every forward pass, inference would be far too expensive to serve.

# MoE forward pass example (Top-K routing)
def moe_forward(x):
    gate = softmax(router(x))            # Router scores experts for each token
    w, idx = topk(gate, k=2)             # Only Top-K experts are activated
    y = sum(w[i] * expert[idx[i]](x) for i in range(2))
    return y

For the model discussed here: 2.8 trillion total parameters, about 104 billion activated per token, meaning an activation ratio of roughly 3.7%. Two separate calculations matter:

• Storage and memory: All expert parameters must reside in memory or storage, scaling with total parameter count.
• Compute and latency: Each token only executes matrix operations across about 104 billion parameters, scaling with activated parameter count.

For inference serving, this creates three typical engineering problems. First, expert parallelism and cross-GPU all-to-all communication scheduling. Second, uneven router load causing some experts to become “hot” bottlenecks. Third, batching strategies that must constantly trade off throughput against latency. Common solutions such as expert parallelism, redundant expert replicas, and load-balancing auxiliary losses are now standard in large model inference systems. But at trillion-parameter scale, the engineering cost of each technique scales up proportionally.

3. Long context: How gated sparse attention reduces cost

A context window of roughly 1 million tokens places harsh demands on the attention mechanism. Standard full attention scales quadratically with sequence length, while the KV cache grows linearly. A rough estimate follows this pattern:

KV cache bytes ≈ 2 × layers × KV heads × head dimension × sequence length × bytes per element

Stretching sequence length from 10,000 to 1 million, then multiplying by concurrent requests, puts enough memory pressure to overwhelm any single-machine solution.

The mainstream industry approach is “hybrid attention”: keep full attention only in some layers to preserve global information flow, and replace the remaining layers with lower-cost attention variants such as sliding window attention, local attention, or linear attention. The model in question uses one such method: replacing some full attention layers with gated sparse attention layers. Through learnable gating parameters, each query interacts only with a small set of dynamically selected keys and values. This significantly compresses attention computation and cache overhead for long sequences while using the gating mechanism to preserve selection quality and mitigate information loss from hard sparsity.

For application developers, the takeaway is that long-context capability cannot be judged by nominal window size alone. Effective context length, recall stability in long-document retrieval, and the time-to-first-token and cost curves under long inputs all matter. Together, these determine whether a “million-token window” is genuinely usable and affordable in production.

4. Large-scale reinforcement learning post-training for agents

A base model has only “read books.” Post-training is what determines whether it can reliably complete real agent tasks. According to the team, this model was built on the previous generation and underwent large-scale joint reinforcement learning covering hundreds of downstream domains. Two terms deserve closer attention.

First, “joint.” Putting tasks from hundreds of domains into a single reinforcement learning process, rather than fine-tuning each domain separately, is intended to enable capability transfer and reduce catastrophic forgetting. Fine-tuning on a single domain often comes at the cost of regressions elsewhere. Joint training forces the model to find more general strategies across multiple objectives. This also matches the real usage pattern of agents, where the model may be asked to do almost anything.

Second, “agent-oriented reward design.” Agent tasks are long-chain, multi-turn, and tool-invoking: the model must read documents, write code, execute shell commands, and correct itself based on errors. Reward signals for such tasks typically come from three sources:

• Executable verification: command exit codes, test pass/fail results, and whether the task artifact is runnable;
• Process supervision: scoring intermediate steps to ease the credit assignment problem under sparse rewards;
• User feedback: confirmation, praise, and complaints after task completion.

Notably, the signals used during training and the signals collected during evaluation are fundamentally the same. This is the key to understanding the evaluation paradigm shift in the next section.

5. Evaluation paradigm shift: From pairwise blind tests to real conversation signals

For a long time, the dominant evaluation method for large models was pairwise blind testing: human or model judges vote between two anonymized responses. This works reasonably well for chat scenarios but has obvious flaws for agent evaluation. Judges can be swayed by surface features such as formatting, length, and tone. Voting mostly happens in single-turn or short contexts, so it cannot reflect actual completion of long-horizon tasks. More importantly, there is an entire execution gap between “looks good” and “actually correct.”

As a result, mainstream agent evaluation platforms are shifting to a different approach: instead of running dedicated votes, they extract signals from real user agent conversations, including:

• Explicit user confirmation of task completion;
• Praise and complaints during conversations;
• Corrective instructions from users, since the need for correction itself indicates that prior execution deviated from expectations;
• Objective failure signals such as shell command execution errors.

At the statistical level, such methods use randomized inference, often called “causal tracking,” to estimate each model’s net improvement rate relative to the average. This helps control for confounding factors such as task difficulty and user population distribution. Conversation signals do carry noise, including user expression ambiguity and attribution difficulty. Randomized inference and sufficiently large sample sizes are therefore prerequisites for the method to work. Compared with pairwise blind testing, it measures whether end-to-end tasks were actually completed, which aligns far better with production environments.

Public data provides an example. One benchmark release was based on more than 17,500 real long-horizon agent task conversations from global users. The top-ranked open-weight model had a net improvement rate of +5.1%, while its previous generation scored only +0.9% — a 4.2 percentage point gain in one generation. Category-level metrics are even more revealing:

Metric	Previous generation	New generation	Interpretation
Shell command execution reliability	-4.5%	+7.9%	Turned from negative to positive; ranked first among open models
Net praise vs. complaints	+1.3%	+7.7%	Significant improvement in subjective satisfaction
Controllability	—	+6.5%	Ability to adjust direction based on feedback; first among open models
Confirmed task completion	—	+3.9%	The strictest end-to-end metric; fourth among open models

Three takeaways:

1. Shell reliability turning from negative to positive shows that the “physical layer” stability of tool invocation is the foundation of agent reliability. If commands fail, subsequent reasoning quality hardly matters.
2. High controllability means users can pull the model back on track with fewer correction rounds, directly lowering human-machine collaboration costs.
3. Completion confirmation scoring lower than other categories is intuitive: end-to-end completion is hardest because the entire chain must work.

Separately, public terminal agent benchmark data shows this model scored 65.7%, surpassing several leading closed-source models on comparable tasks and falling less than 1 percentage point short of the top score. In agent capability, the gap between open and closed models is narrowing rapidly.

For developers, the methodology matters more than the ranking itself. The same signal-collection logic can be built into an agent system: record task confirmation events, correction rounds, and tool invocation failure rates. Use conversation-level data rather than single-turn scores to drive iteration.

6. Open weights do not mean runnable everywhere: The deployment ledger for trillion-parameter models

Open weights are often assumed to mean “download and run.” A trillion-parameter model shatters that illusion. Start with simple arithmetic:

total_params = 2.8e12
bf16_tb = total_params * 2 / 1e12    # BF16: 2 bytes per parameter → about 5.6 TB
int4_tb = total_params * 0.5 / 1e12  # INT4: 0.5 bytes per parameter → about 1.4 TB

Take a high-end inference server with 8 GPUs and 80GB per GPU, totaling 640GB of memory. A rough estimate:

• INT4 quantization reduces weights to about 1.4TB. Housing the weights alone requires at least 3 machines. Add KV cache, activations, and communication buffers, and the practical starting point is a small cluster.
• BF16 full precision is about 5.6TB. Weights alone require 9 or more machines. With redundancy and caching, this becomes a data-center-scale project.

The conclusion is straightforward: consumer-grade single-machine deployment is not possible. Even for many small and mid-sized organizations, 1.4TB of INT4 weights far exceeds single-machine memory. The 5.6TB original weights make downloading, sharding, checksumming, and resumable transfer an engineering project in their own right. Three realistic paths remain:

1. Quantization compression: INT4 or FP8 quantization can shrink the footprint several times over. But precision loss after calibration must be evaluated, especially on code and math tasks sensitive to numerical error, with regression validation.
2. Expert offloading and tiered storage: Keep low-frequency experts in CPU memory or NVMe, and swap them into GPU memory based on routing results. Technically feasible, but paging latency can significantly raise inference delay. This suits offline batch processing better than online serving.
3. Cloud hosting and elastic compute: The most realistic option for most teams. Cloud platforms commonly provide GPU clusters, containerized deployment, object storage, logging, and monitoring. Weight shards live in object storage, multi-machine inference services run on a container platform, and autoscaling plus alerting turn a heavy capital expenditure problem into an operational budget problem.

The build-versus-cloud decision comes down to three questions. Do you genuinely need to modify the model itself through continued training or deep customization? Do you have sustained, large-scale traffic to amortize fixed costs? Do you have an engineering team capable of operating a multi-machine inference cluster? If all three are true, self-hosting may be justified. Otherwise, the value of open weights lies more in auditability, trustworthiness, and portability than in running the model yourself.

7. Action items for developers and enterprises

Evaluation

• Instrument agent systems: task confirmation rate, number of corrective instructions, tool invocation failure rate, and praise-to-complaint ratio.
• Replace single-turn scores with conversation-level metrics, focusing on end-to-end completion of long-horizon tasks.
• Build relative baselines such as net improvement rate rather than relying only on absolute scores.

Engineering

• Treat shell command execution reliability as a foundational metric and include it in regression testing.
• Design correction-friendly human-machine interaction: make the model easier to correct and make user corrections less costly.
• When selecting long-context models, evaluate nominal window, effective window, and cost curve together.

Model selection

• For MoE models, first ask about activated parameter count. That determines inference cost. Total parameter count determines knowledge capacity.
• Open weights do not equal self-deployable. Calculate storage and memory requirements before deciding.
• When compliance audits, data residency, or deep customization are required, “open weights plus dedicated cloud compute” is a common compromise.

The second half of the agent race is no longer about who answers most elegantly, but about who gets the job done. Evaluation methods, training paradigms, and deployment engineering are reorganizing around that goal. For developers, that is the most important lesson from the current wave of trillion-parameter open-weight releases.