Trillion-Parameter MoE Open Models: Architecture, Evaluation, and Deployment Explained

I. When Open Weights Enter the Trillion-Parameter Era

Recently, a research team fulfilled its promise by releasing the complete weights of its flagship large language model on an open-source model hosting community. Public资料显示 that this model adopts a Mixture of Experts (MoE) architecture, with a total parameter count of 2.8 trillion. During inference, it activates approximately 104 billion parameters per token and supports a context window of about 1 million tokens. On third-party agent evaluation leaderboards, it ranks first among open-weight models and leads in sub-categories like command execution, coding, and conversation.

For developers, leaderboard rankings are merely surface-level indicators. The real technical questions worth exploring are:

1. Why does a trillion-parameter model only need to activate about 3.7% of its parameters during inference?
2. How should agentic capabilities be scientifically evaluated, and what do metrics like “net improvement rate” actually mean?
3. What separates a typical engineering team from moving multi-terabyte open weights from “downloadable” to “runnable”?

This article addresses these three questions in turn.

II. MoE Architecture: Decoupling Capacity and Compute

In a dense architecture, parameter count and computational volume are linearly bound: doubling the parameters roughly doubles the inference cost per token. The core idea of the Mixture of Experts (MoE) architecture is to replace the feed-forward networks in the Transformer with multiple parallel “expert” sub-networks, introducing a gating router that dynamically selects a small subset of experts to process each token.

# MoE layer illustration (pseudo-code)
def moe_forward(x, experts, router, top_k=2):
    scores = softmax(router(x))              # Score all experts
    idx, w = top_k(scores, k=top_k)          # Keep only the top k scoring experts
    y = sum(w[i] * experts[idx[i]](x) for i in range(top_k))
    return y

Taking the case in this article as an example: with 2.8 trillion total parameters and roughly 104 billion activated per token, the activation ratio is about 3.7%. This implies three things:

• Model capacity is determined by total parameters: Massive parameters distributed across hundreds of experts carry broader knowledge and skills.
• Per-token compute is determined by activated parameters: The inference FLOPs are comparable to a 100-billion-scale dense model.
• Memory footprint is still determined by total parameters: All experts must reside in memory, which is the root cause of the deployment challenges discussed later.

In engineering, MoE also introduces new challenges like load balancing, routing jitter, and the communication overhead of expert parallelism. This is why the architecture has only truly gained widespread adoption recently as training infrastructure has matured.

III. Long Context: How Gated Sparse Attention Reduces Overhead

The bottleneck for million-level contexts lies in the attention mechanism itself. The compute and KV cache overhead of full attention grow quadratically with sequence length. When the context reaches 1 million tokens, even if the activated parameters are only at the 100-billion level, the attention component becomes the primary cost.

This model’s approach is representative: while maintaining the overall structure, it replaces some full attention layers with gated sparse attention layers. A learnable gate determines which subset of Keys each Query attends to, rather than computing attention scores against all historical tokens. Combined with industry-standard techniques like sliding window attention, Grouped-Query Attention (GQA), and KV cache compression, the overhead of long-context modeling is significantly reduced.

The takeaway for the application side is that context length is not just a “number on a spec sheet.” It determines whether an agent can accommodate an entire code repository, long-horizon task trajectories, and massive tool execution returns in a single session. When evaluating models, focus on the “effective context” (the range where recall and reasoning quality are maintained on long-document tasks) rather than just looking at the nominal window size.

IV. Post-Training: Large-Scale RL Determines the Agentic Ceiling

Pre-training determines the baseline of knowledge, while post-training shapes the capability profile. According to disclosures, this model used the previous generation of the same series as its base, undergoing large-scale joint reinforcement learning (RL) post-training covering hundreds of downstream domains.

Unlike Supervised Fine-Tuning (SFT), which “imitates demonstration answers,” RL post-training allows the model to receive feedback from execution results: whether the code passes tests, whether commands execute successfully, and whether the task is confirmed complete by the user. This type of verifiable reward naturally fits agentic scenarios—the core actions of an agent are tool calling, command execution, and multi-turn iteration, whose success or failure can usually be objectively determined. This also explains why the new generation model’s progress on “execution-type” metrics far exceeds its progress on “conversational” metrics.

V. Evaluation Paradigm: From Pairwise Blind Tests to Real-Session Signals

Traditional model leaderboards mostly use pairwise blind testing: users vote on two anonymous responses. This method is suitable for evaluating chat quality but fails to capture the most critical characteristic of agents—the execution reliability of long-horizon tasks.

Recent agent evaluations have adopted a different approach: instead of initiating dedicated votes, they extract natural signals from real user agent sessions, including:

• Explicit user confirmation of task completion;
• Praise and complaints within the session;
• Correction commands issued by the user (interrupting and requesting a redirect);
• System-level failures like command-line execution errors.

Using causal attribution methods based on randomized inference (known as “causal tracing”), they estimate the net improvement rate of each model relative to the overall average. In this case, the ranking was based on over 17,500 real long-horizon agent sessions from global users. The net improvement rate was +5.1%, compared to only +0.9% for the previous generation.

The sub-metrics are worth more attention, as they depict different facets of an agent:

Metric	Value Change	Reflected Capability
Command execution reliability	-4.5% → +7.9%	Success rate of tool calls and command execution
Praise-complaint net difference	+1.3% → +7.7%	Approximate user satisfaction
Steerability	+6.5%	Ability to adjust execution direction based on feedback
Task confirmation success rate	+3.9%	Proportion of explicit user confirmation that the task is “done”

The insight from this methodology is: evaluating agents should not only focus on whether the final answer is correct, but on the execution quality of the entire process. Enterprises can fully adopt this for internal evaluations—extracting process metrics like retry counts, correction rates, confirmation rates, and error rates from real business session logs often reflects the true capability level better than constructing a set of “test questions.” Of course, any leaderboard has biases: user demographics, task distribution, and session length will all affect the results. Leaderboards serve well as reference coordinates, but should not be the sole basis for model selection.

VI. Deployment Reality: The Gap from “Downloadable” to “Runnable”

Open weights are often misunderstood as “ runnable locally by anyone,” but a massive engineering gap exists between the two. The weight file size can be estimated with a simple formula: Storage volume ≈ Parameter count × Bytes per parameter.

def weight_size_tb(params, bytes_per_param):
    """Returns decimal TB"""
    return params * bytes_per_param / 1e12
print(weight_size_tb(2.8e12, 2.0))   # BF16: approx. 5.6 TB
print(weight_size_tb(2.8e12, 0.5))   # INT4: approx. 1.4 TB

Meaning: storing 2.8 trillion parameters in BF16 precision requires about 5.6 TB; even with INT4 quantization, it requires about 1.4 TB. This is just for the weights themselves—during inference, activations and the KV cache must be added on top. Furthermore, the KV cache for a million-token context will scale linearly with the number of layers and attention heads, easily reaching hundreds of gigabytes.

The conclusion is straightforward: a single machine with consumer-grade GPUs (typically 24GB to 80GB of VRAM) is entirely incapable of deployment. True self-deployment requires a multi-node GPU cluster, a combination of expert parallelism + tensor parallelism + pipeline parallelism, high-speed interconnect networks, and a mature quantized inference engine—conditions that only a few large institutions possess.

For the vast majority of teams, the pragmatic paths are:

1. On-demand cloud API calls: Leverage the elastic scaling of GPU compute, model services, and container platforms provided by cloud platforms like Baidu AI Cloud to avoid upfront hardware investments.
2. Local mid-scale validation: Use 7B to 70B level open-weight models with INT4/INT8 quantization. A single machine with multiple GPUs can run the process and validate ideas.
3. Distillation and fine-tuning for specialized capabilities: Use large models to generate high-quality data, then train smaller models to handle high-frequency, narrow-domain tasks, balancing cost and latency.
4. Read the license terms carefully: Open does not mean unrestricted commercial use. Differences in licensing scope can directly impact technology selection.

VII. Rational Interpretation of Benchmark Scores

On a terminal agent benchmark that evaluates task completion capabilities in real command-line environments, this model achieved a score of 65.7%, higher than a frontier closed-source model’s 58.2%, and less than 1 percentage point behind another closed-source flagship’s 66.4%. The gap between open-weight models and frontier closed-source models in agentic scenarios is narrowing, which is an undeniable trend.

However, one must be wary of “single-point score worship.” The task distribution of benchmarks often differs greatly from real-world business scenarios, and a few percentage points of difference might stem from the style of the prompts rather than a fundamental difference in capability. The correct engineering approach is: treat public leaderboards as a map, and your internal evaluation sets as a ruler—measure candidate models using real tasks and real logs from your own business.

VIII. Conclusion: The Value of Openness and Engineering Fundamentals

Trillion parameters, open weights, and topping agent leaderboards—these keywords collectively point to a structural shift in the LLM industry: frontier capabilities are gradually transitioning from the “black-box service” of closed APIs to a dual-track model of “open weights + cloud hosting.” For researchers, open weights mean that technical routes are reproducible and improvable. For application developers, its greater practical significance lies in intensifying competition in inference services, driving compute costs down continuously.

But regardless of how open the weights are, the fundamentals of implementation remain unchanged: clear scenario definitions, process-oriented evaluations close to business realities, solid agent engineering (tool call retries, error recovery, context compression), and accurately calculating the cost ledger where “total parameters determine memory and activated parameters determine compute.” Understanding these is far more valuable than memorizing any leaderboard ranking.