Trillion-Parameter MoE Model Weights Released: A Full Analysis of Quantization, Deployment, and Evaluation

1. From Model Release to Open Weights: What Happened?

The competition in large language models is shifting from “parameter scale” to “accessibility.” Recently, a leading team released an MoE model with approximately 2.8T total parameters and around 104B activated parameters per token. As promised, they later uploaded the complete weights to an open-source model hosting platform, allowing any team to download, fine-tune, or audit them directly. On the same day, third-party benchmark results revealed that the model ranked 1st among open-weight models and 13th overall in evaluations based on real-world Agent sessions.

For engineering teams, “open weights” implies at least three key benefits:

• Auditable: Security and compliance teams can directly inspect weight files and reproduce inference behaviors without relying solely on the vendor’s claims.
• Customizable: Teams can perform parameter-efficient fine-tuning or domain-specific continued pre-training on proprietary data, translating general capabilities into business value.
• Controllable Deployment: Data remains entirely on-premises, making it suitable for highly regulated industries like finance, healthcare, and government.

The public specifications of the model can be summarized as follows:

• Total parameters: ~2.8 trillion (approximately 2.78 trillion based on weight files)
• Activated parameters per token: ~104B
• Context length: 1 million tokens, with support for text and image inputs
• Weight repository size: ~1.56TB

2. Architecture Breakdown: Synergy of Sparse Activation and Sparse Attention

1. Hybrid Attention: Fast and Slow Division of Labor Across 93 Layers

The model consists of 93 layers: 69 layers utilize a linear attention-like structure, while 24 layers employ gated sparse attention. According to official documentation, compared to its predecessor which primarily used full-attention gated layers, this generation replaces them with sparse alternatives to directly tackle the high costs of long-context modeling.

The underlying technical logic is straightforward:

• Full attention has a computational complexity that grows quadratically with sequence length, and its KV cache memory footprint grows linearly. Under a 1 million token context, this is unfeasible for standard deployments.
• Sparse attention restricts each query to interact only with a selected subset of keys and values, reducing both memory and computation to a sub-linear level.
• Linear attention uses a fixed-size hidden state to summarize history, which is highly efficient for ultra-long streaming sequences.

The hybrid design retains strong global representation capabilities in a minority of layers while leveraging the majority to achieve low-cost long-context processing, striking a balance between quality and overhead. This “hierarchical heterogeneous attention” is becoming a mainstream engineering approach for long-context models.

2. MoE Expert Layers: Decoupling Capacity from Compute

The expert layers are configured with 896 experts. For each token, the router selects 16 experts, and 2 shared experts are always active. The essence of sparse activation is the decoupling of “knowledge capacity” and “per-token compute”: the 2.8T total parameters dictate how much the model “remembers,” while the 104B activated parameters dictate “how fast it thinks per step.” The shared experts carry cross-task common knowledge, reducing redundant routing among specialized experts and aiding in training stability.

3. Vision Module: ~400M Parameter Image Encoder

On the vision side, the model is equipped with a visual encoder containing around 400 million parameters, which encodes images into feature sequences aligned with the main language trunk. For Agent scenarios, tasks like screenshot comprehension and document/chart parsing can be natively executed without bolting on an independent vision pipeline.

3. Quantization Design: MXFP4 Weights and MXFP8 Activations

The weights are stored in MXFP4 format and activations use MXFP8, with official sources confirming the use of Quantization-Aware Training (QAT). This is arguably the most noteworthy engineering detail of this release.

1. What is the MX Format?

MX (Microscaling) is an open, low-bit numerical standard in the industry. It groups a block of elements (typically 32) and shares a single high-precision scaling factor within the block, while the elements themselves are stored in 4-bit or 8-bit precision. Compared to traditional per-tensor or per-channel quantization, block-level scaling can track local numerical distributions, effectively mitigating the accuracy collapse caused by outliers—this is the critical reason why 4-bit quantization is viable today.

2. QAT vs. PTQ

Post-Training Quantization (PTQ) at 4-bit usually results in noticeable accuracy degradation. QAT, on the other hand, inserts quantization simulation during the training phase, forcing parameters to learn within a noisy quantization environment and effectively “absorbing” the quantization error into the weights themselves. Conducting QAT on a 2.8T scale model incurs massive engineering costs, but it delivers the practical benefit of “out-of-the-box deployment with 4-bit weights.”

3. The Storage Ledger

A simple calculation illustrates its value: storing ~2.78 trillion parameters in BF16 (2 bytes per parameter) requires about 5.6TB. Using MXFP4 (about 0.5 bytes per parameter plus block scaling overhead), the actual repository footprint is approximately 1.56TB, saving roughly 70%. The weights are organized as sharded files, natively supporting resumable downloads, parallel fetching, and memory-mapped loading.

4. Deployment Barriers: Memory Ledger and Three Implementation Paths

1. How to Calculate VRAM

The 1.56TB weight is merely the baseline. Factoring in the KV cache (which grows rapidly with long contexts), activations, and framework overhead, third-party developers estimate a viable configuration requires 8 next-generation data center accelerator cards with 288GB of VRAM each. This totals about 2.3TB of VRAM, with an estimated hardware cost of $550,000 to $650,000. Note that this is a third-party estimate; the vendor has not provided official hardware recommendations, and actual configurations should be determined via stress testing.

2. Software Side Compatibility

While MXFP4 weights compress storage, the inference engine must possess corresponding efficient de-quantization and matrix multiplication kernels. Furthermore, MoE’s dynamic routing introduces challenges in expert parallel scheduling and memory fragmentation management. Several mainstream open-source inference engines have recently released deployment guides for this model. When selecting an engine, priority should be given to verifying its support for MX formats and its actual measured throughput.

3. Three Implementation Paths

• Fully On-Premises: Suitable for teams with strict data compliance mandates and deep fine-tuning requirements, though CapEx and operational thresholds are exceptionally high.
• Cloud Elastic Deployment: Spin up inference services using cloud GPU instances, container services, model hosting, and object storage (for weight shards). Cloud platforms like Baidu AI Cloud provide such general capabilities, where pay-as-you-go pricing can significantly lower the cost of trial and error.
• Direct API Access: The model offers standard and high-speed versions, with the latter reportedly being 2 to 3 times faster. This path is ideal for rapid prototyping and the majority of online business operations.

5. API Usage Patterns: Two Common Pitfalls

1. Inference Depth Levels

Inference depth supports three tiers: low, high, and max (default). The max tier allows for deeper reasoning and more stable outputs but incurs higher latency and token costs. In high-concurrency online scenarios, the tier can be downgraded based on task complexity—for instance, using low for customer service Q&A and high for code generation.

2. Multi-Turn Tool Calls Must Return the Chain of Thought

In Agent scenarios involving multi-turn tool calls, the reasoning_content (chain of thought) and tool_calls from the previous turn must be passed back. The model’s intermediate reasoning state serves as the context for its subsequent decisions; if discarded, the model may either repeat its reasoning or exhibit behavioral drift. This is a standard convention for current reasoning model APIs and a frequent source of errors in Agent engineering. A structural example is shown below:

messages = [
    {"role": "user", "content": "Troubleshoot the spike in 5xx errors on production"},
    {"role": "assistant",
     "reasoning_content": "...",   # Chain of thought from the previous turn, must be passed back as-is
     "tool_calls": [{"name": "query_metrics", "arguments": "..."}]},
    {"role": "tool", "content": "..."}  # Tool execution result
]
resp = chat(model="<model-id>", messages=messages, reasoning_effort="high")

6. Evaluation Perspective: “Net Lift” Driven by Real Sessions

1. Why Static Benchmarks Fall Short

Traditional static benchmarks face three major dilemmas: data contamination, score saturation, and detachment from real-world task distributions. Consequently, benchmark scores often diverge from actual user experience.

2. How Real Session Evaluation Works

The benchmark publishing these results sources its data from real-world, long-horizon Agent sessions (this round of statistics was based on over 17,500 sessions). It extracts several types of signals: whether the user explicitly confirmed task success, user praise or complaints, the model’s response quality following a user correction, and whether command-line instructions executed successfully. These are synthesized into a “Net Lift” metric—representing the net improvement in task completion attributable to the model’s intervention.

3. Interpreting the Results

• The model (max tier) achieved a Net Lift of +5.1%, ranking 1st among open-weight models and 13th overall. It also ranked 1st among open models in the Work, Code, and Chat subcategories.
• The previous generation model recorded +0.91% under the same metrics, ranking 6th among open models.
• In sub-metrics, command-line execution reliability scored +7.9%, ranking 1st among open models (compared to -4.5% and 13th for the previous generation).

Ranking “13th overall” indicates a persistent gap compared to the strongest closed-source flagship models. However, ascending from 6th to 1st within the open-weight category, and shifting command-line reliability from a negative to a positive score, marks a definitive generational leap. For enterprises, the evaluation methodology itself is arguably more valuable: rather than blindly trusting leaderboards, teams should build private evaluation sets using real business sessions and real tool calls, measuring model value via “Net Lift” rather than single-point accuracy.

7. Actionable Checklist for Engineering Teams

1. Clarify Requirements First: Only consider weight-level deployment if data compliance, offline operation, or deep fine-tuning is required. Otherwise, start with the API.
2. Calculate the Math: For local deployment, VRAM = weights + KV cache + activations + redundancy. For API, calculate costs based on token pricing and call volume.
3. Progressive Rollout: Validate feasibility via API -> Validate workflows via parameter-efficient fine-tuning -> Evaluate private deployment.
4. Evaluation First: Accumulate real session data, define a Net Lift metric, and conduct A/B testing for any model switching.
5. Verify Kernels: Confirm that the chosen inference engine supports MXFP4/MXFP8 and verify its actual throughput to avoid the trap of “downloads fine, runs poorly.”

Conclusion

The release of open weights for a trillion-parameter model signals that cutting-edge capabilities have entered the “downloadable” era. The real barrier is shifting from “whether weights are available” to “how to deploy them, how to evaluate them, and how to handle the nuances of Agent engineering.” By clearly calculating the four key aspects—quantization, VRAM, protocols, and evaluation—engineering teams can successfully translate the technological dividends of the open-weight wave into tangible business value.