Inside a 2.8T-Parameter Open-Weight Model: Sparse MoE, MXFP4 Quantization, and Real-Session Evaluation

1. Background: Open Weights Enter the Trillion-Parameter Era

A recent open-weight model has publicly released its full weights alongside a companion model card detailing architecture parameters, multi-model benchmark comparisons, and the full license text. With 2.8 trillion total parameters, it ranks among the largest openly accessible open-weight models available today.

The “full weights + model card + license” trifecta is becoming the standard practice for open ecosystems around very large models:

• Full weights: Enable private deployment, continued training, and deep customization without data leaving the premises.
• Model card: Discloses architecture parameters and evaluation methodology, improving transparency and reproducibility.
• License: Defines commercial boundaries, reducing legal uncertainty.

For enterprises, the core value of open weights comes down to three things: compliance and control (sensitive data stays in-house), customization (domain fine-tuning, distillation, and alignment), and cost optimization (self-hosted inference may be more economical than pay-per-call at large scale).

2. Sparse MoE Architecture: Why 2.8T Parameters Activate Only 104B

2.1 Decoupling Capacity from Compute

In a dense model, every parameter participates in computation during each forward pass, so parameter count and per-token compute cost scale linearly. A sparse MoE (Mixture of Experts) architecture replaces the feed-forward network in the Transformer with multiple parallel “experts,” and a router dynamically selects a small subset of experts for each token.

For this model: 2.8T total parameters, only 104B activated per token — an activation ratio of roughly 3.7%. The design logic behind these numbers:

Dimension	Dense Model	Sparse MoE
Parameter scale	Constrained by compute	Can reach trillions
Per-token compute	Proportional to total parameters	Tied only to activated parameters
Memory requirements	Equals all parameters	Still must hold all parameters
Engineering complexity	Relatively simple	Routing balance, expert parallelism, communication optimization

In one sentence: MoE trades memory for compute — the knowledge ceiling the model can “remember” rises substantially, while the cost of each inference “calculation” stays in the hundreds-of-billions range.

2.2 The Cost: Memory and Engineering Complexity

Sparsity is not a free lunch:

1. All parameters must reside in memory or be efficiently loadable: Routing results are unknown before inference, so every expert weight must be available on demand.
2. Load balancing: Popular experts can become overloaded while cold experts go underutilized, requiring auxiliary losses, capacity factors, and other mechanisms to constrain routing.
3. Expert-parallel communication: In multi-machine deployments, tokens must be routed across GPUs (all-to-all communication), making the system highly sensitive to network bandwidth and topology.

2.3 Engineering Implications of a 1M-Token Context

A context window of 1 million tokens is significant for agent scenarios: system prompts, tool call results, codebase snippets, and multi-turn conversation history can all fit into context, reducing truncation and repeated retrieval.

But the memory cost of long context is substantial. The general estimation formula for KV Cache is:

KV Cache ≈ 2 × layers × KV heads × head dimension × sequence length × batch size × bytes per element

Here’s an illustrative calculation with a typical configuration (assuming 80 layers, 8 KV heads, 128 dimensions per head, FP16 storage):

def kv_cache_gb(layers, kv_heads, head_dim, seq_len, batch, dtype_bytes=2):
    """Estimate KV Cache memory usage (GB)"""
    bytes_total = 2 * layers * kv_heads * head_dim * seq_len * batch * dtype_bytes
    return bytes_total / 1024**3
print(kv_cache_gb(80, 8, 128, 1_000_000, 1))  # ~305 GB

A single sequence alone takes roughly 300GB — which explains why production-grade long-context services commonly rely on GQA-style attention variants, KV quantization, paged memory management, and similar general optimizations. When planning long-context resources, KV Cache often consumes more memory than the weights themselves.

3. Capability Gains Come from Post-Training: The Leverage of Reinforcement Learning

Notably, this model’s underlying architecture carries over from the previous generation — total parameters, activated parameters, and context window are all structurally unchanged. The capability improvement comes primarily from increased reinforcement learning investment during post-training. This reinforces an important industry consensus: pre-training determines the “foundation” of the capability ceiling, while post-training determines how high you can build on that foundation.

Several general technical directions in post-training reinforcement learning:

• RLHF (Reinforcement Learning from Human Feedback): Train a reward model to fit human preferences, then optimize the policy for alignment.
• RLVR (Reinforcement Learning from Verifiable Rewards): For tasks with objective criteria like math and code, use unit tests and formal verification as reward signals to reduce reward noise.
• Agentic reinforcement learning: Use the final success or failure of multi-turn tool calls and long-horizon tasks as the reward, directly optimizing “task completion” capability rather than single-turn response quality.
• Training stability engineering: KL divergence constraints to prevent policy drift, reward hacking prevention, and sample efficiency improvements — these details often determine the actual returns from reinforcement learning.

The takeaway for users: when evaluating a new model version, don’t just look at whether architecture parameters changed — look at post-training investment and real-world task performance changes.

4. MXFP4 Quantization: Compressing 5.6TB into 1.56TB

4.1 Why Weights Must Be Quantized

Storing 2.8 trillion parameters at 16-bit precision: 2.8e12 × 2 bytes ≈ 5.6TB, far exceeding what any single machine can hold. This model’s weights use MXFP4 low-precision quantization, compressed to 1.56TB — a compression ratio of roughly 3.6×, averaging about 4.5 bits per parameter (including group scaling factor overhead).

4.2 How MXFP4 Works: Block-Level Scaling + 4-Bit Floating Point

MXFP4 belongs to the Microscaling format family. The core idea: every N consecutive weights share a high-precision scaling factor, and elements within the block are represented in 4-bit floating point. Compared to naive low-bit quantization, “fine-grained grouping + block-level scaling” significantly mitigates errors from outliers, achieving a balance between size and quality at the 4-bit level.

Storage Scheme	Bits per Parameter	Total Footprint for 2.8T Parameters
FP16	16 bit	~5.6 TB
MXFP4 (with scaling factors)	~4.5 bit	~1.56 TB

def weight_tb(total_params: float, bits_per_param: float) -> float:
    """Estimate weight storage footprint (TB) by bit width"""
    return total_params * bits_per_param / 8 / 1e12
print(weight_tb(2.8e12, 16))    # 5.6 TB (FP16)
print(weight_tb(2.8e12, 4.46))  # ~1.56 TB (MXFP4, including scaling factors)

4.3 Deployment Form Factor Analysis

Even compressed to 1.56TB, deployment remains challenging:

• Multi-machine parallelism: A single 8-GPU node (~640GB memory) is far from sufficient; real deployments typically combine multi-machine tensor parallelism with expert parallelism.
• Tiered loading: Use CPU memory and NVMe for hierarchical weight caching, paired with on-demand loading capabilities in the inference framework.
• Resource budgeting: Weights are read-only; KV Cache is dynamic. Weights determine the “minimum configuration,” while concurrency and context length determine the “recommended configuration.”

One more point: quantization bit width primarily optimizes “storage and bandwidth.” Activations, attention computation, and other stages may still retain higher precision — mixed-precision design is the norm for very-large-scale model inference.

5. A New Evaluation Paradigm: What 17,500 Real Agent Sessions Reveal

5.1 Limitations of Static Benchmarks

Traditional static evaluation sets face three persistent problems: data contamination (test questions may have entered training data), score saturation (gaps between top models are smaller than noise), and scenario mismatch (single-turn Q&A scores don’t represent multi-step agent performance).

5.2 Dynamic Evaluation Based on Real Sessions

On the same day the model weights were released, a third-party evaluation leaderboard published ranking data based on over 17,500 real user long-horizon agent sessions: this model ranked first among open-weight models and 13th among all evaluated models; its net improvement rate was +5.1%, compared to +0.9% for the previous generation.

What’s more noteworthy here is the evaluation methodology itself:

• Real traffic: Tasks come from real user sessions spanning work, coding, chat, and other scenarios, closely matching actual demand distribution.
• Long-horizon agent tasks: Assesses multi-step planning, tool use, error recovery, and other composite capabilities.
• Net improvement rate: A corrected metric that accounts for both improvements and regressions — looking only at win rate can mask capability regressions, while net improvement better reflects overall experience changes.

The jump from +0.9% to +5.1% aligns with the timeline of increased post-training reinforcement learning investment, suggesting that training investment and evaluation conclusions form a mutually verifiable feedback loop.

5.3 Recommendations for Building Enterprise Evaluation Systems

1. Ground task sets in business context: Extract high-frequency tasks from real business logs to build private evaluation sets.
2. Weight agent tasks higher: Multi-step tool calls and long-document processing should be separate sets with higher weight than single-turn Q&A.
3. Regression baseline mechanism: Run full regression on version upgrades, using net improvement to judge “whether an upgrade is worthwhile.”
4. Online canary validation: Small-traffic A/B comparison combined with human review, forming an offline + online dual feedback loop.

6. Engineering Essentials for Enterprise Deployment

1. Weight distribution and storage: For terabyte-scale weights, use object storage with sharded checksums to ensure transfer integrity and version traceability.
2. Compute planning: Calculate three budgets — weight memory (starting at 1.56TB), KV Cache (grows with concurrency and context), and framework overhead — before sizing the cluster.
3. Inference serving: Use container platforms for orchestration and elastic scaling, paired with gateways for rate limiting, circuit breaking, and multi-model routing.
4. Observability: Establish monitoring, alerting, and logging services to track metrics like time-to-first-token, throughput, memory utilization, and routing balance.
5. License review: Review commercial use, distribution, and derivative model clauses line by line. If in-house capabilities are insufficient, general cloud services (such as GPU compute, container platforms, object storage, and monitoring services from Baidu AI Cloud) can help quickly set up deployment and evaluation environments.

7. Conclusion

Three technology trends for very-large open-weight models emerge from this model card: architecturally, sparse MoE pushes parameter scale to the trillions by “trading memory for compute”; engineering-wise, microscaling quantization makes terabyte-scale weights practically distributable; methodologically, post-training reinforcement learning becomes the primary lever for capability leaps, while real-session evaluation provides a verifiable feedback loop.

For developers, the significance of the open-weight ecosystem lies in a role shift: from “caller” to “owner.” The real barrier is no longer “can you get the weights” — it’s the engineering competition across three capabilities: memory planning, inference optimization, and evaluation systems.