Open-Weight 2.8T Sparse MoE Model: Architecture, Quantization, and Deployment Guide

1. Background: What Open-Weight Access Means for a Model of This Scale

Recently, a prominent AI research organization released the full weights of its flagship MoE (Mixture of Experts) model for public download, following through on an earlier commitment to make the weights available. For the developer community, this is a meaningful shift: a model that was previously accessible only through an API can now be downloaded, deployed, fine-tuned, and audited directly.

Based on publicly available information, this is a sparse MoE model with approximately 2.8 trillion total parameters and roughly 104 billion activated parameters per token. It supports a 1-million-token context window and multimodal image-text inputs. For teams looking to build their own inference services, perform domain-specific fine-tuning, or conduct model safety audits, open weights are a necessary precondition.

This article provides a systematic technical breakdown across five areas: model architecture, quantization strategy, deployment requirements, inference framework compatibility, and API usage. The goal is to help developers assess whether the model is usable, how to use it, and what it will cost.

2. Model Architecture: Sparse Attention and Expert Routing

2.1 Layer Count and Attention Types

According to the official model description, the model has a total of 93 layers:

• 69 layers use KDA (a linear attention variant)
• 24 layers use Gated DSA (gated sparse attention)

Compared with the previous generation, which replaced all full attention layers with gated sparse attention, the key change here is the use of sparse attention in place of full attention to reduce compute and memory overhead in long-context scenarios. This aligns with the current mainstream optimization direction for ultra-long-context models: full attention scales quadratically with sequence length, while sparse attention compresses the effective compute to a manageable level.

2.2 Expert Layer Design

The expert layers contain 896 experts in total, with 16 activated per token, plus 2 shared experts. This combination of fine-grained experts and shared experts is a common pattern in recent MoE architectures:

• Fine-grained experts: smaller individual capacity and a larger total count, enabling more flexible routing and improved utilization of model capacity.
• Shared experts: always participate in computation, carrying general knowledge and reducing redundancy across experts.

From an engineering perspective, the 16-of-896 sparsity means that the actual activated parameters per token represent roughly 3.7% of the total. This is the key reason a 2.8T-parameter model can run on reasonable hardware at all.

2.3 Vision Encoder

The vision component is a roughly 401-million-parameter vision Transformer (MoonViT-V2) responsible for encoding image inputs. Its relatively small size compared with the language backbone suggests that the multimodal capability is primarily achieved through lightweight alignment rather than a large-scale vision backbone trained from scratch.

3. Quantization Strategy: MXFP4 Weights + MXFP8 Activations

3.1 Why Quantization Is Mandatory

If the 2.8T parameters were stored in BF16, the weights alone would require approximately 5.6 TB of memory, far beyond what a single machine can reasonably accommodate. The model therefore uses a mixed quantization scheme with MXFP4 for weight storage and MXFP8 for activations, and the organization states that it employed quantization-aware training (QAT).

MXFP4 and MXFP8 belong to the microscaling floating-point family, which shares exponents at the block level to balance dynamic range and storage efficiency. The officially reported figures are:

Metric	Value
Repository size	~1.56 TB
safetensors parameter count	~2.78 trillion
Weight storage format	MXFP4
Activation format	MXFP8

The reduction from 5.6 TB to 1.56 TB represents a compression ratio of approximately 3.6x, which is what makes the weights downloadable and deployable in the first place.

3.2 The Value of Quantization-Aware Training

It is important to emphasize that QAT is fundamentally different from post-training quantization (PTQ). PTQ converts an already-trained model to lower precision and typically incurs noticeable accuracy loss. QAT, by contrast, simulates quantization error during training so that the model weights adapt to the low-bit representation. For routing-sensitive architectures like MoE, QAT can significantly reduce quality degradation caused by shifts in expert selection.

For deployment teams, the practical implication is that the low-bit weights can be loaded directly without performing another round of quantization calibration. However, the inference framework must support the corresponding MXFP4/MXFP8 operators; otherwise, the weights will be dequantized before computation and the memory advantage will be lost.

4. Local Deployment: Hardware Requirements and Cost Estimates

4.1 Memory Requirements

The 1.56 TB of weights is only the static footprint. Actual inference also requires:

• KV cache: with a 1-million-token context, the KV cache can reach hundreds of gigabytes depending on the attention implementation and batch size.
• Activations and intermediate tensors: MoE routing and expert-parallel communication introduce additional overhead.
• Framework reservation: inference engines typically need extra memory for graph capture, workspaces, and other runtime structures.

As a result, a practical configuration should have more than 2 TB of total memory to be safe.

4.2 A Reference Configuration

Third-party developers have estimated that one viable configuration is a server with 8 high-memory accelerators totaling approximately 2.3 TB of memory, with hardware costs in the range of $550,000 to $650,000. Several caveats apply:

• This is a third-party estimate, not an official hardware recommendation.
• The cost does not include data center space, power, networking, or operations.
• Actual prices vary significantly with market conditions.

For most teams, this means that self-hosting is a capital-intensive decision that should be evaluated against expected business volume.

4.3 Pre-Deployment Checklist

[ ] Total accelerator memory >= 2 TB (including KV cache headroom)
[ ] Inference framework supports MXFP4 weight loading
[ ] Framework supports MXFP8 activation computation
[ ] Inter-accelerator bandwidth meets expert-parallel communication requirements
[ ] Operational capability for multi-node distributed inference
[ ] Power and cooling budget evaluated

5. Inference Framework Support: Three Paths

Public information indicates that mainstream inference frameworks have published deployment recipes, falling into three categories:

1. High-throughput serving frameworks: suitable for online API services, supporting continuous batching and PagedAttention-style memory management; the preferred choice for production environments.
2. Lightweight high-performance frameworks: focused on low latency and easy deployment, suitable for small-to-medium-scale or edge scenarios.
3. Specialized acceleration frameworks: deeply optimized for specific hardware and quantization formats, typically achieving better throughput/latency ratios.

5.1 Selection Guidance

Scenario	Recommended Direction	Key Considerations
High-concurrency online APIs	High-throughput serving framework	Batching efficiency, KV cache management
Low-latency interaction	Lightweight framework	Time-to-first-token, per-request overhead
Cost-sensitive batch processing	Specialized acceleration framework	Per-token cost, quantization operator coverage
Research / fine-tuning	Native training framework	Weight loadability, distributed strategies

5.2 A Typical Startup Approach

# Pseudocode: load quantized weights and enable tensor parallelism
# Refer to the official framework recipe for actual parameters
serve --model <local_weight_path> \
      --tensor-parallel-size 8 \
      --quantization mxfp4 \
      --max-model-len 1000000 \
      --enable-expert-parallel

The key points are: tensor parallelism should match the number of available devices, expert parallelism requires additional communication configuration, and long-context inference requires explicitly setting max-model-len; otherwise, the default value may be far below 1 million.

6. API Usage: An Alternative When Self-Hosting Is Not an Option

For teams that do not want to bear the hardware cost, API access is available. Public information indicates:

• Both a standard model name and a high-speed variant exist, with the high-speed version approximately 2 to 3 times faster than the standard one.
• The interface is compatible with mainstream conversational API conventions.
• reasoning_effort accepts low, high, and max, with max as the default.
• For multi-turn tool calls, the reasoning_content and tool_calls from the previous assistant turn must be passed back in the next request.

6.1 Correct Multi-Turn Tool Call Handling

This is the most common pitfall. If the reasoning content from a previous turn is not passed back, the model loses context in the next turn, breaking the tool-calling flow. Example:

messages = [
    {"role": "user", "content": "Check tomorrow's weather"},
    {"role": "assistant",
     "content": None,
     "reasoning_content": "Need to call the weather tool...",
     "tool_calls": [{"id": "call_1", "function": {...}}]},
    {"role": "tool", "tool_call_id": "call_1", "content": "Sunny, 25C"},
]
# The next request must include the reasoning_content and tool_calls
# from the assistant message above

6.2 Choosing reasoning_effort

• low: suitable for simple Q&A and classification; lowest latency.
• high: balances quality and cost.
• max: the default, suitable for complex reasoning, code generation, and agent tasks.

For production use, it is recommended to set the value dynamically based on request type rather than running every request at max and incurring unnecessary cost.

7. Capability Positioning: Where It Stands Among Open-Weight Models

Public leaderboards show that the model ranks 1st among open-weight models and 13th overall in agent-related evaluations. This positioning indicates two things:

1. Within the open-weight subset, it is in the top tier.
2. There remains a gap compared with the strongest closed-source models, with several models ranked ahead of it overall.

The practical takeaway for developers is: if data compliance requirements mandate private deployment, this is one of the strongest currently available options; if maximum capability is the priority and API usage is acceptable, a broader comparison is warranted.

8. Summary and Recommendations

The core takeaways from this open-weight release can be summarized as follows:

• Scale: 2.8T total parameters, 104B activated per token, 1-million-token context, multimodal input.
• Architecture: 93 layers, 69 KDA + 24 gated sparse attention layers, 16-of-896 expert routing.
• Quantization: MXFP4 weights + MXFP8 activations, trained with QAT, ~1.56 TB repository.
• Deployment: requires 2 TB-class memory; third-party hardware estimates range from $550K to $650K.
• Ecosystem: mainstream inference frameworks already provide deployment recipes.
• API: compatible with mainstream interfaces; multi-turn tool calls require passing back reasoning content.

Recommendations by team type:

• Large enterprises / cloud providers: evaluate private deployment, focusing on quantization operator coverage and long-context throughput validation.
• Mid-sized teams: prefer API access with limited fine-tuning over heavy capital investment.
• Research institutions: open weights present a good opportunity for auditing and secondary development; a reasonable starting point is analyzing attention sparsity and routing stability.
• Individual developers: start with the API, pay attention to reasoning_effort and tool-call conventions, and get agent workflows running before optimizing for cost.

Open weights for very large models fundamentally turn capability into an asset that can be audited, customized, and controlled independently. Whether the hardware and operational costs are justified depends on your data compliance requirements, business scale, and engineering capacity. There is no universal answer — but with weights available, the question is at least one you can now act on.