2.8T-Parameter Open-Weight MoE Model Released: Architecture, Quantization, Deployment, and Evaluation

In early November, a model team made good on the open-weights pledge made when the model launched, uploading the complete weights of a 2.8T-parameter Mixture-of-Experts (MoE) model to a public model hosting platform. Any team can now download, deploy, fine-tune, and audit them. This is more than a routine “model release”: it marks the arrival of open weights for ultra-large MoE models in the 2.8T-parameter class, complete with 1M-token context and image-plus-text input capabilities.

1. What “Open Weights” Actually Opens Up

For development teams, the essential difference between open weights and an API is the difference between renting a capability and owning an asset:

• Private deployment: Finance, healthcare, and government scenarios with hard data-sovereignty constraints can only run local inference if they hold the weights;
• Fine-tuning: With the weights in hand, you can apply LoRA, full-parameter fine-tuning, or continued training to turn a general-purpose model into a business-specific one;
• Auditing: Security teams can run backdoor detection, bias evaluation, and red-team testing against the weights — something a black-box API cannot offer;
• Supply chain control: No lock-in to a single inference provider, and no exposure to supply interruptions caused by service changes, rate limiting, or deprecation.

One caveat: “open weights” does not mean “open everything.” The training data, data mixture, and training code are not necessarily published with the weights, and the boundaries of commercial use are defined by the model card and license terms — review them clause by clause before integrating.

2. Architecture Breakdown: Hybrid Attention + Sparse MoE

2.1 Attention: from full attention to a “lightweight + sparse” hybrid

The model has 93 layers: 69 use gated linear attention based on the Delta rule (a lightweight linear attention variant), and 24 use gated sparse attention. According to the team, this generation replaces full attention layers with gated sparse attention compared with its predecessor, and the goal is straightforward: model long contexts more cheaply.

Understanding the trade-off means going back to complexity:

• Full attention scales as O(n²) with sequence length; at a million tokens, both memory and compute become unmanageable;
• Linear attention drops complexity to roughly O(n), but global retrieval is weak;
• Sparse attention “compresses the context and selectively interacts with key tokens,” preserving global retrieval at near-linear cost.

The hybrid design — cheap linear attention for pattern memory in most layers, a few sparse attention layers for global retrieval — is the optimal balance between cost and capability, and it is what makes million-token contexts genuinely feasible in engineering terms.

2.2 Expert layer: 896 experts, ~3.7% activation

The expert layer contains 896 experts; a router activates 16 per token, and 2 shared experts apply to every token. From this, the key ratio falls out: activation ≈ 104B / 2.8T ≈ 3.7%. Two direct benefits:

• Inference cost ≈ a 104B dense model: each token computes only its selected experts, so FLOPs are comparable to a dense model of the same active size;
• Knowledge capacity ≈ 2.8T: total parameters determine how much domain knowledge the model can “store” — the core dividend of a “small activation, large capacity” architecture.

Shared experts matter just as much: they carry the general knowledge that every token needs, so routed experts don’t have to duplicate it. This improves parameter utilization and makes routing training more stable.

2.3 Multimodal: a ~400M-parameter vision encoder

The vision side is a Vision Transformer encoder of roughly 400M parameters that converts images into tokens concatenated with the text sequence. Combined with million-token context, such a model is naturally suited to complex inputs that mix long documents, charts, and screenshots — financial report analysis, engineering drawing Q&A, UI automation, and similar scenarios.

3. Low-Bit Quantization: MXFP4 Weights, MXFP8 Activations, and QAT

Weights are stored in MXFP4 and activations use MXFP8, and the team says quantization-aware training (QAT) was applied. Three keywords, one at a time:

3.1 What MXFP4 is

The MX (Microscaling) format is a low-bit scheme defined by an open industry standard: vectors are split into fixed-size blocks (typically 32 elements per block), each block shares one high-precision scale factor, and elements within a block are stored as 4-bit floats. Compared with naive INT4/FP4, it is far more robust to outliers — a small number of extreme values in LLM weights are exactly what destroys naive quantization accuracy, and block-level scaling mitigates precisely that.

3.2 The storage math checks out

The repository occupies about 1.56TB, and the parameter count tallied from the weight files is about 2.78 trillion. A quick sanity check: 2.78T parameters × 0.5 bytes ≈ 1.39TB; add the components kept at higher precision (embedding layers, attention and shared modules) plus sharding metadata, and the 1.56TB figure is essentially self-consistent. In other words, 4-bit-dominant storage brings “downloading a 2.8T model” from roughly 5.6TB at FP16 down to 1.56TB, sharply cutting distribution and memory costs.

3.3 Why QAT is needed

Post-training quantization (PTQ) compresses finished weights directly and loses noticeable accuracy. QAT instead simulates quantization error in the forward pass during training, letting the weights “adapt in advance” to a low-bit environment, so that critical capabilities survive an aggressive MXFP4/MXFP8 configuration. For enterprise users the benefit is immediate: the weights you download are already in a low-bit-optimized form, and frameworks can load them in the corresponding format as-is.

4. Local Deployment: First, Do the Memory Math

4.1 The hardware threshold

A community third-party estimate offers one workable configuration: a single server with 8 of the latest-generation flagship data center GPUs (about 2.3TB of memory in total), with hardware alone costing roughly $550,000–$650,000. Note that this is a third-party estimate; the model team has not published official hardware recommendations. The basic memory formula:

Required VRAM ≈ weight size + KV cache + runtime overhead
Weight size ≈ total parameters × bytes per parameter (MXFP4 ≈ 0.5 bytes)

Two easily underestimated points:

• All MoE experts must be memory-resident: routing happens dynamically per token, so you cannot load only “the frequently used experts.” The only way to save memory is to reduce bytes per parameter;
• KV cache for long contexts: a million-token KV cache is an astronomical figure for a full-attention model; this model’s hybrid attention design is exactly what compresses that overhead to an engineering-feasible level, and it is the precondition for million-token context to work at all.

If a single node’s memory is insufficient, you need tensor parallelism and expert parallelism across machines plus high-speed RDMA interconnect — and both network cost and scheduling complexity go up.

4.2 Deployment path

The good news: mainstream open-source inference frameworks published deployment recipes on the day the weights were released, covering the model’s quantization format and sparse attention operators. A practical checklist:

1. Weight distribution: for a 1.5TB-scale download, use object storage with resumable transfers and verify file hashes;
2. Version pinning: lock the inference framework and acceleration library versions per the official recipe to avoid operator incompatibilities;
3. Load testing: stress-test throughput and time-to-first-token with realistic context lengths and concurrency;
4. Observability: connect logging and monitoring/alerting, and watch routing balance and memory headroom.

Teams that prefer not to build their own cluster can instead host the weights on a cloud GPU container platform for private deployment. Providers such as Baidu AI Cloud offer container platforms, object storage, logging, and monitoring services that cover the full pipeline from weight distribution and deployment orchestration to operations monitoring — a practical option for teams with private deployment requirements but no dedicated hardware operations capacity.

5. API Integration: Reasoning Effort and Multi-Turn Tool Calling

5.1 Basic integration and reasoning effort

The official API is aligned with the weights’ capabilities and compatible with the industry-standard conversational API protocol (message array + tool JSON Schema + SSE streaming), so common SDKs can plug in directly. An accelerated endpoint is also available — officially stated to be 2–3× faster than the standard one — for latency-sensitive, high-frequency scenarios. The model exposes a reasoning_effort parameter with three options — low, high, and max — and max is the default. The trade-off logic:

• low: minimal thinking, lowest latency — suited to simple tasks like classification and extraction;
• high: a balance of quality and latency — suited to most everyday tasks;
• max: maximized deep thinking — suited to complex, long-horizon Agent tasks, at the cost of response time and token consumption.

Set the level dynamically by task type rather than globally fixing max — in batch workloads, this single setting has a substantial impact on cost.

5.2 Multi-turn tool calling: always pass back reasoning_content and tool_calls

This is the easiest pitfall to hit during integration: in multi-turn tool calling, you must pass back the previous assistant message’s reasoning_content (chain of thought) and tool_calls (call intent) along with the tool execution results. Example code:

messages = [{"role": "user", "content": "List the top 5 directories by disk usage on this machine"}]
while True:
    resp = chat(model=MODEL, messages=messages,
                tools=TOOLS, reasoning_effort="max")
    msg = resp.message
    # Key: keep the full chain of thought and call intent, append them to the context
    messages.append({
        "role": "assistant",
        "content": msg.content,
        "reasoning_content": msg.reasoning_content,
        "tool_calls": msg.tool_calls,
    })
    if not msg.tool_calls:       # No more calls — task complete
        break
    for call in msg.tool_calls:  # Run the tools locally and return the results
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(run_tool(call), ensure_ascii=False),
        })

Why pass back the chain of thought? An Agent’s typical loop is “think → call a tool → observe the result → think again.” If you return only the tool results and discard the reasoning, the model has to “rethink from scratch” in the next round, which invites repeated calls, direction drift, and even contradictions with the previous round’s conclusions. Returning reasoning_content preserves the model’s working memory — the engineering foundation for long-horizon task reliability.

6. Real-Session Evaluation: How Should Agent Capability Be Measured

6.1 The limits of static benchmarks

Traditional benchmarks have two chronic problems: questions may leak into pretraining data and inflate scores; and multiple-choice testing is disconnected from the real shape of Agent work — dozens of consecutive steps and multi-turn tool calls.

6.2 A “net improvement” method based on real sessions

An evaluation leaderboard built on real user Agent sessions takes a different approach: it extracts four signal types from more than 17,500 real long-horizon sessions — whether the user explicitly confirmed task success; user praise and complaints about the model; how the model responded after user corrections; and whether shell commands actually executed successfully. Positive and negative signals are normalized into a “net improvement” score, with separate leaderboards for categories such as Work, Code, and Chat, directly measuring task completion from the user’s perspective.

6.3 A signal worth reading

Comparison data from the same leaderboard shows: this generation’s open-weight model (at its highest reasoning tier) posted a net improvement of +5.1% — first among open-weight models and 13th overall — while the previous generation managed only +0.91%, ranking sixth. By sub-category, Bash reliability improved from −4.5% in the previous generation to +7.9%, the largest gain of any dimension. The right reading is not “yet another model topping a chart” but an industry signal: the real-world Agent capability of open-weight models is closing the gap with the closed-source first tier at a generational pace, and the difference is visibly narrowing on real tasks. Going a step further, enterprises can reuse the same signal system to build internal evaluations on their own traffic — which fits their scenarios far better than any external leaderboard.

7. Practical Advice: Self-Host or Call the API

Prefer self-hosting	Prefer the API
Data must never leave your domain	Rapid prototyping
Deep fine-tuning or continued training required	Highly variable traffic peaks and valleys
Inference volume large enough to amortize hardware costs	No dedicated inference engineering team
Security audit and compliance requirements	Strict cost ceilings

The two paths are not mutually exclusive. A common evolution: validate the product shape and prompts with the API first, then migrate high-frequency scenarios to self-hosted or cloud-hosted private deployment once the business stabilizes, using the same weights to keep results consistent. The real significance of open weights is that they hand the choice back from providers to teams.