2.8-Trillion-Parameter Open-Weights Model: Architecture Analysis and Local Deployment Guide

    A Chinese AI lab has released the full weights of its flagship model—2.8 trillion total parameters, approximately 1.56 TB of MXFP4-quantized weights, and 1 million token context support—making it the largest open-weights model to date. This article breaks down the Mixture-of-Experts architecture, sparse attention design, and quantization-aware training approach, then provides a realistic assessment of hardware requirements, parallelization strategies, licensing constraints, and cost considerations for teams evaluating local deployment versus API usage.

    Background: Open Weights Enter the Trillion-Parameter Era

    In early November, a major Chinese AI lab fulfilled its commitment by publishing the complete weights of its flagship model on mainstream model hosting platforms, available for direct download without application or approval. The model contains 2.8 trillion total parameters, with weight files totaling approximately 1.56 TB—by far the largest open-weights model ever released, exceeding all previous open models by a substantial margin.

    The model first went live via API and web interfaces in late October, with the team promising full weights within two weeks. The release was delivered on schedule. The weight repository includes 112 files, comprising 96 sharded safetensors format files. Mirror distribution through Chinese model-hosting communities helped alleviate bandwidth pressure for cross-border downloads of such large files.

    The significance extends beyond “another open-source model release.” This release narrows the capability gap between open weights and closed-source APIs: a system claiming to match or exceed top-tier closed models on multiple benchmarks is now downloadable and self-hostable. However, two practical realities remain: the license terms are not fully permissive, and the hardware requirements for local deployment are prohibitive for most teams. This article examines the architecture, quantization, deployment requirements, and licensing considerations.

    Model Architecture: How 2.8 Trillion Parameters Are Organized

    Mixture-of-Experts (MoE) with Sparse Activation

    The model adopts a typical MoE architecture: 2.8T total parameters, yet each forward pass activates only approximately 104 billion parameters per token—an activation ratio below 4%. Specifically, the configuration uses 896 routed experts, with a gating network selecting 16 per token, plus 2 shared experts handling general knowledge. This design gives the model trillion-parameter knowledge capacity while keeping per-token inference computation at roughly the level of a hundred-billion-parameter model.

    There is a critical engineering implication developers must understand: lower activated parameters do not translate to lower VRAM usage. During inference, all 2.8T parameters must reside in VRAM or system memory because routing decisions occur dynamically at every token and every layer—there is no way to predict which 16 experts the next token will use. Sparsity saves compute (FLOPs), not storage.

    Long Context and Sparse Attention

    The model has 93 layers and supports up to 1 million tokens of context. Compared to the previous generation, the most significant structural change is replacing full attention with sparse attention in 24 of its layers. Full attention scales quadratically in compute with sequence length; at million-token context, the KV cache and compute overhead of the attention mechanism alone would overwhelm conventional hardware. Sparse attention allows queries to focus only on selected key positions, substantially reducing computational load in long-sequence scenarios. This is the core enabler for million-token context in production.

    Native Multimodality

    The model is natively multimodal, integrating a self-developed vision encoder that processes image inputs directly. “Native” means vision capabilities were jointly trained with language abilities from the pretraining stage, rather than bolting on a visual module after the fact on a text-only model. This approach generally yields better cross-modal alignment but also means deployment must include the vision encoder in the inference pipeline, adding complexity.

    Quantization Scheme: MXFP4 Weights + MXFP8 Activations

    Weights use MXFP4 format (4-bit floating point with shared scale factors, i.e., micro-scaling quantization), activations use MXFP8, and quantization was performed during training via Quantization-Aware Training (QAT)—not post-training quantization (PTQ). This distinction matters significantly: QAT allows the model to adapt to low-precision representations during training, resulting in far less accuracy degradation than post-hoc compression. This is why the 2.8T model can be compressed to a 1.56 TB download while largely maintaining quality.

    Note that the official release includes only this quantized version—no BF16 or FP8 precision weights are available. At BF16, 2.8T parameters would occupy approximately 5.6 TB; the quantized version reduces this to 1.56 TB, a compression ratio of roughly 3.6×, consistent with the theoretical expectation for 4-bit quantization.

    Performance: How to Interpret the Benchmark Data

    Per the vendor’s self-reported figures, the model excels in programming and agent-based tasks: 65.7% on a terminal operation benchmark (vs. 66.4% for a top-tier closed model, and well above another closed flagship at 58.2%); 51.7% on a software engineering long-task benchmark, surpassing the closed model’s 46.3%. However, it does not lead across the board: on scientific terminal tasks and hard reasoning benchmarks, scores are notably lower than the comparison targets (38.1% vs. 63.3%; 39.4% vs. 60.9%). The vendor reports leading in 16 of 37 benchmarks—roughly split. A third-party intelligence index ranks it fourth globally.

    A measured approach to reading these numbers:

    1. Distinguish task types. Coding, tool calling, and multi-step agent execution are this model’s strengths; scientific reasoning and hard mathematical reasoning are comparatively weaker. Model selection should map to your core use case, not overall scores.
    2. Treat self-reported results with caution. All comparison data originates from the vendor’s own testing; test conditions, prompts, and sampling parameters may not match third-party reproduction. Independent evaluations and validation on your own workloads are essential.
    3. Leverage reproducibility. The unique value of open weights is that anyone can reproduce and verify results on their own test sets—something closed APIs cannot offer.

    Local Deployment: Why 8 GPUs Is Only the Starting Point

    VRAM Requirements

    1.56 TB of weights means you need at least that much memory just to load them. With flagship data-center GPUs featuring 80 GB VRAM per card, 8 cards provide 640 GB—far short. Even with newer accelerators offering 141 GB per card, 8 cards total approximately 1.13 TB, still insufficient. Realistic deployments typically need:

    • 8 accelerators with 141 GB or more for weights (with tensor parallelism sharding), plus additional space for KV cache—often 12 to 16 cards total;
    • Or a hybrid “GPU + CPU memory offload” scheme: place some expert layers in large-capacity system memory, swap over high-speed interconnects, trading latency for feasibility.

    Additionally, KV cache must be provisioned: at million-token context, even with sparse attention, the KV cache can reach hundreds of gigabytes. Higher concurrency at long context amplifies VRAM pressure further.

    Inference Frameworks and Parallelization Strategies

    For MoE models of this scale, deployment typically requires a combination of parallelization techniques:

    • Tensor Parallelism (TP): Split individual layer weights across multiple GPUs to reduce per-card memory usage.
    • Expert Parallelism (EP): Distribute the 896 experts across devices by expert dimension; the routing layer handles dispatch and aggregation.
    • Pipeline Parallelism (PP): Slice the 93 layers into segments across nodes.

    An illustrative deployment plan:

    1. deployment_plan = {
    2. "model_params": "2.8T (MXFP4, ~1.56TB)",
    3. "gpus": 16, # 141GB-class accelerators
    4. "parallelism": {"tp": 2, "ep": 8, "pp": 1},
    5. "kv_cache_budget_gb": 200, # Adjust by concurrency and context length
    6. "max_context": 1_000_000,
    7. "fallback": "CPU offload for cold experts"
    8. }

    Mainstream open-source inference frameworks are actively adding support for this architecture, but expert-parallel tuning and quantized kernel performance for extremely large MoE models remain hard engineering problems. Before production deployment, run throughput and latency benchmarks on target hardware.

    Cost Considerations: Self-Hosting vs. API

    A rough estimate: a 16-GPU server with flagship accelerators costs on the order of several million RMB in capital expenditure. Even renting cloud GPU compute translates to substantial monthly costs. Self-hosting becomes economical only when inference volume is high enough, data compliance requires keeping data within your own network, or you need deep customization (continued training, domain fine-tuning). For most small and mid-sized teams, a more rational path is: use the API for day-to-day calls, complete critical validation and evaluation on short-term rented compute, and invest in self-hosting only after business volume justifies the cost. Cloud-based GPU bare-metal instances with object storage for weight distribution can convert upfront capital expenditure into flexible operational costs, lowering the barrier to experimentation.

    Licensing: Open Does Not Mean Unrestricted

    Very large open-weights models typically ship with conditions. Common clauses include: commercial entities above a monthly active user or revenue threshold must contact the vendor for separate authorization; attribution must be retained when building on the model’s outputs; and using the model’s outputs to train competing models is prohibited. Before using this model in any commercial product, read the license in full—ideally with legal review—and pay particular attention to the definition of “large-scale commercial use” and the terms governing derivative models.

    Summary and Recommendations

    A 2.8T-parameter, 1.56 TB, QAT-quantized, million-context open-weights model marks the point where the open-source ecosystem formally enters the same arena as closed-source flagships. But “downloadable” does not mean “deployable”: the hardware barrier of 8 to 16 flagship accelerators, the engineering complexity of MoE expert parallelism, and the attached license restrictions mean this model is, for now, practical primarily for large teams and research institutions with substantial compute resources. For individual developers, tracking the community’s subsequent more aggressive quantization versions and distilled smaller models is a more realistic path forward. The true value of open weights lies in auditability, reproducibility, and customizability—and this race has only just begun.