2.8T Open-Weight MoE Model: Local Deployment and License Essentials

    A technical overview of a 2.8T-parameter open-weight MoE model covering its architecture, VRAM estimation for local deployment, multi-GPU serving strategies, how to interpret benchmark results, and the key commercial restrictions in its custom license.

    Model Overview

    The model has 2.8T total parameters with downloadable weights totaling 1.56TB, making it one of the largest open-weight models with publicly released weights. At inference time, only about 104B parameters are activated per token. It is positioned as a natively multimodal Agent model with a 1M-token context window. The weights are split into 96 safetensors shards across 112 files.

    The official release includes only one version: a quantization-aware training (QAT) product using MXFP4 weight precision and MXFP8 activation precision. No BF16 or FP8 high-precision versions are provided, meaning what you load locally is already quantized weights. Because QAT simulates quantization error during training, the accuracy loss after compression is smaller than post-training quantization—but it is still not equivalent to full precision.

    Architecture Highlights

    1. Expert network: The model has 896 experts and a depth of 93 layers. For each token, the routing mechanism selects 16 experts, plus 2 always-on shared experts. Total capacity is enormous, while per-step compute depends only on the activated parameters.

    2. Long context: According to the publisher, the original 24-layer full attention mechanism was replaced with sparse attention (DSA) to reduce the compute cost of long-context inference. Sparse attention saves compute on very long inputs, but its impact on accuracy should be validated against your own long-text tasks. The publisher’s documentation does not include a full accuracy comparison.

    3. Vision: A built-in vision encoder means image understanding requires no separately attached vision model, simplifying integration for multimodal applications.

    VRAM Estimation

    Start with the weights. The full 1.56TB of weights must fit into VRAM, system memory, or disk. The table below shows the minimum number of GPUs needed for weights alone, plus an empirical recommended count that accounts for KV cache and activation overhead.

    GPU VRAM Minimum GPUs (weights only) Recommended GPUs (incl. KV cache & activations)
    80GB 20 24+
    141GB 12 ~16
    192GB 9 ~12

    Eight 80GB GPUs total 640GB—far below 1.56TB—so the full weights cannot be loaded directly. An “8-GPU setup” only works if part of the weights is offloaded to CPU memory or NVMe storage, with a significant slowdown. Throughput drops substantially in this mode; it is only suitable for verifying that the model runs.

    KV cache size depends on the number of concurrent requests, context length, and layer count. It grows quickly in long-context, high-concurrency scenarios, so reserve VRAM for it during deployment planning.

    Deployment

    Multi-GPU deployment typically combines tensor parallelism (TP), expert parallelism (EP), and pipeline parallelism (PP). Inference frameworks such as vLLM and SGLang provide corresponding options, but parameter names and model support vary by version. Always check the documentation for your framework version and validate in a small-scale environment first.

    Recommended steps:

    1. Confirm the framework version supports this model’s architecture and quantization format.
    2. Calculate GPU count from the VRAM table and reserve headroom for the KV cache.
    3. Choose a parallelism combination. For multi-node deployment, check inter-node network bandwidth—expert parallelism is highly sensitive to communication latency.
    4. Test first-token latency and throughput with fixed inputs; only consider production rollout after results meet your targets.

    Three common mistakes: estimating VRAM from parameter counts alone while ignoring the actual footprint of the quantization format; loading weights directly from a network file system, which makes startup extremely slow; and failing to reserve VRAM for the KV cache, causing out-of-memory errors as soon as concurrency ramps up.

    How to Read the Performance Data

    The following benchmark results are all vendor-reported. The model performs strongly in tests favoring code execution and engineering tasks, such as Terminal-Bench 4.0 and SWE Marathon, with some metrics above certain closed-source models. In tests emphasizing deep reasoning and scientific knowledge, such as Terminal-Bench-Science and HLE-Diamond, it still trails the industry’s top tier. Across 37 benchmarks, results are mixed.

    Include it on your candidate list, with a focus on scenarios like tool calling, long-code analysis, and automated workflows—but validate with your own business data. For math- and derivation-heavy use cases, run rigorous evaluations before committing. Benchmark versions, prompts, and toolchains all affect results, so confirm you’re comparing apples to apples.

    License Compliance Check

    The model uses a custom license. Overall, the grant is close to MIT—free use, copying, modification, distribution, fine-tuning, and commercial use are permitted—but with two commercial restrictions:

    1. Revenue threshold: Companies offering the model as a service (e.g., via API) with cumulative revenue exceeding $20 million over 12 months must sign a separate agreement with the publisher before commercial use.
    2. Traffic threshold: Commercial products with more than 1 million monthly active users (MAU) also require a separate agreement.

    This makes the model open-weight rather than fully unrestricted open source. Most startups and small-to-mid-size developers will not trigger these conditions, but large internet platforms and cloud providers need separate evaluation. The checklist below supports an initial pre-commercial review; final conclusions should follow the license text and be confirmed by legal counsel.

    Check item What to verify Result
    Product form Do you offer the model as an API or hosted service?
    Revenue Is 12-month cumulative revenue near or above $20 million?
    MAU Is your commercial product’s MAU near or above 1 million?
    Distribution Do redistributed weights or derivative models retain the license text and copyright notice?
    Fine-tuned models Are fine-tuned versions handled per the license requirements?
    Records Is your threshold analysis and legal conclusion documented?

    Download and Storage

    Downloading 1.56TB of weights requires a stable network environment. At ideal bandwidth, 1Gbps takes roughly 3.5 hours; 100Mbps takes about 35 hours. Disk read speed can also become a bottleneck at load time, so use local NVMe drives or a parallel file system, and verify the integrity of every shard after download completes.

    When choosing cloud resources, compare options based on VRAM needs, network bandwidth, and storage capacity—not just per-GPU pricing.

    Conclusion

    This 2.8T open-weight MoE model puts ultra-large-scale weights in the hands of more developers, but its deployment requirements and license boundaries are more specific than the word “open” suggests. Calculate your VRAM, review the license, and benchmark against your own tasks before deciding whether it fits your project.