Local Deployment of a 2.8T-Parameter Open-Weight MoE Model: Architecture, Memory, Precision, and Licensing
What it actually takes to run a 2.8-trillion-parameter mixture-of-experts model that is now publicly downloadable. This article breaks down the architecture behind the ~104B active parameters per token, the 896-expert routing scheme, sparse attention for million-token contexts, the single MXFP4/MXFP8 quantization-aware-trained release, the memory math that makes eight GPUs a starting point rather than a comfortable setup, the custom license and its revenue and MAU thresholds, and how to read vendor benchmarks. It closes with a practical checklist for deciding whether the model can run, runs well, can be used commercially, and is worth using at all.
1. Why trillion-scale open weights matter
A model development team has released the full weights of a very large MoE (Mixture of Experts) model with roughly 2.8 trillion total parameters. The weight files come to about 1.56 TB, which puts it among the strongest openly downloadable models currently available.
The reason this “weights-level” release deserves attention is not that it tops any leaderboard. It is that a parameter scale previously confined inside a handful of very large vendors has, for the first time, landed in a form that individual developers and small teams can download directly.
One common misconception needs clearing up first: being able to download the weights is not the same as being able to run them comfortably on your own hardware. Once you cross into trillion-parameter territory, hardware requirements, memory budget, quantization format, and license terms all become real constraints.
This article only covers the engineering questions that come after the weights land: what is inside the architecture, how the license should be read, whether normal teams can actually run it, and which budgets need to be calculated before the first inference request.
2. Architecture: why 2.8T parameters activate only 104B
The model uses a sparse MoE architecture, whose defining trait is “very large total parameter count, very few parameters per token.” The details:
- ~2.8T total parameters, ~104B active per token. The model stores 2.8 trillion parameters, but each token only routes through about 104 billion of them during the forward pass. This decoupling of capacity from compute is the whole point of MoE.
- 896 experts, 16 selected per token, plus 2 shared experts. Think of the experts as a pool of sub-networks with different specialties. A router picks the 16 most relevant experts for each token, while the 2 shared experts are always active and carry general-purpose capability. Compared with dense models, where every token activates every expert, sparse activation cuts inference compute substantially.
- 93 layers, with a context window of 1 million tokens. A context that long puts enormous pressure on both memory and attention computation, which sets up the next section.
- Native multimodality with a built-in vision encoder. The model handles text and images together by design, rather than through a bolt-on module.
The most important lesson here for engineering practice: when evaluating an MoE model, total parameters and active parameters are two separate metrics that must be read separately. Total parameters determine memory and storage footprint; active parameters determine inference compute. Looking at only one of them leads to wrong conclusions.
3. Sparse attention: the change that makes long context affordable
Compared with the previous generation, this model changes one thing structurally: some layers replace full attention with a sparse attention mechanism (DSA). The impact is concentrated in long-context workloads.
Under full attention, every token attends to every token in the sequence, so compute grows quadratically with sequence length. At a million tokens, that cost becomes effectively unpayable. Sparse attention looks only at the handful of most relevant tokens, pushing attention cost from quadratic toward roughly linear while preserving long-context capability and cutting both compute and memory use.
The takeaway for developers: if your workload leans heavily on long documents, large codebases, or very long multi-turn conversations, whether sparse attention is supported should be a first-class selection criterion. It largely determines whether long-context inference is feasible and what it costs.
4. Mixed precision and quantization: why there is only one release
A notable detail of the release is that the team published a single precision variant only, using MXFP4 weights with MXFP8 activations, produced through quantization-aware training (QAT) rather than by brutally compressing a finished model.
- MXFP4 (4-bit) weights. Compressing parameters to 4 bits is what brings the weight footprint down to roughly 1.56 TB and makes consumer-grade and professional-grade hardware a plausible target at all.
- MXFP8 activations. Activations stay at 8 bits, balancing accuracy against efficiency.
- Why QAT matters. Quantization-aware training simulates quantization error during training, so the model learns to work at low precision. At the same bit width it usually loses less accuracy than post-training quantization (PTQ).
One caution: because the only published variant is quantized, with no high-precision counterpart such as BF16 or FP8, precision-critical scenarios need extra care. Strict scientific computation and financially regulated inference are the obvious examples. Validate the actual accuracy loss of the quantized build on your own target task first; do not assume it is equivalent to an original high-precision checkpoint.
5. Local deployment: do the memory math first
“A multi-GPU server to start with” is the most realistic constraint of this class of model, and the reason is simple arithmetic:
- Weight footprint. At 4 bits, 2.8 trillion parameters need on the order of 1.4 TB of memory just to hold the weights, before KV cache, activations, and framework overhead are counted.
- KV cache. A 1-million-token context produces a very large key-value cache that grows linearly with concurrency and sequence length. It is the invisible memory killer.
- The combined picture. With both added in, weights around the 1.56 TB scale essentially rule out a single machine with a few accelerators. You generally need multiple data-center-class accelerators with large HBM, using tensor or pipeline parallelism. Eight GPUs is a starting point, not a comfortable configuration.
Practical advice for teams:
- Build the memory budget sheet first. Itemize weights, KV cache, parallelism sharding, and framework headroom, then decide GPU count and model.
- Evaluate cloud elasticity before building. Standing up an eight-GPU cluster is a large upfront investment. If utilization will be intermittent, GPU instances and container platforms that scale up on demand and release when idle often win on cost. Weights and datasets can live in object storage, with a message queue handling job scheduling and monitoring and alerting watching memory and throughput.
- Align the quantization toolchain. Since the release ships a specific mixed-precision format, confirm that your inference framework and hardware support it natively. Otherwise you may trigger extra dequantization overhead that eats the entire performance gain.
6. License: this is open weights, not open source
This is the most misread and most important point to get right on its own.
The model ships under a custom license, tagged as “other” rather than MIT or Apache 2.0 on hosting platforms. The body of the license is indeed permissive in the MIT style: free use, copy, modify, distribute, fine-tune, and commercial use are all allowed. On top of that, it adds two restrictions aimed at very large external-facing businesses:
- Enterprises that offer the model as a service (MaaS) and exceed a cumulative revenue threshold over any 12 months (for example, USD 20 million) must sign a separate agreement before commercial use.
- Products with monthly active users (MAU) above a certain scale (for example, 100 million) are likewise bound by the additional terms.
Several firm conclusions follow:
- Open weights and open source are not the same thing. Standard open-source licenses, as defined by the OSI, generally do not permit discriminatory restrictions tied to specific use cases such as revenue or user count. A license carrying threshold clauses of this kind is better described as an open-weights license: source or weights are available, but not fully free.
- The vast majority of small and medium businesses, individual developers, and internal research are unaffected. Normal commercial use, fine-tuning, and redistribution are all fine.
- Only at a considerable revenue or MAU scale do you need to revisit the additional terms and assess whether a separate agreement with the copyright holder is required. Well before you scale, license review should be part of your compliance process.
7. Reading the benchmarks correctly: watch out for self-reported scores
Releases of this kind usually come with a comparison table claiming wins over a leading international closed model on several benchmarks. Three principles apply when reading such data:
- Separate self-reported from third-party results. Vendor-reported scores are naturally subject to selective presentation. Independent third-party evaluations and head-to-head comparisons carry more weight.
- Read the shape, not the total. The typical profile of this model is strong on coding and agentic tasks, weaker on science and hard reasoning. It can trade blows with top models on some coding and software-engineering benchmarks, but trails clearly on scientific tasks and high-difficulty composite reasoning. Selection must match the workload: if your core is code generation, tool use, or multi-step agent tasks, it is competitive; if your core is rigorous scientific reasoning, validate carefully.
- Re-test with your own data. No public score replaces measurement on your real business dataset. Build an evaluation set close to your actual workload and compare candidate models on accuracy, latency, throughput, and cost per request. That is the only actionable basis for a decision.
8. Go-live checklist
For a trillion-scale open-weights model, work through the following:
- Can it run? Do the memory math first (weights around 1.4 TB plus KV cache), confirm GPU count and model, and evaluate elastic cloud GPUs as the priority option.
- Does it run well? Verify native framework support for the mixed-precision format, then re-test accuracy and throughput on real business data.
- Can it be used commercially? Check the license’s additional terms. Below the revenue and MAU thresholds you can use it with confidence; before scaling, do a compliance review.
- Is it worth using? Ignore the total score and read the shape: coding and agent strong, scientific reasoning weaker. Match that profile to your own workload.
Trillion-scale open weights lower the bar for frontier models from “cannot be built” to “affordable, if you do the math.” What determines success is not how large the model is, but whether you have accounted for architecture, memory, precision, and licensing.
