2.8T Open-Weight MoE Model: Architecture, Benchmarks, License, and 8-GPU Deployment
A 2.8T-parameter open-weight flagship model has landed: roughly 1.56TB of weights and an 8-accelerator starting point. This article breaks the release down across four layers — architecture, benchmark claims, licensing, and deployment — and hands you a reusable VRAM estimation method plus a self-hosting decision checklist. You will learn how the 2.8T figure is actually assembled (896 routed experts, 16 selected per token, 2 shared experts, 93 layers), how to read vendor-reported benchmark tables against a closed frontier model, why "open weights" here is not an OSI-approved open-source license, and what "8 GPUs minimum" really requires in practice.
A 2.8T-parameter model with roughly 1.56TB of weights and an 8-accelerator starting point landed on schedule as an open-weight flagship, and it immediately became a talking point among engineering teams. Behind the noise, three questions need answering first: how was this scale actually assembled, how should the official benchmark results be read, and what exactly does “open” mean here? This article takes the release apart across four layers — architecture, evaluation, licensing, and deployment — and provides a VRAM estimation method and a deployment decision checklist you can apply directly.
From API Preview to Released Weights
The model first appeared as an API and a web interface, and the team committed to publishing the full weights within a month. On November 4, the weights showed up on major model hosting platforms as expected: the repository is publicly downloadable, with no application or approval step. Links circulated widely across technical communities, with some people starting the download overnight and others immediately testing whether it would run locally.
The release process itself shows a clear trend: “open” releases from top-tier labs are shifting from verbal promises to verifiable timelines. The model is validated through a hosted service first, then the weights are opened on a committed schedule — service-side feedback polishes the model, and the open release amplifies ecosystem reach. The download side is worth noting too: the weight repository is made up of hundreds of files, more than 90% of which are shard files, for a total of roughly 1.56TB. Some teams also mirror the weights onto community platforms so that users can pull from faster sources.
Architecture: Where 2.8T Comes From
Total parameters vs. activated parameters: two separate ledgers
According to the official model card, this is a natively multimodal agent model, continued from the previous generation’s base model:
- 2.8T total parameters, with only about 104B activated per token;
- A mixture-of-experts (MoE) architecture with 896 routed experts, 16 selected per token, plus 2 shared experts that always participate in computation;
- 93 layers in the backbone, with a 1M-token context window;
- A built-in vision encoder, so image inputs can be handled directly.
Behind these numbers sit two entirely different resource ledgers:
| Dimension | What drives it | Order of magnitude |
|---|---|---|
| VRAM capacity | All 2.8T weights must stay resident in memory | ~1.56TB, a hard floor |
| Compute per token | ~104B activated parameters | Roughly a 100B-class dense model |
| Memory bandwidth | Every generated token requires reading the activated weights | The second bottleneck, on top of capacity |
This is exactly where MoE earns its cost-efficiency: total parameter count determines knowledge capacity, while activated count determines inference cost. Most of the 2.8T “bulk” sits idle most of the time, yet the real compute overhead is billed like a 100B-class dense model. The trade-off is that the VRAM capacity floor rises sharply. That is also why “a very large model” and “a very expensive machine” are not the same thing: the cost is driven mainly by memory capacity, not by compute.
Routing: 896 experts, choose 16, plus 2 shared
A standard decoder stacks attention sublayers and feed-forward sublayers alternately, and MoE modifies the feed-forward part. After the attention output, each token passes through a gating network that scores all experts and selects the 16 highest-scoring ones; the token’s computation is completed only on the parameters of those 16 experts. Shared experts are not gated — every token uses them, which makes them a natural home for general capabilities. Routed experts specialize in different patterns, expanding total capacity while keeping per-token compute under control.
The engineering difficulty here is load balancing. If a handful of “hot experts” attract a disproportionate share of tokens, some accelerators are overloaded while others sit idle — wasting VRAM and dragging down throughput. Three countermeasures are common:
- Add a load-balancing loss during training so expert assignment stays as even as possible;
- Set an expert capacity ceiling, so overflow tokens spill to the next-best expert or are skipped;
- At inference time, dynamically adjust expert placement across devices based on live load.
Routing also affects model behavior directly. When the model encounters an input pattern it never saw during training, the gate may select an ill-suited combination of experts, showing up as an abrupt shift in answer style or a clear weakness on specific task types. Run coverage tests against your own workloads before going live.
1M-token context and native multimodality
A 1M-token context is roughly a document of several million Chinese characters, thousands of PDF pages, or hundreds of hours of meeting transcripts — a good fit for repository-level code understanding, long-report analysis, and multi-turn agent tasks. But long context carries two costs you must confront:
- VRAM: the KV cache grows linearly with sequence length;
- Compute: attention cost typically grows quadratically with sequence length.
“Native multimodality” means visual capability was folded into the model during pre-training, rather than being bolted on as a separate vision module attached to a text-only model. Models like this usually achieve more natural semantic alignment on mixed image-and-text material such as screenshots, scanned documents, and UI mockups. In a local deployment, the vision encoder consumes part of the VRAM budget, and image inputs generate additional visual tokens — both have to be counted in the capacity plan.
Sparse attention: only 24 layers touched
Compared with the previous generation, the architecture changed in exactly one place: 24 layers of full attention were replaced with sparse attention (DSA). Sparse attention lets each token compute attention over only a subset of historical positions, cutting the compute overhead that otherwise grows quadratically with sequence length. What you gain is cheaper long-context inference, while the remaining layers keep their high-quality global modeling ability.
Note, though, that this optimization saves compute, not the KV cache. The indexing mechanism itself may add a small amount of extra VRAM overhead. For memory budgeting, still plan for the full KV cache.
Low-precision weights: the official release ships one version only
The weights use MXFP4. The MX family is an industry-standard microscaling low-precision floating-point format that compresses a block of values down to 4 or 8 bits while sharing a scale factor across the block. The vendor states this version is the product of quantization-aware training: quantization error is simulated during training so the model adapts to low precision in advance, which is why accuracy loss is markedly smaller than with post-training quantization.
The problem is that this is the only version released — there is no higher-precision fallback. That matters a great deal for anyone deploying, and we will come back to it below.
Reading the Benchmarks: Vendor-Reported Numbers and a Mixed Record
The official comparison table covers more than a dozen benchmarks, with same-tier international closed frontier models as the opponents, and every number is vendor-reported. The result can be summed up in one line: coding and agent capability are close to or locally ahead of the first tier, while science and parts of reasoning trail clearly behind.
- Coding: the terminal-operation benchmark scores slightly below the compared closed flagship, while the long-horizon software engineering benchmark leads clearly;
- Weak spots: the science reasoning benchmark scores below two-thirds of the opponent, and the hard knowledge benchmark gap exceeds 20 percentage points.
Cross-checking the vendor’s claims against the leaderboards: the vendor says it leads the comparison model on 16 of 37 benchmarks, for an overall “wins and losses split roughly evenly” picture — advantages concentrated in coding, terminal operation, and agent tasks; disadvantages concentrated in science and complex reasoning. One third-party evaluation organization gives the model a composite intelligence index of 55, ranking fourth globally.
When you read numbers like these, three principles help:
- Self-reported is not independently reproduced: benchmark scores are affected by evaluation prompts, sample counts, tool-calling configuration, and reasoning budgets (thinking tokens). When comparing against your own models, make sure evaluation conditions are identical.
- A leaderboard is not your business: general benchmarks cannot reflect performance on your private codebases, domain documents, and business workflows — your own evaluation sets are the final authority.
- Weak spots define the usage boundary: for scientific computing and complex reasoning tasks, prepare fallbacks, such as routing to a stronger model or adding a self-verification step in the pipeline.
License: Open Weights Is Not Open Source
This is the point most worth spelling out separately. The repository carries a custom license, tagged as an “other” license on the hosting platform rather than a standard open-source license such as MIT or Apache. Comparing term by term, the new license matches the license terms of the earlier models in the same family — only the name changed.
The core grant is indeed MIT-style: free use, copy, modify, distribute, fine-tune, and commercial use. Two restrictions are layered on top, aimed specifically at “large-scale external business”:
- Companies offering the model as a model-as-a-service externally, with cumulative trailing-12-month revenue above USD 20 million, must sign a separate agreement before commercial use;
- Commercial products with more than 100 million monthly active users likewise require a separate agreement.
Two examples show where the line sits:
- A SaaS company with USD 20 million in revenue embeds the model in its own product and sells it to customers. As long as the model is not sold separately as an external service, the first clause is not triggered.
- A cloud provider offering this model externally via API, with revenue well above the threshold, must sign an agreement first.
The distinction between “open weights” and “open source” is worth expanding on:
| Dimension | Typical open-source license (e.g. MIT, Apache-2.0) | Custom open-weight license discussed here |
|---|---|---|
| Weight access | Available | Public download, no application required |
| Commercial use | Free commercial use | Permitted in principle; separate agreement needed once revenue or MAU thresholds are met |
| Modification and redistribution | Allowed | Allowed |
| Training data and code | Open in some projects | Generally not open |
| OSI-approved | Yes | No |
For enterprises preparing to adopt the model, it is worth putting a model license review process in place:
- Legal assessment: confirm whether your business could hit the revenue or MAU thresholds, and what your revenue expectation is for the next 12 months;
- Usage registry: record model version, download source, deployment location, and intended use so it can be traced later;
- Redistribution review: if you bundle the model into a commercial product or serve it via API, verify the license terms before launch;
- Periodic re-review: license terms can change with new versions, so review again before upgrading the model.
Can You Run It Locally? The VRAM Math Behind “8 GPUs Minimum”
The conclusion first: “8 GPUs minimum” assumes 8 accelerators with at least 141GB of memory per card and native support for low-precision floating-point compute — and even then, it only means “it boots,” not “it runs comfortably.” Ordinary workstations, consumer graphics cards, and even common 8-card servers cannot hold these weights.
Capacity budget: 1.56TB spread across 8 cards
Two unit conventions have to be kept apart: the TB figure on the download page is normally decimal, while VRAM is measured in binary GiB.
| Configuration | Total VRAM (GiB) | Relation to total weight size |
|---|---|---|
| 8 × 141GB cards, decimal capacity | 1128 | Holds only about 72% of the weights |
| 8 × 141GB cards, binary capacity (131GiB × 8) | 1048 | Not enough |
| 8 × 192GB cards | 1536 | Just fits, with almost no headroom |
| 16 × 141GB cards | 2256 | Some headroom |
So 8 × 192GB cards just barely hold the weights, and the remaining space still has to accommodate the KV cache, activations, and communication buffers. In practice the usable headroom is often only a few tens of GiB, which forces a choice between long context and high concurrency.
For comparison, 8 mainstream 80GB training cards total 640GB — not even half the weights — which makes that configuration a non-starter. To run comfortably, you generally need 16 × 192GB cards or 32 × 141GB cards.
Also note that the 1.56TB includes everything, the vision encoder among it, so the weights actually resident in memory during inference are slightly fewer — but the overall magnitude is unchanged.
Three gates to clear before deployment
- Precision support: both the hardware and the inference framework must natively support MXFP4 weights and MXFP8 activation computation. Otherwise you have to dequantize first, at which point weight size at least doubles, VRAM requirements double with it, and the 8-card configuration immediately stops working.
- Operator coverage: support for the new operators behind sparse attention, native multimodality, and an oversized vocabulary varies widely across mainstream inference frameworks. When some operators are missing, the framework falls back to generic implementations, which sharply degrades performance or prevents startup altogether. Before deploying, run a smoke test with a single short request to confirm the model produces output.
- Context budget: the vendor claims 1M-token support, but the KV cache drains VRAM fast. Estimating with 93 layers and 50% of attention layers using sparse attention, the KV cache for 1M tokens costs roughly:
| Setting | Precision | Approximate KV cache |
|---|---|---|
| Retain everything | BF16 | ~465GiB |
| Retain everything | FP8 | ~232GiB |
| Retain everything | 4-bit | ~116GiB |
| Retain partial history in sparse layers only | BF16 | ~279GiB |
The headroom on 8 × 192GB cards cannot even hold a 1M-token BF16 KV cache. To approach million-token context you need FP8/4-bit KV cache, selective history retention in the sparse layers, plus offloading techniques such as context chunking and on-demand loading.
Parallelism: how to split across 8 cards
| Strategy | What is split | Communication profile | Typical use case |
|---|---|---|---|
| Tensor parallelism (TP) | In-layer matrices split by row or column | Collective communication at every layer; sensitive to inter-card bandwidth | Single node with 8 cards, relying on fast interconnect |
| Expert parallelism (EP) | The 896 experts distributed across cards | All-to-all communication driven by token routing | Combined with TP |
| Pipeline parallelism (PP) | Layers divided into multiple stages | Activations transferred across nodes; pipeline bubbles | Larger card counts, cross-node deployment |
| Context parallelism (CP) | Split along the sequence dimension | Attention computation needs to exchange KV across cards | Extremely long context |
The typical single-node 8-card combination is TP × EP = 8, meaning tensor parallelism and expert parallelism both use all 8 cards. Inter-card interconnect inside a node is usually “in-band” communication with high bandwidth and low latency, and performs far better than cross-machine networks.
If you need to scale to 16 or 32 cards, combinations like “TP8 × PP2” or “TP8 × PP4” work: try to keep each TP group inside a single machine and let only pipeline parallelism cross machines. Where possible, prefer a cluster with stronger in-rack cross-node interconnect. The sample configuration below illustrates the basic parameters for a single-node 8-card deployment:
# Illustrative configuration: parameter names must be adjusted to your inference frameworkmodel_path: /mnt/models/2p8t-moeweight_dtype: mxfp4quantization: mxfp4 # requires native support in both hardware and inference frameworktensor_parallel_size: 8expert_parallel_size: 8pipeline_parallel_size: 1max_model_len: 262144 # size to your VRAM budget; do not set 1M directlykv_cache_dtype: fp8 # try a lower KV precision when VRAM is tightgpu_memory_utilization: 0.92 # leave room for activations and communication buffersenable_chunked_prefill: true # chunk long prefills so long requests do not block othersenable_expert_load_balance: true
Performance expectations: fast is not the same as cost-effective
Per-token compute for an MoE model is roughly equivalent to a 100B-class dense model, but generating every single token requires reading the weights of about 104B parameters.
- At low concurrency, performance is bound mainly by VRAM bandwidth: even with accelerator bandwidth on the order of 8TB/s, the theoretical ceiling is only a few hundred tokens per second;
- At high concurrency, tokens from multiple requests can be batched together so a single weight read serves several requests, which raises throughput significantly — but per-request latency increases;
- In long-context scenarios, sparse attention eases the compute pressure, while the VRAM and bandwidth cost of the KV cache remains.
One limitation deserves an honest note: because the original materials did not publish the model card for the model used, this article cannot verify details such as the attention structure and KV head counts. The estimates above assume “50% of the 93 layers use sparse attention, with KV head count and hidden dimension within the common range for comparable models,” so actual values may deviate considerably. Confirm the VRAM budget against the structure given in the model card. Whether a sparse-layer KV policy is actually usable also depends on real support in your inference framework. The estimation method itself is general: substitute the real values for the parameters above and you get your own numbers.
Finally, the arithmetic: a full node of 8 × 192GB cards typically costs on the order of a million RMB to procure, and with datacenter space, power, and operations on top, the total cost of ownership over 3–5 years is not small. By comparison, calling a hosted API billed per token suits teams with spiky traffic that do not want to build their own cluster. How to choose is covered in the next section.
Self-Host or Call the API? A Decision Checklist
Run through the six questions below and consider self-hosting only if you meet three or more:
- Capability need: does the business genuinely require 2.8T-class capability, such as large-scale codebase understanding, very long document analysis, or complex agent orchestration?
- Data security: do compliance requirements forbid the data from leaving your environment?
- Cost scale: is daily token consumption large enough that amortized self-hosting cost is clearly below API cost?
- Hardware availability: can you obtain accelerators with more than 141GB per card that support low-precision compute, along with the matching high-speed interconnect and datacenter capacity?
- Engineering capability: does the team have distributed inference deployment, performance tuning, and troubleshooting skills?
- License compliance: could you hit the USD 20 million revenue or 100 million MAU thresholds, and can you accept the uncertainty of signing a separate agreement?
If the answer is “not yet,” several pragmatic middle paths remain:
- Hybrid architecture: handle sensitive data with a local model and route complex tasks to a hosted model, balancing compliance and quality;
- Right-sized models: many open-weight models in the tens-of-billions to hundreds-of-billions parameter range run on one or two cards, and are already sufficient for most business scenarios;
- Cloud deployment: on a cloud provider, spin up inference clusters on demand using heterogeneous compute instances, container services, and high-speed networking, turning fixed investment into elastic spend.
Conclusion
The arrival of this 2.8T-parameter open-weight model marks the open ecosystem’s parameter scale entering the trillion-parameter era, and it also exposes several practical problems in putting open models to work:
- Architecture: MoE decouples total capacity from inference cost, but shifts the pressure onto VRAM capacity, routing load balance, and low-precision operator support;
- Evaluation: vendor-reported data and third-party rankings are both reference points only; strengths in coding and agent work paired with weaker scientific reasoning define the model’s usable boundary;
- Licensing: the large-scale commercial thresholds layered onto a custom license are a reminder that “open weights” does not mean “free commercial use”;
- Deployment: “8 GPUs minimum” really means 8 accelerators with more than 141GB that natively support MXFP4, and million-token context additionally requires KV cache optimization.
For developers, the rational approach to any “bigger and stronger” open model comes down to three questions: does the license permit it, do the machines fit it, and is it worth it for the business? Answer those three clearly, then decide whether to download, deploy, or keep waiting.
