2.8T Open-Weight MoE: Understanding VRAM Requirements and Commercial Licensing
A research team recently released the full weights of a 2.8T parameter Mixture-of-Experts (MoE) model, making it one of the largest open-weight models available. Before deploying this model, developers must address two critical questions: licensing constraints and hardware feasibility. This article analyzes the model's architecture, performance, commercial license terms, and provides a detailed guide on estimating VRAM requirements for local deployment.
1. Background
A research team has publicly released the complete weights of its flagship MoE model as planned. With a total of 2.8T parameters and a download size of approximately 1.56TB, it is currently one of the largest open-weight models available. The developers claim it excels in agent and programming tasks. For developers, two primary questions must be answered before adoption: Can the license be used for your business? And can your existing hardware handle it?
2. Architecture: 2.8T Total Parameters, 104B Active Per Inference
This model is a native multimodal agent model. According to public technical documentation, it activates only 104B parameters per inference, utilizing a sparse activation MoE design. The model consists of 93 layers and contains 896 experts. For each token processed, the routing mechanism selects 16 experts, while 2 additional shared experts handle general features. The model includes a built-in visual encoder and features a context window of 1 million tokens.
Regarding attention, 24 layers have been replaced with sparse attention (DSA). Since full attention computation grows quadratically with sequence length, sparse attention reduces the computational overhead for long texts by limiting the interaction range between tokens. While sparse attention primarily saves computation, its impact on KV Cache VRAM depends on the specific implementation and should be verified through testing before deployment.
The weights use the MXFP4 format, and activations use MXFP8, both products of Quantization-Aware Training (QAT). QAT introduces quantization noise during training, allowing the model to adapt to low-precision representations. The official release provides only this version, meaning inference frameworks must natively support this mixed-precision computation, or you will need to adapt it yourself.
3. Performance: Strong in Coding and Agents, Weak in Scientific Reasoning
According to the comparison tables released by the developers, the model performs well in code execution tests like Terminal-Bench and engineering tasks like SWE Marathon, with some scores surpassing top-tier closed-source models. The team stated that the post-training phase focused on optimizing tool calling, code generation, and logical planning capabilities.
However, on scientific reasoning benchmarks such as Terminal-Bench-Science and HLE-Diamond, the model scores significantly lower than industry benchmark models. Overall, it is better suited for vertical scenarios like automated DevOps and coding assistance, rather than serving directly as a general-purpose scientific reasoning foundation.
Before citing these figures, it is essential to verify the benchmark versions, evaluation settings, and comparison models, as scores from different versions cannot be directly compared.
4. License: Open Weights Do Not Equal Open Source
“Open source” typically refers to software following OSI-approved protocols, such as MIT or Apache 2.0. This model uses a custom protocol labeled “other” on the platform, classifying it as open weight rather than open source.
According to the agreement summary, users can freely use, copy, modify, distribute, fine-tune, and use it commercially. However, the agreement imposes restrictive clauses on large-scale operations. If an enterprise provides the model as an external service and exceeds a specific revenue threshold within a 12-month period, or if the monthly active users (MAU) of a commercial product exceed a certain limit, a separate commercial agreement must be signed with the developers prior to commercial use. Refer to the original agreement for the exact thresholds.
The purpose of such clauses is to prevent large cloud providers from directly using open weights to build competing API services. For fast-growing startups whose user bases might approach these limits, it is advisable to have legal counsel review the original agreement line by line during the early selection phase and document the conclusions in the architecture evaluation.
5. Local Deployment: How to Calculate VRAM, Are 8 GPUs Enough?
First, calculate the VRAM required for the weights alone. 1.56TB is approximately 1560GB. With MXFP4 quantization, the weight size is essentially the same as the download size, meaning the weights alone require about 1560GB of VRAM. The “780GB” conversion mentioned in some texts is incorrect and should not be used.
Based on per-GPU VRAM, the minimum GPUs required just for the weights are:
- 80GB per card: At least 20 cards
- 141GB per card: At least 12 cards
- 192GB per card: At least 9 cards
To fit the weights onto exactly 8 GPUs, the average VRAM per card must be around 195GB or more. This figure does not account for KV Cache and framework overhead; the actual number of cards required will be higher.
A comprehensive VRAM estimate can be calculated using the following formula:
VRAM Requirement ≈ Weight Size + KV Cache + Activation & Framework Overhead
The KV Cache can be estimated using:
KV Cache ≈ 2 × Number of Layers × Number of KV Heads × Head Dimension × Bytes per Element × Context Length × Concurrent Requests
With a context length of 1 million tokens, the KV Cache will be substantial. It scales linearly with context length and concurrent requests; the higher the concurrency, the more total VRAM limits the number of servable requests. The number of layers, KV heads, and head dimension should be read directly from the model configuration file rather than estimated based on experience.
During MoE inference, expert parameters require frequent reading, so the interconnect bandwidth between GPUs (e.g., NVLink) directly impacts latency. Interconnect bandwidth can become a bottleneck and should be tested with actual workloads before finalizing hardware selection.
Regarding download and storage, the 1.56TB file should be downloaded using a tool that supports resumable downloads, and the official hash values must be verified. If multiple copies or versions are saved simultaneously, storage requirements will multiply. The cold start time is roughly equal to the file size divided by the read bandwidth. At 2GB/s, loading 1.56TB takes about 13 minutes.
6. Selection Checklist
- License: Read the original agreement thoroughly, confirm the revenue and MAU thresholds, and obtain a written conclusion from legal counsel.
- VRAM: Use the formulas above to calculate weights, KV Cache, and framework overhead to determine the GPU type, count, and total VRAM.
- Framework: Ensure the inference framework supports MXFP4 and MXFP8 mixed-precision computation.
- Performance: Test latency, throughput, and output quality using real request samples from your business; do not rely solely on public benchmark scores.
- Cost: Compare the Total Cost of Ownership (TCO) of local private deployment versus public cloud inference.
For most teams, it is recommended to first verify effectiveness through cloud APIs or small-scale trials. Once business value is confirmed, you can then decide whether to invest in local deployment. This approach carries lower risk and avoids premature costs in hardware and compliance.
