2.8 Trillion Parameter Open-Weights Model: Local Deployment, VRAM Requirements, and License Explained
A 2.8T-parameter native multimodal MoE model has released its full weights for download, and that changes the deployment question from "can we get it" to "what does running it actually take." This article breaks the release down across four dimensions: architecture (896 routed experts plus 2 shared experts, ~104B active parameters per token, 93 layers with a hybrid full/sparse attention design, 1M-token context), format (MXFP4 weights and MXFP8 activations produced by quantization-aware training, with no higher-precision release), licensing (open weights with use restrictions, where general commercial use is unrestricted but large-scale MaaS or high-MAU commercial products require a separate agreement), and deployment (a 1.56TB weight package that makes per-GPU memory the hard constraint, with an 8-GPU node as the realistic starting point). It also covers how to read vendor-reported benchmark numbers, a VRAM estimation method you can run before pulling the weights, and a pre-launch engineering checklist.
Introduction: Trillion-Scale Open Weights Open a Different Door
Over the past two years, the capability ceiling of open-weight models has steadily closed in on closed frontier models. Most open models, however, have been constrained by parameter count and licensing terms, which makes it hard for enterprises to treat them as the foundation of a production system.
Recently, a Chinese LLM vendor delivered on its earlier commitment to “publish full weights within a set period”: a native multimodal model with roughly 2.8 trillion total parameters, released as a weight package of about 1.56TB that anyone can download without applying for access. Among the weights currently available openly, it sits in the largest tier.
Once the weights are public, “can we download it” stops being the question. Three questions actually matter:
- What’s inside: how are trillion-scale parameters organized, and what does sparse activation imply?
- How does the license work: is commercial use free, and which scale of use requires a separate negotiation?
- Can it run, and how: weights that don’t fit on a single GPU demand what hardware and what deployment approach?
The short answers:
- Model side: a sparse Mixture-of-Experts (MoE) architecture with about 2.8T total parameters that activates roughly 104B per token. Context length is 1 million tokens, with image input supported.
- Format side: weights are 4-bit float quantized, activations are 8-bit float quantized, both produced by quantization-aware training. The vendor has not published a higher-precision version.
- License side: “open weights + use restrictions.” Ordinary commercial use is unconstrained, while very large-scale external model services and commercial products require a separate agreement.
- Deployment side: per-GPU memory capacity is the hard constraint. Eight GPUs is the starting configuration, per-GPU memory is typically in the hundreds of gigabytes, and the integrated server form factor is the most common deployment.
1.1 Release timeline: from promise to delivery
The model first became available on October 24 through an API and a web interface, at which point the vendor committed to publishing the full weights before November 4. On November 2 the weight repository was created on a public code hosting platform. On the morning of November 4 it was still closed; that evening it opened officially. The package consists of 112 files, 96 of them safetensors shards, totaling about 1.56TB, downloadable without an application.
1.2 Why “downloadable” deserves its own discussion
“Open weights” and “API-only access” are two entirely different ways to use a model:
| Dimension | API-only access | Downloadable open weights |
|---|---|---|
| Data flow | Inference requests must leave the corporate network | Inference can run entirely within a private environment |
| Customizability | Limited to what the API exposes | Fine-tuning, quantization, distillation, including distilling smaller models |
| Deployment location | Cloud service | Own data center, private cloud, or cloud GPU instances |
| Cost structure | Operating expense billed per token | Upfront hardware or compute investment plus operations staff |
| Barrier to entry | Low | High: VRAM, networking, inference framework, license review |
Put simply, open weights turn “using someone else’s capability” into “owning your own capability,” at the price of hardware investment and operational responsibility. For teams with data-sensitive workloads, a need for deep customization, or long-term cost-control goals, this path is worth evaluating. For teams with highly variable call volume and no GPU operations experience, API access remains the more economical choice.
2. Model Architecture: How 2.8T Parameters Are Organized
2.1 Architecture overview
According to the official model card, the model is positioned as a native multimodal model built for agentic tasks, continued from training on the previous generation. “Native multimodal” means text and images are jointly modeled in a single model rather than bolting a separate vision module onto a language model. The publicly disclosed architecture parameters are:
| Item | Parameter |
|---|---|
| Model type | Native multimodal, agent-oriented |
| Total parameters | ~2.8T |
| Active parameters per token | ~104B (~3.7% of total) |
| Expert structure | 896 routed experts + 2 shared experts, 16 routed experts selected per token |
| Layers | 93 |
| Attention | Full attention on most layers, 24 layers replaced with sparse attention |
| Context length | 1M tokens |
| Vision encoder | Dedicated vision encoder, supports image input |
| Weight / activation precision | 4-bit float (MXFP4) / 8-bit float (MXFP8) |
2.2 MoE: capacity from total parameters, cost control from active parameters
The core idea of MoE is to split the feed-forward network into a large number of structurally identical “experts,” with each token passing through only a few of them. Sparse activation has two immediate effects:
- Compute stays bounded: ignoring parameters that cannot be split by expert, such as attention and the shared experts, the actual expert-side compute per token is about 104B × 2 ≈ 208 GFLOPs, roughly 3.7% of what a dense 2.8T-parameter model would need.
- Knowledge capacity is ample: 2.8T total parameters mean knowledge across different domains, languages, and tasks can be spread across different experts, reducing interference between parameters.
Two design choices in the expert structure stand out:
- Division of labor between routed and shared experts: the 896 routed experts are selected dynamically per token, while the 2 shared experts are always active for every token. The shared experts carry general knowledge, letting the routed experts specialize, which helps mitigate “expert collapse” — the problem where a handful of experts are repeatedly selected while the rest are undertrained.
- 16 experts active per token: selecting 16 out of 896 candidates leaves plenty of routing space. It also means every forward pass must read 16 sets of expert weights from memory on demand, so inter-GPU communication grows when you deploy with expert parallelism.
This leads to the key distinction that matters for the rest of this article: memory footprint is driven by total parameters, compute cost is driven by active parameters. All weights must be resident, yet each token touches only about 3.7% of them. That is exactly why models like this have a high memory threshold but a comparatively low compute threshold.
2.3 Long context: a mix of full and sparse attention
A 1M-token context length is a hard requirement for long-document parsing, understanding large code repositories, and long-horizon agentic tasks. But the compute and memory cost of standard full attention both grow quadratically with sequence length, and a 1M-token context is effectively unserviceable under a conventional implementation.
This model’s approach: most of the 93 layers keep full attention for global information exchange, and 24 layers are replaced with sparse attention. Sparse attention computes attention scores only for a subset of token pairs, trading a small accuracy loss for lower long-sequence cost. This hybrid “full + sparse attention” layer design is a common approach when model vendors balance long-context capability against inference cost. When evaluating, pay close attention to how much sparse attention layers degrade on very long inputs, rather than only looking at the context limit on the spec sheet.
2.4 Quantization-aware training: 4-bit weights, 8-bit activations
Weights are stored as 4-bit float (MXFP4) and activations are computed as 8-bit float (MXFP8). Both are microscaling float formats: a group of values shares one scale factor, extending the representable range in small blocks. Compared with conventional quantization where an entire tensor shares a single scale factor, block-level scaling better preserves outliers in the value distribution.
MXFP4 is an open format specification driven by the Open Compute Project (OCP), and hardware and inference framework support is still expanding. Formats like this are produced through quantization-aware training (QAT): the training phase simulates the error introduced by low-precision inference so the model adapts to 4-bit storage early, which typically costs less accuracy than post-training quantization (PTQ).
What does releasing only a 4-bit version imply?
- You can’t simply move back to higher precision: no BF16 or FP8 version has been published. If you dequantize or continue training yourself, you must validate the accuracy regression on your own, and the risk is entirely yours.
- It imposes framework and hardware requirements: the inference framework must support the format, otherwise you fall back to a dequantization path. After dequantization the weight volume multiplies and no longer fits on a single GPU.
- Ecosystem fit needs early validation: the level of support for 4-bit formats in fine-tuning, distributed inference, and secondary quantization tooling determines whether further development is even possible.
3. Reading the Benchmark Numbers
3.1 Key results (all vendor-reported)
The publisher released a comparison table against external frontier models. To keep the focus technical, the two reference models are labeled Model A and Model B below. All figures are vendor-reported and have not been reproduced by an independent evaluation body:
| Benchmark | This model (2.8T MoE) | Model A | Model B |
|---|---|---|---|
| Terminal tasks (Terminal-Bench 4.0) | 65.7% | 66.4% | 58.2% |
| Long-horizon software engineering (SWE Marathon) | 51.7% | 46.3% | — |
| Scientific terminal tasks (Terminal-Bench-Science) | 38.1% | 63.3% | — |
| Hard knowledge reasoning (HLE-Diamond) | 39.4% | 60.9% | — |
In addition, a third-party evaluation organization’s composite intelligence index gives it 55 points, fourth globally.
3.2 Three signals worth noting
- Strengths concentrate in coding and agentic tasks: terminal-task performance trails Model A by only about 0.7 percentage points, while long-horizon software engineering leads by about 5.4 points. That matches the “native multimodal + agent” positioning.
- Weaknesses are equally clear: scientific terminal tasks trail Model A by about 25 percentage points and hard knowledge reasoning by about 21.5 points. This is a set of single-axis tradeoffs, not an across-the-board lead.
- “More wins than losses” is not “leading everywhere”: the vendor states that 16 of 37 benchmarks beat Model A — fewer than half. Which benchmarks are chosen, whether the baseline versions are aligned, and whether the inference configurations are comparable all affect the final ranking.
3.3 How enterprises should validate: build your own benchmark
Public leaderboards only offer directional guidance. The actual decision should rest on a private evaluation close to your business:
- Construct a business evaluation set: sample 200–500 items from real tickets, code review records, and document-QA logs, covering both high-frequency scenarios and typical long-tail cases.
- Define scoring criteria: coding tasks can use objective metrics such as compile pass rate and unit-test pass rate; generation tasks can use human blind review or “strong model as judge,” with the scoring standard fixed in advance.
- Standardize the inference configuration: align prompts, sampling parameters, tool-calling strategy, and maximum output length. Agentic scores are highly sensitive to the toolchain and scaffolding, so results are not directly comparable across different configurations.
- Decide per scenario: if your core workload is code generation, terminal operations, and multi-step tasks, this class of model deserves serious evaluation. If the core need is scientific computing or hard reasoning, validate carefully with a private evaluation before committing.
4. Licensing: Where Open Weights Stop and Open Source Begins
4.1 Core clauses in the license text
This is the part of the release most likely to be misread. The repository uses the vendor’s own custom license, labeled “other” (license: other) on general-purpose platforms. It is neither a standard open-source license like MIT or Apache-2.0, nor on the Open Source Initiative (OSI) approved list. The text is word-for-word identical to the previous generation’s license, with only the name changed.
The body of the license is an MIT-style grant: free to use, copy, modify, distribute, fine-tune, and use commercially. On top of that, two restrictions apply to large-scale commercial scenarios:
- Large-scale external model service: enterprises that provide model capability to users as a service (model-as-a-service, MaaS) with cumulative revenue exceeding $20 million over the trailing 12 months must sign a separate agreement with the copyright holder before commercial use.
- Large-scale commercial product: commercial products whose monthly active users exceed a certain scale (the original text is truncated at this point; the specific threshold is governed by the full official license) are likewise subject to the restriction.
4.2 Why “open weights” is not “open source”
The essence of open source is granting users unrestricted freedom to modify, use, and redistribute. Among the OSI’s baseline requirements for open-source licenses are explicit terms that a license must not restrict the field of use and must not discriminate against persons or groups for the activities they engage in. Clauses that impose revenue or user-scale preconditions do not meet these requirements. A more accurate description is therefore: open weights (Open Weights) + use restrictions (Use Restrictions).
| Use case | Constraint |
|---|---|
| Personal study, technical research, internal evaluation | Generally unconstrained |
| Internal enterprise use (no change in service form) | Generally unconstrained |
| Embedded in a commercial product and distributed (below the user-scale threshold) | Generally unconstrained |
| External MaaS with trailing 12-month revenue > $20M | Separate agreement required |
| Commercial product MAU above the threshold | Separate agreement required |
4.3 Four points for compliant deployment
- Determine the “service form”: if your product exposes model capability externally through an API, self-assess against the MaaS revenue threshold.
- Clarify the “revenue basis”: the clause uses cumulative revenue over the trailing 12 months, not a single month’s turnover within a conventional financial period. Confirm with finance whether the statistical scope includes affiliated entities within the group.
- Clarify the “user basis”: define how MAU is counted (by account, device, or IP) and whether cross-platform users are deduplicated before launch.
- Retain an evidence chain: keep weight-file hashes, a snapshot of the license version, and download and signing records, in case of an audit or dispute.
4.4 A practical rule of thumb
Small and mid-sized teams and individual developers are essentially unaffected and can freely use, fine-tune, and commercialize. Teams with scaled revenue or a large user base should complete a license assessment at the project-initiation stage, rather than discovering a re-signing requirement shortly before launch.
5. Can You Run It Locally: VRAM Math and 8-GPU Deployment
5.1 The memory threshold: where 1.56TB comes from
Start with the size estimate. At 4-bit storage, 2.8T parameters occupy roughly 2.8T × 0.5 bytes ≈ 1.4TB. Add the scale factors of the microscaling format, tensor metadata, and per-shard serialization overhead, and the actual download size is about 1.56TB, matching the figure the vendor reports.
Divide that volume by per-GPU memory and you reach the intuitive conclusion that it is “almost impossible”:
- Per GPU 80GB: about 19.5 cards needed, so exactly 20;
- Per GPU 96GB: about 16.25 cards, so 17;
- Per GPU 141GB: about 11.06 cards, so 12.
“Eight GPUs to start” corresponds to mainstream high-end accelerators with roughly 192GB of memory per card. This is also why models of this class are typically deployed on integrated GPU servers rather than a handful of PCIe add-in cards.
5.2 Breaking down memory usage: three costs beyond weights
Total memory usage consists of four parts:
- Model weights: about 1.4TB after 4-bit quantization, by far the dominant share;
- KV cache: dependent on layer count, KV head dimension, sequence length, and concurrency. The longer the context and the higher the concurrency, the faster it grows;
- Runtime overhead: communication buffers, transient activation tensors, inference-framework memory pool fragmentation, and so on;
- Parallel replica overhead: parameters that cannot be split by expert — shared experts, embedding layers — must be replicated within each parallel group.
Eight GPUs as the starting point means each GPU carries about 1.56TB ÷ 8 ≈ 195GB of weight loading, with the remaining memory left for KV cache and runtime overhead. Compressing the quantization further (2-bit, for instance) or switching to a model with fewer parameters could theoretically bring this under 8 GPUs, but accuracy regression and memory fragmentation risk require additional validation.
5.3 A reusable VRAM estimation method
Without actually pulling the weights, you can estimate memory usage quickly with the pseudocode below:
def estimate_gpu_memory(params_T, bits, n_gpus, ctx_len, kv_bytes, scale_factor, runtime_gb):# 1) Weight footprint: params x bytes per param + block scale factorsweight_tb = params_T * 1e12 * bits / 8 / 1e12 * scale_factor# 2) Per-GPU weight shareper_gpu_weight_gb = weight_tb * 1024 / n_gpus# 3) KV cache (illustrative values; rely on the model card and measurements)per_gpu_kv_gb = kv_bytes * ctx_len / 1e9 * 1024 # simplified illustration# 4) Runtime overheadper_gpu_runtime_gb = runtime_gbtotal_gb = per_gpu_weight_gb + per_gpu_kv_gb + per_gpu_runtime_gbreturn total_gb# Example: 2.8T params, 4-bit quantization, 8 GPUs, 1M context# Note: tune kv_bytes and runtime_gb for the actual hardware and framework setupprint(estimate_gpu_memory(2.8, 4, 8, 1_000_000, 0.002, 1.15, 8))
Leave at least 10%–20% of memory as headroom to absorb concurrency spikes, KV cache fragmentation, and overhead changes from framework upgrades.
5.4 Inference framework and hardware requirements
Loading and computing 4-bit float weights depends on the inference framework supporting the format. If it doesn’t, you’re limited to a dequantization path: after dequantization the weight volume multiplies, won’t fit on a single GPU, and the 8-GPU deployment fails outright.
On the hardware side, accelerators with native 4-bit float compute support — newer-generation architectures supporting 4-bit/8-bit float mixed precision — are the current mainstream choice. Confirm three things before purchasing:
- Does the target inference framework already support this quantization format?
- Does the target hardware natively support 4-bit/8-bit float compute?
- Are the framework version and the hardware driver compatible?
5.5 Deployment shape: single-node 8-GPU vs. multi-node scaling
- Single node, 8 GPUs: cards communicate over high-speed interconnect (such as NVLink), with ample bandwidth and controllable latency. This is the typical deployment shape for this class of model.
- Multi-node scaling: for higher concurrency or longer contexts, a hybrid strategy of tensor parallelism + expert parallelism + pipeline parallelism works, but cross-node bandwidth becomes the new bottleneck. Multi-node deployments should have RDMA-class bandwidth between nodes (100Gbps and above).
In addition, MoE’s sparse activation means every token reads 16 sets of expert weights from memory, so memory bandwidth utilization directly determines decode-phase throughput. Memory capacity determines whether it runs; memory bandwidth determines how fast it runs.
6. From “It Runs” to “It Runs Well”: An Engineering Checklist
6.1 Seven checks before launch
| Check | Key question | Reference practice |
|---|---|---|
| License compliance | Does the large-scale commercial restriction apply? | Legal review + retain license snapshot |
| VRAM capacity | Is per-GPU memory ≥ weight share + KV cache + runtime overhead? | Estimate with the method in 5.3 |
| Quantization support | Does the framework natively support 4-bit weights? | Version confirmation + small-scale load test |
| Context needs | What is the longest real input in your workload? | Analyze the distribution of actual request lengths |
| Concurrency planning | How large is the KV cache at peak concurrency? | Load testing + dynamic batching |
| Monitoring and alerting | Are memory, throughput, and latency observable? | Log service + monitoring and alerting |
| Disaster recovery | How do you degrade when a GPU fails? | Multiple replicas + circuit breaking and fallback |
6.2 The general cloud deployment path
For teams without their own data center, cloud deployment is the more realistic path and typically involves three stages:
- Weight distribution: host the 1.56TB weight files in object storage, with parallel chunked downloads and resumable transfer enabled;
- Compute supply: use high-performance GPU cloud hosts or bare-metal instances with 192GB of memory per card, purchased as whole-node (8-GPU) configurations;
- Serving and observability: orchestrate the inference service on a container platform, connect a log service and monitoring/alerting, and enable elastic scaling and self-healing.
If a business already runs on Baidu AI Cloud, the model can be deployed on top of this common set of capabilities, connected over the internal network to existing data and business systems to reduce the compliance and latency issues that come with data leaving its environment.
6.3 Cost analysis: the break-even point between self-hosting and API calls
Whether to self-host ultimately comes down to cost:
- API cost ≈ monthly average token volume × unit price;
- Self-hosted cost ≈ GPU instance cost (or hardware depreciation) + operations staff + network and storage.
A rule of thumb: once daily call volume is high and stable enough to fully utilize an integrated GPU node, the marginal cost of self-hosting is usually below pay-as-you-go. If call volume fluctuates widely and peaks diverge sharply from the average, elastic API access is more economical. For any specific company, a sensitivity analysis against the actual business curve is required.
7. Conclusion: What Trillion-Scale Parameters Tell Us
This weight release sends three signals:
- The capability frontier is expanding: trillion-scale MoE + million-token context + native multimodality is becoming the standard combination for frontier models;
- The usage threshold is dropping: open weights let enterprises deploy locally and customize deeply, but memory and licensing remain hard constraints;
- Evaluation systems are diverging: there will always be a gap between vendor-reported results and real business performance, and only a private evaluation can close it.
For developers, the rational approach is: estimate memory first, then the license, then the cost. Once those three accounts are settled, a trillion-parameter model is no longer a number that merely looks good — it becomes an engineering option you can actually deploy.
