Open-Weight 2.8T-Parameter Model: Architecture Breakdown and Local Deployment Costs
This article dissects a newly released 2.8-trillion-parameter open-weight model and translates its specs into practical engineering decisions. It explains how the MoE architecture reaches 2.8T total parameters while activating only 104B per token, why 24 layers of sparse attention change the long-context cost profile, and how MXFP4 weights shrink the model to roughly 1.56TB. It also reads the vendor-reported benchmarks with a critical eye, clarifies the gap between open weights and open-source licensing, and walks through the real local-deployment bill: VRAM per GPU, KV-cache growth at 1M-token context, decode-stage bandwidth limits, and sample sizing code. Readers will learn what to verify before downloading 1.56TB of weights and how to decide between self-hosting and cloud.
1. Open Weights: Trillion-Parameter Models Become Downloadable
On October 24, a vendor launched a model with 2.8 trillion total parameters through an API and a web interface, and committed to releasing the full weights by November 4. On November 2, the weight repository was created on a hosting platform; it remained closed during the day on November 4 and officially opened that evening. The repository contains 112 files, 96 of which are sharded weight files totaling about 1.56TB. No application is required to download it, and the weights can also be pulled through model-hosting and community platforms.
This is one of the largest publicly downloadable weight sets available today, an order of magnitude larger than most previous open-weight models. For engineering teams, “can we download it” is no longer the real question. Three questions matter more:
- What is this weight set made of, and what are its compute characteristics?
- Does the license cover our use case, and are there boundaries on commercial use?
- How much VRAM, bandwidth, storage, and operations cost does local deployment require?
This article addresses these three points.
2. Anatomy: Where the 2.8 Trillion Parameters Come From
2.1 Sparse Activation: Total Parameters ≠ Compute
The model uses a typical mixture-of-experts (MoE) architecture with fine-grained expert routing:
- Expert scale: 896 routed experts, 16 selected per token; plus 2 shared experts that every token passes through.
- Activation scale: 2.8 trillion total parameters, but only 104 billion are activated per token — an activation ratio of about 3.7%.
- Layers and context: 93 layers, a native 1M-token context window, and a vision encoder that can process images, making it a native multimodal model.
- Technical lineage: The model was continued-pretrained from the previous generation, not rebuilt from scratch.
One point is critical here: total parameters determine VRAM usage, activated parameters determine compute. For high-throughput inference, it is usually best to keep all 2.8 trillion weights resident in VRAM, because any token can be routed to any expert. If VRAM is insufficient, you can offload in tiers to host memory or NVMe, but latency and throughput will drop significantly — unsuitable for low-latency, high-concurrency scenarios.
As a result, this class of model is closer to “storage-bound” than “compute-bound.” Once downloaded, the weight footprint translates directly into VRAM pressure, while compute depends only on the 104 billion activated portion.
Fine-grained experts also suit load balancing in inference serving. Compared with a handful of “mega” experts, many small, specialized experts give the router more flexibility. When routing is uneven, strategies such as redundant experts and group routing can help — provided the inference framework supports them.
2.2 Attention Redesign: 24 Layers of Sparse Attention for Long Context
Compared with the previous generation, the architecture changed in only one place: 24 layers of full attention were replaced with sparse attention (DSA), aiming to save compute for million-token contexts.
Standard attention compute grows quadratically with sequence length. At a 1M-token input, even a single layer and a single attention head produce an attention matrix on the order of 10¹² elements. Sparse attention lets each query attend only to a carefully selected subset of historical tokens, bringing the attention portion’s cost down from quadratic to nearly linear.
This yields an easily overlooked implication: the cost of long context falls mainly on the remaining full-attention layers. So when evaluating long-document throughput, don’t just look at the advertised context limit — run real tests and watch first-token latency, long-text throughput, and KV-cache usage.
2.3 Quantization: Quantization-Aware Training, and Only This One Version
The weights use MXFP4 (4-bit float, with one 8-bit exponent scale shared across every 32 elements), and activations use MXFP8 (8-bit float). This is the product of quantization-aware training (QAT): quantization error is simulated during training or continued training, rather than compressed afterward, so accuracy loss is usually smaller.
But note carefully: the vendor released only this single quantized version — no BF16, FP8, or other precisions. This has two direct consequences:
- Inference-friendly: VRAM usage is about one quarter of BF16 weights.
- Retraining-unfriendly: there is no high-precision master. Any later fine-tuning must be done on the quantized weights, or via parameter-efficient methods such as low-rank adaptation (LoRA) layered on top of the 4-bit weights.
Deployment feasibility also depends on framework support. MXFP4 and MXFP8 are not natively supported by every inference engine, operator library, and hardware platform, and implementations of quantized weights, activation quantization, and expert routing differ across versions. Before going live, confirm whether the target framework supports the quantization format and the corresponding MoE operators, and whether there is any performance fallback.
3. How to Read the Numbers: Vendor-Reported Results Need Three Lenses
The vendor’s reported results are below. Note that all of the following figures come from the vendor’s public materials or model card and were not independently re-tested in this article. Because the source material did not provide complete verifiable links, and details such as the evaluation framework version, prompts, and sampling parameters were not fully disclosed, treat these uniformly as vendor materials rather than third-party measured conclusions.
| Evaluation dimension | This model | International flagship A | Another flagship B | Note |
|---|---|---|---|---|
| Terminal coding benchmark 4.0 | 65.7% | 66.4% | 58.2% | Vendor material, not independently re-tested |
| Long-horizon coding task set | 51.7% | 46.3% | — | Vendor material, not independently re-tested |
| Scientific terminal benchmark | 38.1% | 63.3% | — | Vendor material, not independently re-tested |
| Hard problem set (Diamond tier) | 39.4% | 60.9% | — | Vendor material, not independently re-tested |
The vendor claims the model leads flagship A on 16 of 37 benchmarks. Another claim states that a certain independent evaluator’s intelligence index ranks it fourth globally. Since this article did not obtain the full text of a verifiable independent report, this ranking is only a lead and should not be used directly as a selection criterion.
First, capability distribution is uneven. Coding and agentic tasks are on par with or ahead of the international flagship, while science and some hard-reasoning tasks show a clear gap — more than 20 percentage points between the two dimensions. Choose a model based on targeted evaluation of your own tasks, not on an overall ranking.
Second, self-reported results carry systematic bias. Prompts, sampling parameters, and evaluation framework versions all affect scores, so cross-vendor numbers should not be compared directly. Teams whose task shape resembles terminal coding can reference its coding capability; if your business is mainly scientific reasoning, re-evaluate from scratch.
Third, independent evaluations and vendor figures may disagree. The value of independent evaluation is a unified, reproducible method, but it too covers only limited dimensions. The truly reliable approach is to run a small-scale validation with your own business samples, your own prompts, and your own hardware.
Additionally, the model card is dated October 24. Whether inference engines and operator libraries have finished adapting to this new structure determines how close community experience can get to the official numbers — and this directly affects framework choice at deployment time.
4. Two Barriers: Licensing and Compute
4.1 Open Weights ≠ Open-Source License
Before downloading, the first step is to read the model card and license. Models at this scale typically use a custom license rather than an OSI-approved open-source license. Common constraints include:
- Commercial services above a certain monthly active user scale require separate authorization;
- Derivatives must be named in a prescribed way;
- Use is bound by an acceptable use policy;
- Terms such as attribution and share-alike may apply.
The specific constraints are governed by the model card and license published with this weight repository. To judge whether commercial use is allowed, check three things: whether your business scale crosses the licensing threshold; whether derived models meet the naming obligations; and whether your distribution form is compliant. When in doubt, have legal review it before integration.
4.2 Open Weights ≠ Reproducible Training
These releases usually include only weights, configuration, and the necessary tokenizer files — not training data, training code, or the training recipe. This means:
- You cannot audit the data sources;
- You cannot continue pretraining on your own data;
- If a copyright or compliance dispute arises, there is no source-level basis for tracing.
For industries with strict data-compliance requirements, this matters more than the license terms.
4.3 Quantized Weights Raise the Cost of Retraining
Because there is no high-precision version, there are two common approaches:
- Fine-tune by layering low-rank adapters such as LoRA on top of the 4-bit quantized weights;
- Dequantize to high precision first, then train all parameters — but this requires loading the full ~1.56TB of weights, multiplying both VRAM and storage costs.
So if your goal is deep secondary training rather than inference serving, evaluate in advance whether you can accept the training-path limits imposed by quantized weights.
5. Local Deployment Cost: The VRAM Bill for 1.56TB of Weights
5.1 VRAM: ~195GB per Card Across 8 GPUs
With 1.56TB of weights across 8 accelerators, a rough estimate is about 195GB per card:
1.56 TB ÷ 8 = 0.195 TB ≈ 195 GB/card
This is pure weight usage only — it does not yet account for framework overhead, activations, temporary buffers, and the KV cache. With 8 cards at 192GB each, total VRAM is about 1536GB, which is below the 1560GB of weights, so even the weights alone barely fit. It is therefore better to plan around 256GB-class per-card VRAM, leaving room for the KV cache and framework overhead.
If you use more common 141GB-class accelerators, fitting the weights alone needs about 12 cards. If your business requires million-level long context or multiple concurrent streams, plan for 16 cards or more.
The real meaning of “8 cards as a starting point” is the hardware floor of a single node with 8 high-VRAM accelerators. If per-card VRAM is only 192GB, 8 cards are not enough to reliably hold 1.56TB of weights plus long-context KV cache.
5.2 Context: KV Cache Pressure at 1M Tokens
The KV cache grows linearly with context length. A common estimation formula is:
KV bytes per token ≈ 2 × layers × KV heads × head dim × bytes per elementTotal KV cache ≈ KV bytes per token × sequence length × concurrency
Here, “2” accounts for the Key and Value parts. For example, with 93 layers, combined KV heads and head dim of 1024, and BF16 at 2 bytes per element:
2 × 93 × 1024 × 2 ≈ 380,928 bytes/token ≈ 0.36 MB/token1M tokens ≈ 360GB
With more KV heads, larger head dimensions, or multiple concurrent requests, the KV cache rises substantially. This model’s sparse attention covers only 24 layers; the cache cost of the remaining full-attention layers is still considerable.
So “supports 1M tokens” does not mean “1M tokens is free.” Long-context scenarios must budget for the KV cache as well.
5.3 Compute: Decode Stage Is Bound by Memory Bandwidth
During autoregressive decoding of an MoE model, each token only needs to read the weights of its activated experts, but the volume is still large. A rough estimate (for reference only):
104B activated params × 0.5 bytes (4-bit) ≈ 52 GB/token52 GB ÷ 1.56 TB ≈ reading the entire weight set once roughly every ~30 generated tokens
In addition, MoE expert routing naturally forms a “dispatch–compute–combine” pipeline with frequent cross-card communication, so interconnect bandwidth is tightly coupled to decode speed. With the same 8 cards, a machine with high-speed interconnect will feel very different from one linked only by a general-purpose network.
5.4 Hardware, Concurrency, and Throughput Estimates
Below is illustrative code to estimate VRAM and concurrency under a weights-only deployment. All values are illustrative; rely on real framework measurements.
def estimate(gpu_memory_gb=256, gpu_count=8,weight_gb=1560, reserve_ratio=0.1,kv_per_token_mb=1.0):"""Rough estimate of max total concurrent context under a weights-only deployment.All parameters are illustrative and do not represent any specific model's real metrics.reserve_ratio: reserve fraction for framework, activations, temp buffers, etc.; 10% here for illustration.kv_per_token_mb: rough per-token KV cache summed across all layers and KV heads;1MB here for illustration; recompute from the model card and framework implementation for real values."""total = gpu_memory_gb * gpu_countweights = weight_gbreserve = total * reserve_ratiokv_budget_gb = total - weights - reserveprint(f"Total VRAM: {total} GB, weights: {weights} GB, reserve: {reserve:.1f} GB, KV budget: {kv_budget_gb:.1f} GB")if kv_budget_gb <= 0:print("Insufficient VRAM: cannot fit weights and reserve")return 0max_tokens = kv_budget_gb * 1024 / kv_per_token_mbprint(f"Concurrent context limit: ~ {int(max_tokens):,} tokens")return max_tokens# Illustrative: 8×256GB, 1.56TB weights, ~1MB KV cache per tokenestimate()# 8×192GB: insufficient VRAM, cannot fit pure weights and reserveestimate(gpu_memory_gb=192, gpu_count=8)# 16×192GB: sufficient VRAM, can support longer contextestimate(gpu_memory_gb=192, gpu_count=16)
These numbers reveal a counterintuitive conclusion:
- 1536GB of VRAM is below the 1560GB of weights; 8×192GB cards can barely fit the pure weights, so “8 cards as a starting point” only holds when per-card VRAM exceeds 192GB — ideally 256GB-class.
- With 8×256GB cards, after subtracting weights and reserve, KV-cache headroom is still limited; truly supporting 1M-token context usually requires 16×192GB cards or higher.
- The hardware floor is not the experience floor. Whether it runs depends on VRAM; whether it serves reliably depends on the KV cache, interconnect, framework, and concurrency strategy.
For budgeting, the pseudocode below runs the three gates — “license, VRAM, throughput” — in one pass:
# Pre-deployment check (pseudocode, flow only)def preflight(model_card, target):# 1) License: is business scale, derivative naming, and distribution form compliant?assert license_compatible(model_card.license, target.business)# 2) VRAM: weights + KV cache + reserve < available cluster VRAMneed = weight_footprint(model_card) + kv_footprint(target.max_context, target.concurrency)assert need < target.cluster_gpu_memory * (1 - RESERVE)# 3) Throughput: stress-test with real interconnect and parallelism, confirm SLO is metbench = run_smoke_test(model_card, target, prompt_set=REAL_TRAFFIC_SAMPLE)assert bench.p95_ttft < target.ttft_slo and bench.tokens_per_sec >= target.tps_min
6. Turning “Open Weights” into Real Value: Four Practical Recommendations
Choose the right form. The value of fully open weights is control: data can stay local, inference behavior is auditable, and the business can run customized evaluation and fine-tuning for its own scenarios. The prerequisite is that licensing, compute, and operations capabilities are all in place. When conditions are insufficient, validating business value through a cloud API first is the lower-cost first step.
Serve on elastic cloud infrastructure. Self-hosting needs a whole node with 8 high-VRAM cards, and idle costs are high when utilization is low. If you choose to run it through a cloud provider, you can combine general capabilities:
- Use elastic GPU resources to serve inference;
- Use object storage to store and distribute weight shards;
- Use a container platform to orchestrate and autoscale the inference service;
- Use logging services and monitoring alerts to track first-token latency, throughput, and VRAM usage.
Set up the download and distribution pipeline. Downloading 1.56TB of weights over standard single-threaded HTTPS takes about a day and a half even at a saturated 100Mbps; multi-threaded segmented downloads usually cut this by an order of magnitude. Tools such as aria2c support multiple connections, resumable downloads, and segmented downloads:
# Illustrative commands: 8 connections, 16 segments, auto resume# Replace the URL with the actual weight file addressaria2c -x 8 -s 16 -k 16M -c -d ./weights -o shard_001.part <weight-file-url># If the repository provides a shard manifest, download them one by onearia2c -x 8 -s 16 -k 16M -c -d ./weights -i weight_urls.txt# Verify each file after downloadsha256sum -c checksums.txt
If the repository is already split into 96 shards, download and verify them shard by shard, segment by segment. After downloading, always verify each file against the checksum values published with the repository to avoid load failures from corrupted shards. Also reserve at least twice the weight footprint in storage — one copy for the download cache and one for the unpacked replica.
Run scenario-based evaluation. Of the 37 vendor-reported benchmarks, the 16 leading ones suggest the advantage is likely concentrated in coding and agentic scenarios. Before going live, run a small-scale evaluation with your own business samples, emphasizing long context, multimodality, and tool calling, then decide whether to commit to a full-card budget.
7. Conclusion
This weight release marks a new leap in the scale of open-weight models, but between “downloadable” and “actually usable” lie three gates: licensing, VRAM, and operations. The 2.8-trillion-total-parameter, 104-billion-activated-per-token MoE design keeps compute low through sparse activation; MXFP4 weights compress the footprint to 1.56TB, which also makes it one of the highest-barrier open-weight models to deploy locally today.
For most teams, the safer path is to assess licensing and scenarios first, then decide between self-hosting and cloud. Before downloading 1.56TB of weights, complete at least three checks: confirm the license covers your business, confirm the VRAM and KV-cache budget is sufficient, and confirm the inference framework supports the quantization format and MoE structure. Only when all three pass can a trillion-parameter model truly turn into business value.
