2.8T-Parameter MoE Model Weights Released: Architecture, License, and Deployment Guide

    A model team has opened full downloadable weights for what it calls the strongest open model currently available: 2.8T total parameters, roughly 1.56TB of files, and local inference that needs at least 8 GPUs to start. This article sets aside the "strongest" claim and focuses on the engineering questions that actually decide adoption: how the model is built, how its custom license defines the boundaries of permitted use, what local deployment costs in hardware and operations, and how to evaluate its capabilities objectively instead of chasing a leaderboard rank. It walks through the two-step weight release, the MoE sparse architecture and sparse attention, the 4-bit weight plus 8-bit activation precision path, how to download and verify the shards, the license's commercial thresholds, the VRAM and interconnect budget, and a three-role action checklist.

    1. Release Recap: A Two-Step Weight Rollout

    In late October, an AI model team first launched a new flagship model, offering both an API and a web interface, and promised to release the full weights in early November. On the evening of November 4, the weights went live for download with no application required. The repository contains 112 files, 96 of which are tensor shards totaling about 1.56TB.

    A few points are worth noting:

    • Record size: This is the largest open-weight model released to date, far exceeding any previously public model.
    • Clear release cadence: First validate load through a hosted service, then open the weights to serve private-deployment needs—a common path for leading model teams.
    • Weights ≠ capability: Open weights mean you can download locally, inspect, and fine-tune—not just call an API. But between “can download” and “can run” lies a whole set of hardware and engineering costs.

    2. Where 2.8T Comes From: Reading the MoE Sparse Architecture

    Total vs. active parameters: two numbers, two cost types

    The model is a natively multimodal agent model, continued from the previous generation, built on a Mixture-of-Experts (MoE) architecture. It has 2.8T total parameters, but each token activates only 104B—about 3.7%.

    A simple formula makes the MoE structure clear:

    1. Compute per token ≈ 2 × active parameters ≈ 2 × 104B ≈ 208 GFLOPs (forward)
    2. VRAM and storage needs ← determined by total parameters
    3. Compute and decode speed ← determined by active parameters

    For developers, this structure means directly:

    • Weights must be fully in place—you cannot “load only what you use.”
    • Compute is only about 3.7% of what a same-size dense model would require.
    • The VRAM threshold is high, but throughput potential is large.

    896 routed experts + 2 shared experts

    The model contains 896 routed experts, and each token selects 16 of them, plus 2 shared experts that are always active for every token. In other words, each token actually passes through 18 experts.

    • Shared experts carry general capability, similar to a dense model’s base layer.
    • Routed experts provide specialization, with a gating network deciding which expert handles each token.
    • The average probability of a single routed expert being selected by any given token is only 16/896 ≈ 1.8%.

    This design has two implications for deployment:

    • Experts are naturally suited to expert parallelism (EP): different experts sit on different accelerators, and each card only needs to hold a subset of them.
    • Routing is hard to keep perfectly balanced, and load skew slows down the whole batch of inference, so capacity planning must leave headroom.

    Native multimodality and million-token context

    The model uses a native multimodal design: a vision encoder (a vision encoding structure designed for language models) converts images into visual tokens that are fed into the backbone alongside text tokens, rather than bolting on a separate image-text module. Context length reaches 1 million tokens—enough to hold long documents, medium-sized codebases, or long-form audio/video transcripts. The vision encoding part typically does not participate in 4-bit quantization, and it is a small piece that must be counted in the size breakdown later.

    Agent orientation: mutually reinforcing with sparse activation

    With its agent training objective, the model targets task chains like “view a screen or document → plan steps → call tools → adjust based on results.” This kind of workload consumes far more tokens than ordinary Q&A:

    • Every turn must carry the system prompt, tool descriptions, and the history of the trajectory.
    • Tasks require multiple iterations.
    • Images and long documents keep entering the context.

    The low activation ratio happens to ease this problem. Most of the compute in agent scenarios occurs during token-by-token decoding, and MoE makes each step compute only about 104B parameters, keeping per-call cost more manageable.

    3. The Only Structural Change: Sparse Attention

    Compared with the previous generation, only one structural change was made: full attention in some layers (24 layers in total) was replaced with sparse attention. The remaining layers keep full attention, forming a “hybrid attention” layout.

    Why the change was necessary

    Under full attention, every token interacts pairwise with all preceding tokens. Let the context length be L:

    • The attention cost of a single layer for a single token scales with L.
    • The total cost over the whole sequence scales with L².

    With a rough estimate based on a common multi-head attention implementation, 24 layers of full attention at a 1-million-token context account for about 27% of the entire forward pass compute. The 5.7G and 104B here are illustrative model sizes, used only to convey the order of magnitude.

    If all 93 layers used full attention, converted at the same model size, this ratio would approach 65%. After sparsifying 24 layers, that portion of overhead is compressed to less than 40% of the original.

    The general idea of sparse attention and how to validate it

    The core of such methods is to let each query interact only with a “most relevant” subset of keys and values. Common strategies include:

    • Local windows: attend to neighboring tokens.
    • Global/summary tokens: keep a small number of globally accessible key tokens.
    • Routing- or clustering-based selection: dynamically pick tokens by semantic relevance.

    The cost is that some precise retrieval of long-range dependencies may degrade. So these schemes typically rely on having “seen” the sparse pattern during training to compensate for the accuracy loss. For users, what matters is validating with task-specific testing rather than trusting the architecture description alone.

    4. Precision Path: 4-Bit Weights + 8-Bit Activations, and Only One Version

    The team released only this single precision version; there are no higher-precision weights.

    • Weight format: 4-bit microscaling floating point (MXFP4), where 32 elements share one 8-bit scale exponent.
    • Activation format: 8-bit floating point (MXFP8).
    • Training method: Quantization-Aware Training (QAT), which simulates quantization error during training.

    Size self-consistency: where 1.56TB comes from

    A simple bit of arithmetic cross-checks the number:

    1. Bare 4-bit: 2.8T × 0.5 bytes ≈ 1.40 TB
    2. Add scale factors: 4.25 bit/param ≈ 1.49 TB
    3. Plus vision encoder, tokenizer, and a few non-quantized tensors ≈ 1.56 TB

    1.56TB matches “2.8T parameters + 4-bit quantization” closely. It also means the file size is essentially the theoretical floor, with no obvious redundancy.

    QAT vs. PTQ

    • Post-Training Quantization (PTQ): after training finishes, use a small amount of calibration data to determine quantization parameters. It is easy to apply, but at extremely low bits like 4-bit, fine-grained errors tend to be amplified.
    • QAT: quantization error is folded into the optimization objective during training, letting weights “adapt” to a low-bit representation. Accuracy loss is smaller, at the cost of extra training effort.

    Therefore, even if the deployment side does its own 4-bit quantization, it is hard to beat the official QAT version in accuracy. Conversely, QAT also means these weights are hard to convert losslessly to higher precision, or to compress further to lower bits.

    Implications of a single precision version

    • Users cannot pick a precision tier; they must accept the accuracy tradeoff that comes with 4-bit weights.
    • At inference, 4-bit weights are usually dequantized to 8-bit for compute. Actual performance depends heavily on how natively the hardware and software support the microscaling format.
    • If the weights can only be loaded and then uniformly converted to 16-bit float for compute, their footprint balloons to about 5.6TB, far exceeding a single multi-GPU node’s capacity, and the deployment plan changes accordingly.

    5. The Weight Files: Getting and Verifying 96 Shards Across 1.56TB

    The repository contains 112 files, 96 of which are tensor shards; the rest are configuration and tokenizer-related files. The reason for sharding is practical: a single file that is too large is awkward to download and load in parallel, whereas splitting it makes multi-threaded downloads, resumable transfers, and distributing tensors across different accelerators easier.

    A checklist for downloading and verifying:

    1. Verify integrity first: the weight shards use a plain tensor binary format (such as safetensors), which does not execute deserialization code at load time—safer than older weight files based on serialization frameworks. After downloading, first confirm the shards’ total byte count matches the index records.
    2. Reserve enough disk: the download process typically needs 1.5–2× the weight size in free space; for a 1.56TB model, plan for 3TB or more.
    3. Check configuration completeness: the tokenizer, chat template, and vision encoder config are all required; missing any will cause inference errors.
    4. Mirror multiple copies: keep multiple replicas in object storage for distribution across clusters and for easy rollback.

    The following example is written against a common tensor-shard format and is for illustration only:

    1. import hashlib
    2. import json
    3. import os
    4. INDEX = "model.safetensors.index.json" # shard index file
    5. with open(INDEX, encoding="utf-8") as f:
    6. index = json.load(f)
    7. # Collect and de-duplicate all shard filenames from the index
    8. shards = sorted(set(index["weight_map"].values()))
    9. expected = index["metadata"]["total_size"]
    10. # Step 1: verify the total shard byte count to quickly catch truncation or missing files
    11. actual = sum(os.path.getsize(p) for p in shards)
    12. assert actual == expected, f"Shard total byte count mismatch: {actual} != {expected}"
    13. def sha256(path, chunk_size=8 << 20):
    14. h = hashlib.sha256()
    15. with open(path, "rb") as fp:
    16. for block in iter(lambda: fp.read(chunk_size), b""):
    17. h.update(block)
    18. return h.hexdigest()
    19. # Step 2: compute a digest per shard and compare against publisher-provided checksums
    20. for p in shards:
    21. print(p, sha256(p))

    6. License Analysis: Open Weights ≠ Open Source

    This is the point most in need of a separate explanation. The repository carries a custom license (labeled “other” on the platform), not a standard open-source license. According to the team, this license is word-for-word identical to the previous generation’s, only renamed, with the same usage terms carried over.

    Granting party: permissive, commercial use allowed

    The license is modeled on a permissive license and grants the rights to use, copy, modify, distribute, fine-tune, and commercially use the model for free.

    Two additional restrictions: aimed only at large-scale external business

    • Offering the model as a service externally: enterprises whose cumulative revenue exceeds $20M over 12 months must sign a separate agreement before commercial use.
    • Commercial product scale: when embedding into a commercial product that reaches a certain monthly active user (MAU) scale, a separate negotiation is required (see the license text for the exact threshold).
    Use case Extra authorization needed?
    Academic research, internal evaluation, personal experimentation Usually not
    Internal enterprise use, not offered as a service Usually not
    Embedded in a commercial product below the revenue/MAU thresholds Usually not
    Offering model-as-a-service with >$20M cumulative revenue over 12 months Separate agreement required
    Commercial product reaches the agreed MAU threshold Separate negotiation required

    Teams whose revenue and MAU are both close to the thresholds are advised to do a compliance assessment early. The license terms follow the latest official text.

    “Open source” and “open weights” are not the same thing

    • Open source: represented by the Open Source Definition maintained by the Open Source Initiative (OSI), it typically requires no field-of-use restrictions and no additional conditions.
    • Open weights: the weights can be downloaded, but the license may carry usage terms.

    From this angle, a custom license with revenue and MAU thresholds attached qualifies as “open weights,” not open source in the strict sense. For teams preparing to go commercial, this boundary must go into the compliance review checklist.

    7. Deployment Cost: How Much Hardware to Run It Locally

    The VRAM math: a four-part budget

    VRAM component Determined by Notes
    Weights themselves total params × bytes per param ~1.56TB, split across accelerators
    Activations and temporary buffers batch size, sequence length grow with concurrency
    KV cache context length × concurrency non-trivial at million-token context
    Communication and runtime overhead parallel strategy, framework version comm buffers, VRAM fragmentation, etc.

    Take a single 8-GPU node as an example: weights alone require about 200GB per card:

    1. 1.56 TB ÷ 8 ≈ 195 GB/card

    That still leaves out activations, KV cache, and communication buffers. So the precise meaning of “8 GPUs to start” is that each card needs on the order of 200GB of VRAM, and you must scale to 8 cards before it even fits.

    Why you can’t use too few cards

    Bound by the per-card VRAM ceiling, using a few cards fewer leaves only two options:

    • Use accelerators with larger per-card VRAM.
    • Offload weights to host memory or storage, which sharply degrades performance.

    More cards mean finer parallel splitting and less weight pressure per card, but inter-card communication overhead and scheduling complexity rise as well.

    Interconnect: the hidden bottleneck of MoE deployment

    The core of expert parallelism is All-to-All communication: each token is routed to experts spread across different cards, and the results are sent back. Communication volume grows with batch size and sequence length. If cards are connected only by a general-purpose bus rather than a dedicated high-speed interconnect, expert parallelism easily gets bottlenecked on communication. So for multi-node deployment, pay special attention to intra-node interconnect bandwidth and inter-node networking.

    VRAM estimation code

    1. # Rough planning-stage estimate only; actual usage should come from the inference framework's memory analysis
    2. WEIGHT_BYTES = 1.56 * 1024 ** 4 # ~1.56TB of weights
    3. KV_BYTES_PER_TOKEN = 256 * 1024 # illustrative value: KV cache bytes per token; calibrate to your actual config
    4. def min_gpu_memory(num_gpus: int, seq_len: int, concurrency: int) -> float:
    5. """Return the minimum per-GPU VRAM requirement (GiB)"""
    6. kv_total = seq_len * concurrency * KV_BYTES_PER_TOKEN
    7. per_gpu = (WEIGHT_BYTES * 1.1 + kv_total) / num_gpus # 1.1 is the runtime overhead factor
    8. return per_gpu / 1024 ** 3
    9. print(min_gpu_memory(num_gpus=8, seq_len=32_768, concurrency=4))

    Choosing a deployment mode

    • Local deployment: data stays inside the network and can be deeply customized and fine-tuned. The prerequisite is large-VRAM accelerators, high-speed interconnect, and professional operations capability.
    • Cloud deployment: provision GPU resources on demand via cloud instances or containerized inference platforms, avoiding procurement and operations burdens—well suited to validation and bursty traffic.
    • Hybrid mode: handle sensitive data locally, and offload peak traffic or complex tasks to the cloud.

    The architectural gains aren’t a free lunch

    Sparse attention reduces compute on long sequences, but the inference framework must implement the corresponding sparse attention operators. If the framework falls back to dense attention compute, the compute load rises sharply, and million-token context also brings a massive KV cache—VRAM runs out first. Be sure to confirm the inference framework supports this structure before deploying.

    8. Capability Evaluation: How to Rationally View the “Strongest Open Model”

    The team’s comparison data is self-reported: of 37 benchmarks, 16 outperform the flagship closed model used as the reference.

    • Where it leads: overall ahead on coding and agent tasks. Terminal-based automated coding scored slightly below the reference model, but it clearly leads on tests aimed at long-horizon software tasks.
    • Where it lags: a clear gap on some science benchmarks and some high-difficulty reasoning tests.

    Separately, third-party evaluation data ranks its comprehensive intelligence index fourth globally. Overall, it is a “specialist” model: strong on coding and agents, weaker on science and some reasoning.

    The team recommends a four-step approach to judgment:

    1. Treat self-test data only as a reference: between public leaderboards and real deployment, there are always differences in prompt design, decoding parameters, and evaluation scripts.
    2. Look at task distribution: the closer the evaluation set is to your own business, the more reliable the conclusion.
    3. Separate “intelligence” from “engineering usability”: latency, concurrency, stability, context length, image input support, and tool-calling format often decide whether it can go live more than the rank does.
    4. Model the unit economics: fold throughput, latency, and license constraints into a per-task cost, then compare it against the API-calling approach.

    Illustrative evaluation code:

    1. import statistics
    2. import time
    3. tasks = load_tasks("tasks.jsonl") # each entry has fields like prompt and scoring criteria
    4. def evaluate(model, tasks):
    5. results = []
    6. for task in tasks:
    7. start = time.perf_counter()
    8. output = model.generate(task["prompt"])
    9. latency = time.perf_counter() - start
    10. results.append(
    11. {
    12. "score": task["score"](output), # rule-based check or model-graded review
    13. "latency": latency,
    14. }
    15. )
    16. latencies = sorted(r["latency"] for r in results)
    17. return {
    18. "pass_rate": sum(r["score"] > 0 for r in results) / len(results),
    19. "p50_ms": latencies[len(latencies) // 2],
    20. "p95_ms": latencies[int(len(latencies) * 0.95)],
    21. }

    9. Action Checklist: Three Roles, Three Paths

    Individual developers and researchers

    • Ask three questions first: Can per-card VRAM reach the ~200GB level? Can disk spare 3TB or more? Is there a high-speed interconnect between cards?
    • If all three are no, validate through API calls or cloud resources first, then decide whether to invest in hardware.

    Enterprise technical teams

    • Clear the license gate before deploying: confirm whether revenue and MAU touch the additional terms.
    • During deployment, first confirm the framework’s support for sparse attention and the microscaling quantization format, then plan VRAM and concurrency.
    • At go-live, evaluate and account for cost using your own business task set.

    Platform and operations teams

    • Manage weight versions and distribution with object storage.
    • Deliver inference services on a container platform to enable elastic scaling.
    • Track latency and resource utilization with logging and monitoring/alerting systems.

    Conclusion

    The significance of this weight release is not the title of “strongest open model,” but that it puts the complete design of a trillion-parameter-scale MoE model in front of developers. Architecture, precision, license, and hardware—each is an inspectable engineering variable. For a team, the rational approach is not to chase leaderboard rankings, but to drop the model into its own tasks, hardware, and compliance framework and run a full validation. In this sense, the general-purpose capabilities Baidu AI Cloud offers—cloud instances, containerized inference, object storage, and logging and monitoring—are well suited to carrying such an end-to-end validation process, from download and verification through to production launch.