Open 2.8T-Parameter Weights: What 1.56TB and the 8-GPU Minimum Really Mean

    This article breaks down a newly released open-weight model with 2.8T total parameters (104B activated per token) and a 1.56TB weight footprint. It covers the 11-day path from API launch to weight release, the sparse MoE architecture and attention changes behind the 1M-token context, why the 4-bit MXFP4 quantization is the only official version, how to read the vendor's self-reported benchmarks, and the custom license's scale-based thresholds. For engineering teams, the core takeaway is that this release raises both the technical and commercial bar, and the real questions are VRAM headroom, parallel topology, and whether you cross the license thresholds—not whether you can download it.

    1. Timeline: From API to Open Weights in Eleven Days

    A model vendor first launched a new-generation large model on October 24 via API access and a web interface, and committed to publishing the full weights before November 4. On November 4, the weights appeared on a major international open-weight hosting platform as promised—the repository required no application and could be pulled directly.

    A few details of the release format are worth attention from an engineering perspective:

    • The repository was created on November 2, remained closed on the morning of the 4th, and officially opened in the evening—a typical “reserve the address, release on a schedule” pattern;
    • The weights were packaged into 112 files, 96 of which are tensor shards, with companion index files describing the mapping between shards and parameter names;
    • The total size is roughly 1.56TB, several times that of any previously open-weight model;
    • The vendor provides only a single quantized version, with no higher-precision alternative weights.

    The significance here is not “another large model went open,” but that an open-weight model’s scale has crossed the threshold of the single-machine, single-GPU era for the first time: it now falls into a range that must be treated as distributed systems engineering.

    2. Architecture: The 2.8T-Total / 104B-Activated Trade-off

    2.1 Sparse Expert Structure: Trading Capacity for Compute

    The model is a natively multimodal agent model, continued from the previous generation, and is built around a fine-grained mixture-of-experts (MoE) core:

    Dimension Value Engineering implication
    Total parameters 2.8T Sets the VRAM floor and weight size
    Activated parameters per token 104B Sets per-step compute and latency
    Routed experts 896 Fine-grained split, boosts combinatorial expressiveness
    Experts selected per token 16 ~1.8% sparse activation ratio; compute cost far below a dense model
    Shared experts 2 Carry general capabilities, avoiding redundant learning of common knowledge
    Layers 93 Deep network, stronger long-range dependency modeling
    Context length 1M tokens Prerequisite for long-document and repo-level code tasks

    The point most easily misread is that 2.8T and 104B are not contradictory. The sparse structure decouples “knowledge capacity” from “compute volume”—all parameters must stay resident in VRAM, but each token passes through only about 3.7% of the weights. So deployment cost for this class of model is driven mainly by VRAM capacity, while throughput depends on activation scale and routing efficiency. The two must be evaluated separately; you cannot underestimate the hardware bar just because “only 104B is activated.”

    2.2 Attention Changes: Compute Throttling for Long Context

    Relative to the previous generation, the structural change is concentrated in one place: of the 93 layers, 24 switch from standard full attention to a sparse attention mechanism (DSA). Full attention’s compute and memory overhead grow quadratically with sequence length and become the primary bottleneck at a 1M-token context; sparse attention limits the key-value range each token actually attends to, compressing this overhead back toward near-linear.

    Note that this is a hybrid design of “some layers sparse, some layers full attention.” Full-attention layers handle global information aggregation, while sparse layers save compute on ultra-long sequences, and the two are stacked alternately. The specific layer distribution and sparse pattern should be taken from the official configuration, and should not be inferred backward from the layer-count ratio alone.

    2.3 Native Multimodal

    On the vision side, a self-developed ViT-family encoder connects into the main network—a native multimodal design rather than a bolted-on splice. The benefit of the native approach is that vision tokens and text tokens share the same attention and expert routing, so mixed image-text long-context inference needs no cross-module transfer of intermediate representations; the cost is that both training and inference pipelines are heavier, and vision input significantly increases prefill-stage latency.

    2.4 Quantization: 4-bit Weights Are the Only Official Version

    The weights use a 4-bit micro-scaling float format (MXFP4), with 8-bit activations (MXFP8). This is the product of quantization-aware training (QAT), not brute-force post-training quantization applied after training completes. QAT lets the model adapt to low-precision numerical distributions during training, so accuracy loss is typically much smaller than post-hoc quantization at the same compression ratio.

    But engineering must accept three constraints:

    1. No fallback: The vendor ships only this one quantized set. To run BF16 or FP8, you must either dequantize or retrain yourself—the former loses the compression benefit, and the latter carries very high cost and risk;
    2. Hardware fit: Low-precision micro-scaling formats only accelerate on compute units that natively support them; otherwise they must be dequantized to higher precision before computation, which raises VRAM use and lowers throughput. Support must be verified per hardware model;
    3. Fine-tuning risk: Full-parameter fine-tuning on 4-bit weights tends to break the existing quantized numerical layout, so parameter-efficient methods like low-rank adaptation are usually the only option, and they require small step sizes and strict validation.

    The configuration structure can be sketched as follows (field names are illustrative only and do not represent official keys):

    1. {
    2. "num_hidden_layers": 93,
    3. "sparse_attention_layers": 24,
    4. "max_position_embeddings": 1048576,
    5. "moe": {
    6. "num_experts": 896,
    7. "num_experts_per_token": 16,
    8. "num_shared_experts": 2,
    9. "total_params": "2.8T",
    10. "activated_params_per_token": "104B"
    11. },
    12. "quantization": {
    13. "weight_format": "mxfp4",
    14. "activation_format": "mxfp8",
    15. "training_aware": true
    16. },
    17. "modality": { "vision_encoder": "vit-family" }
    18. }

    3. Benchmarks: How to Read Vendor Self-Reported Numbers

    The vendor’s entire comparison table is self-reported, and two international closed-source flagship models are mixed in on the same table as references. Once organized, a clear strong/weak distribution emerges:

    Benchmark dimension This model Closed-source flagship A Closed-source flagship B
    Terminal coding tasks 65.7% 66.4% 58.2%
    Long-horizon software engineering 51.7% 46.3% —
    Scientific terminal tasks 38.1% 63.3% —
    Hard academic reasoning 39.4% 60.9% —

    The vendor’s framing is that this model exceeds the reference models on 16 of 37 benchmarks. Checking table by table, the conclusion is not “leading across the board” but strong on coding and agent tasks, weak on science and some reasoning tasks. A separate independent third-party evaluation agency gave it an intelligence index of 55, ranking fourth globally.

    Three points to keep in mind when reading this kind of data:

    • Self-reported benchmarks carry selection bias. Vendors tend to disclose task families where they perform well; judging real capability requires looking at the full task distribution, not a single peak.
    • Agent-type and knowledge-type benchmarks cannot be extrapolated to each other. The former tests multi-turn tool calling and long-horizon planning; the latter tests knowledge density and reasoning rigor. They emphasize different training goals, and the scores are not comparable.
    • Benchmark contamination risk. Open weights mean anyone can inspect whether the training data covers the evaluation sets. When reproducing, use your own or a private evaluation set, and fix the decoding parameters (temperature, top-p, max output length), while reporting both pass@1 and stability across multiple samples.

    The engineering recommendation is to treat this model as a candidate with a clearly skewed capability profile, run a small-scale A/B test on your own business set, and not copy the leaderboard ranking when making selection decisions.

    4. License: Open Weights Are Not Open Source

    This is the most important point of this release to examine separately. The repository carries a custom license, and the license tag on the hosting platform is marked “other,” rather than a standard open-source license like MIT or Apache. This license is word-for-word identical to the previous generation’s, with only the name changed.

    The terms can be split into two layers:

    The first layer is an MIT-style core: It permits free use, copy, modify, distribute, fine-tune, and commercial use, with no restrictions on user identity or research purpose, and no field-level additional restrictions.

    The second layer is two scale-threshold restrictions, targeting only large-volume external business:

    1. A company that offers the model as a service (MaaS) and whose cumulative revenue over any 12 consecutive months exceeds USD 20 million must sign a separate agreement with the model owner before commercial use;
    2. A commercial product whose monthly active users exceed a certain high threshold likewise triggers an additional licensing requirement (the original information is truncated here; the specific figure must be confirmed against the license text).

    The essence of this wording is “friendly to small and mid-scale, reclaiming negotiating power from ultra-large scale.” It does not violate the inclusive spirit of open source, but it does not constitute a standard open-source license, and enterprises cannot clear it directly along the Apache-2.0 path during compliance review.

    The decision logic can be written as a minimal function:

    1. # Always verify the numbers and thresholds against the official license text; this only illustrates the decision structure
    2. MaaS_REVENUE_THRESHOLD_USD = 20_000_000 # MaaS revenue over 12 consecutive months
    3. MAU_THRESHOLD = None # truncated in the source; verify yourself
    4. def needs_separate_agreement(maas_revenue_12m_usd: float,
    5. product_mau: int) -> bool:
    6. hits_revenue = maas_revenue_12m_usd > MaaS_REVENUE_THRESHOLD_USD
    7. hits_mau = MAU_THRESHOLD is not None and product_mau > MAU_THRESHOLD
    8. return hits_revenue or hits_mau

    Compliance checklist for implementation:

    • Clarify your own form: internal tool, external API service, or embedded commercial product—the three fall under different terms;
    • Establish a reconciliation mechanism for revenue and MAU figures; threshold decisions require traceable data sources, not estimates;
    • Every link in the distribution chain (including secondary distribution and mirror hosting) must carry forward the original license; custom licenses typically require retaining the notice;
    • Trademarks and naming: most custom licenses forbid implying endorsement with the official name, so product naming and marketing materials should avoid it;
    • Output-side compliance: open weights can be arbitrarily fine-tuned by any downstream party, and responsibility lies with the deployer—content safety and abuse protection must be built in-house and cannot rely on the model owner.

    5. Deployment Engineering: What an 8-GPU Minimum Actually Means

    5.1 The First-Principles VRAM Budget

    The weights must stay resident in VRAM, so start with the most conservative lower bound:

    1. Per-GPU weight use ≈ 1.56 TB / 8 GPUs ≈ 195 GB/GPU ≈ 182 GiB/GPU

    This number immediately makes the problem clear: 8 GPUs is the minimum bar, meaning each GPU needs on the order of 192GB of VRAM just to hold the weights, with almost no headroom left after loading. If you have 80GB-class accelerators, the weights alone require about 20 cards, and with runtime overhead you typically need 24 or more for usable service.

    1. WEIGHT_BYTES = 1.56e12 # decimal TB conversion
    2. GIB = 1024 ** 3
    3. def min_cards(weight_bytes: float, card_gib: float,
    4. reserve_ratio: float = 0.15) -> int:
    5. """reserve_ratio: fraction reserved for KV cache, activations, and framework overhead"""
    6. usable = card_gib * GIB * (1 - reserve_ratio)
    7. return -(-int(weight_bytes) // int(usable))
    8. print(min_cards(WEIGHT_BYTES, 80)) # 80GiB cards: ~23 cards
    9. print(min_cards(WEIGHT_BYTES, 192)) # 192GiB cards: ~9 cards

    5.2 Don’t Ignore the KV Cache

    For a long-context model, the real VRAM killer is the key-value cache, which grows linearly with concurrency and sequence length:

    1. KV(GiB) = 2 × layers × KV heads × head dim × bytes per element × token count / 1024³

    Plugging in illustrative values (93 layers, 8 KV heads, head dim 128, BF16 compute):

    1. Per token ≈ 0.36 MiB
    2. 1M-token single request ≈ 363 GiB

    In other words, even if the weights just barely fit, a single million-token context request’s cache could swallow an entire GPU’s VRAM. Real deployment almost always requires three things: setting the maximum context to what the business actually needs (rather than maxing out the default), enabling low-precision KV cache storage, and adopting paged cache management to improve fragmentation utilization. Sparse attention layers reduce compute overhead, but do not necessarily reduce cache use proportionally, so this needs to be measured empirically.

    5.3 Choosing a Parallelism Strategy

    1. # Illustrative launch command; parameter names depend on the inference framework in use
    2. launch_serve \
    3. --model /mnt/models/large-moe-2p8t \
    4. --tensor-parallel-size 8 \ # intra-layer split, lowers per-GPU weights and communication latency
    5. --expert-parallel-size 8 \ # expert-dimension split, mitigates expert load imbalance
    6. --max-model-len 131072 \ # set to actual business need, don't max out by default
    7. --gpu-memory-utilization 0.92 \
    8. --kv-cache-dtype fp8

    Key selection points:

    • Tensor parallelism splits each layer’s weights; it involves frequent communication but is latency-friendly, and is usually confined within a single machine’s high-speed interconnect domain;
    • Expert parallelism distributes parameters by expert; it moves little data but introduces expert load imbalance, requiring routing balancing and capacity-factor tuning;
    • Pipeline parallelism is a must for cross-machine scaling, at the cost of pipeline bubbles; with too few micro-batches, utilization drops noticeably;
    • For multi-machine deployment, cross-machine bandwidth is often the decisive factor, so the network topology must be settled during selection, not during stress testing.

    5.4 Delivery: Self-Hosted or Cloud

    At this scale, the upfront investment and operational complexity of a self-built cluster are both extremely high. The cloud path is usually a combination of general capabilities: elastic GPU compute that scales with task tidal patterns, object storage to carry the distribution and version management of the 1.56TB weights, a container platform to orchestrate inference services and autoscaling, and logging plus monitoring alerts covering first-token latency and throughput metrics. Cloud platforms such as Baidu AI Cloud provide this class of general infrastructure capability; specific specs and quotas need to be verified against actual business needs.

    Don’t cut corners on the download either: with 96 shards plus index files, resumable downloads and integrity checks are essential. After pulling, first compare the shard count and the index mapping for consistency before entering the load process, or you’ll only discover corrupted files after tens of minutes of weight loading.

    6. Selection Advice and Risk Checklist

    Suitable scenarios: repo-level code understanding and generation, long-document processing, multi-turn tool-calling agent workflows, and compliance-sensitive scenarios that need controllable weights and private fine-tuning.

    Risk checklist:

    1. Rigid hardware bar: The 8-GPU minimum with very high per-GPU VRAM needs basically rules out local deployment for small teams;
    2. No precision fallback: With only the 4-bit quantized version, both the fine-tuning room and the hardware fit range are locked down;
    3. Clearly skewed capability: Strong on coding and agents, weak on science and some reasoning—not suited to directly taking on rigorous reasoning tasks;
    4. Non-standard license: The two scale thresholds need upfront legal evaluation; when revenue or MAU nears a threshold, plan the commercial path in advance;
    5. High operational complexity: VRAM and throughput tuning, expert load balancing, and long-context cache management all require dedicated inference engineering capability;
    6. Benchmarks can’t be copied: Self-reported leaderboards are only a screening signal; validation on your own business set is the decision basis.

    One-line conclusion: This is an open-weight release that raises both the engineering and commercial bars at the same time. What a technical team really needs to answer is not “can we download it,” but three questions: “is per-GPU VRAM enough, how do we arrange the parallel topology, and will we cross the license thresholds?”