2.8T-Parameter Open-Weight MoE: Architecture, License, and 8-GPU Deployment Explained

    A 2.8T-parameter mixture-of-experts model with 104B active parameters was released as open weights on November 4, 2026, in a single 1.56 TB MXFP4 release. This article breaks down what the release actually contains: how the sparse MoE and sparse attention layers are structured, how MXFP4 weights and MXFP8 activations shape hardware and fine-tuning constraints, why "open weights" is not the same as open source under a custom MIT-style license with two large-scale commercial thresholds, and how to size GPUs for 1.56 TB of weights plus KV cache. It also covers benchmark caveats, parallelism strategies, throughput tuning, and a compliance checklist for commercial use.

    1. What Happened: The Weights Landed, With Conditions

    In late October 2026, a China-based model team launched a trillion-parameter-class model first as an API and a web interface, and promised to release the full weights by November 4. The weights appeared on mainstream model hosting platforms on that date: the repository was publicly downloadable with no application required. The repository had been created a few days earlier and stayed closed until the evening of the release day.

    The repository contains 112 files, 96 of which are weight shards, totaling roughly 1.56 TB. That makes it the largest open-weight model released to date. The sheer size throws three questions into sharp focus:

    • What is inside: how 2.8T parameters are organized, how the sparse structure controls compute, and what releasing only a quantized version implies;
    • How the license works: the difference between “open weights” and open source, and where the commercial red lines sit;
    • Whether it can run: what a 1.56 TB weight footprint demands in hardware, and which cost-reduction paths exist.

    This article does not judge the model’s quality. It addresses only these three questions and offers a reusable framework for evaluating them.

    2. Dissecting the Model: How 2.8T Parameters Are Built

    2.1 Positioning: A Natively Multimodal Agent Model

    The model is positioned as a natively multimodal Agent model, trained as a continuation of the previous generation. “Natively multimodal” means visual input was modeled jointly with text during the pre-training stage, rather than being bolted on afterwards as a separate vision module. That choice determines its potential on tasks that combine reading images, operating tools, and multi-step execution.

    Agent capability comes down to long-horizon planning and tool calling, and it is most sensitive to two metrics:

    • Per-step cost: a complex task typically requires many rounds of reasoning and tool interaction;
    • Context length: historical traces, code snippets, and page content accumulate quickly.

    With 93 layers and support for a 1M-token context, the architecture is clearly designed to pave the way for long-horizon agentic work.

    2.2 Sparse MoE: 2.8T Total, 104B in Compute

    The model has 2.8T total parameters and activates only 104B per token, an activation rate of roughly 3.7%. It uses a mixture-of-experts (MoE) structure:

    • Routed experts: 896 routed experts, with 16 selected per token;
    • Shared experts: 2 shared experts are always active, carrying general capabilities and reducing the pressure on routed experts to repeatedly learn the same things;
    • Training vs. inference: training typically requires expert load-balancing strategies to avoid overloading popular experts and under-training cold ones; at inference time, the number of active parameters determines the compute per token, while total parameters determine memory footprint.

    The key implication: the compute budget is close to that of a 100B-class dense model, but the memory budget is that of a 2.8T model. This is the fundamental tension of very large MoE models, and it explains two things:

    • why throughput per unit of compute can be high;
    • why the first bottleneck after the weights land is not “can it compute” but “can it fit”.

    Fine-grained routing also carries overhead of its own. Every token must pass through the routing network to compute gating scores, select experts, and trigger All-to-All communication across GPUs or across machines. The more experts and the finer the parallel partitioning, the more visible this cost becomes. Consequently, real-world MoE throughput often depends on the efficiency of routing and communication rather than on peak floating-point throughput alone.

    2.3 The Only Structural Change From the Previous Generation: Sparse Attention

    Compared with the previous generation, the architecture changed in one place only: 24 layers of full attention were replaced with sparse attention to save compute on long contexts.

    In dense attention, every token is compared pairwise with all preceding tokens, so the cost grows quadratically with sequence length. With a 1M-token input, attention would consume a substantial share of both compute and memory bandwidth. Sparse attention instead works as follows:

    • each query token performs attention only against a small number of selected historical tokens;
    • global information is carried through mechanisms such as compressed representations.

    A common implementation first builds a token-level index score, then runs exact attention only over the k highest-scoring tokens. The cost drops from O(L²) to roughly O(L·k), where k is independent of sequence length.

    One common misconception needs clearing up: sparse attention reduces the compute and bandwidth cost of attention, but the KV cache must still hold all historical tokens, so memory usage still grows linearly with context length. In short, sparse attention saves compute, not memory.

    2.4 Quantization: MXFP4 Weights and MXFP8 Activations

    Weights use the MXFP4 format and activations use MXFP8, the product of quantization-aware training.

    • MXFP4: a microscaling 4-bit floating-point format with 1 sign bit, 2 exponent bits, and 1 mantissa bit, where every group of 32 elements shares a scale factor. Its dynamic range is far wider than that of traditional 4-bit integers;
    • MXFP8: an 8-bit floating-point activation format that likewise uses block scaling;
    • Quantization-aware training (QAT): quantization error is folded into the optimization objective during training, so the model adapts to low-precision inference early. The accuracy loss is typically noticeably smaller than with post-training quantization.

    The size of the release can be verified with simple arithmetic:

    1. total_params = 2.8e12 # total parameter count
    2. active_params = 104e9 # active parameters per token
    3. bits_per_weight = 4.0 # MXFP4: roughly 4 bits per weight
    4. # weight size (TB = 10^12 bytes)
    5. weight_tb = total_params * bits_per_weight / 8 / 1e12
    6. print(f"Weight size: {weight_tb:.2f} TB") # 1.40 TB
    7. print(f"Activation rate: {active_params / total_params:.1%}") # 3.7%
    8. # prefill for a 100k-token prompt, compared with a dense model of the same active size
    9. # only the MoE expert portion is compared; attention cost is identical for both
    10. prompt_tokens = 100_000
    11. dense_flops = 2 * prompt_tokens * active_params # forward MACs are roughly 2x parameter count
    12. moe_flops = dense_flops * (16 + 2) / (896 + 2) # scaled by the share of selected experts per layer
    13. print(f"Prefill compute ratio: {moe_flops / dense_flops:.1%}") # about 2.0%

    The measured weight size is about 1.56 TB, roughly 11% above the theoretical value. The difference comes mainly from the mixed-precision representation of non-expert parts such as embeddings and attention, plus file-format and sharding overhead.

    Only this single quantized version was published, with no BF16 or FP8 release. The practical impact shows up in four areas:

    1. Hardware binding: consumer GPUs and some accelerators do not natively support MX formats, so values must be dequantized or emulated in software at load time, and speed drops noticeably;
    2. Limited re-quantization: scaling “back up” from 4 bits to 8 or 16 bits cannot recover information that was already discarded;
    3. Fine-tuning constraints: full-parameter fine-tuning generally requires optimizer states to be kept in 16-bit floating point as well, and that memory cost is often several times the weights. Full-parameter fine-tuning on a quantized base therefore requires specialized low-bit training approaches;
    4. Evaluation baselines: when benchmark scores are compared against higher-precision models, part of the accuracy gap comes from quantization itself.

    3. Reading the Results: Strong at Coding and Agents, Weaker at Science and Reasoning

    All figures below are vendor-reported with no third-party reproduction. “Flagship A” and “Flagship B” stand in for contemporaneous closed-source flagship models.

    Vendor-reported benchmark This model Flagship A Flagship B Reading
    Terminal coding benchmark 65.7% 66.4% 58.2% Slightly below Flagship A, above Flagship B
    Long-horizon software engineering benchmark 51.7% 46.3% Not listed Ahead of Flagship A
    Scientific coding benchmark 38.1% 63.3% Not listed Clearly behind
    Knowledge reasoning benchmark 39.4% 60.9% Not listed Clearly behind

    The vendor states that 16 of 37 benchmarks beat Flagship A. A third-party evaluation platform gives an intelligence index of roughly 55, placing the model fourth globally.

    3.1 What This Table Says

    • Not a clean sweep: going through the table line by line, the outcome is mixed. Coding and agentic tasks are close to the leading tier, while scientific computing and deep reasoning show a clear gap.
    • Consistent with positioning: the model continues from a previous generation built around agents, and it overtakes Flagship A on long-horizon software engineering, which suggests iteration resources went into tool use and long-horizon execution. The weakness in scientific reasoning may also be related to the quantized release.
    • No “overall superiority”: vendor marketing that says the model “surpasses some model” generally refers to those 16 out of 37 benchmarks, not all of them.

    3.2 Three Points for Reading Vendor-Reported Results

    1. Only comparable under the same settings: differences in temperature, prompt template, maximum generation length, and reasoning budget (such as the number of thinking tokens) can shift results by several percentage points. More importantly, agentic benchmarks are extremely sensitive to the “scaffold”, including tool definitions, error retries, context management policy, and sandbox environment. Put the same model in a different framework and the score can change materially. Whether it can complete real tasks is therefore more informative than a few extra points on a benchmark.
    2. Look at the task distribution, not the average: model selection should be driven by your own business’s real task set. Sample several hundred representative tasks to build a private evaluation set, and make decisions on task completion rate rather than an aggregate benchmark score.
    3. Parameter count is not capability: active parameters, training data quality, and post-training strategy matter just as much. Parameter count is only a prior condition.

    4. Hardware Requirements: Where 1.56 TB and 8 GPUs Come From

    4.1 From Weight Size to Minimum GPU Count

    The estimation logic: required memory is roughly weight size plus KV cache; divide weight size by per-GPU memory and round up. Using 1.56 TB of weights:

    1. import math
    2. def min_gpus(weight_tb, gpu_mem_gb, kv_cache_gb=0, overhead_ratio=0.05):
    3. """
    4. Estimate the minimum number of GPUs needed to satisfy memory requirements (illustrative logic)
    5. weight_tb: weight size (TB, counted in 10^12 bytes)
    6. gpu_mem_gb: memory per GPU (GB)
    7. kv_cache_gb: KV cache headroom (GB)
    8. overhead_ratio: margin ratio for activations, framework overhead, and so on
    9. """
    10. weight_gb = weight_tb * 1000
    11. required_gb = (weight_gb + kv_cache_gb) * (1 + overhead_ratio)
    12. return math.ceil(required_gb / gpu_mem_gb)
    13. scenarios = [
    14. ("192GB x 8 GPUs", 192, 0),
    15. ("192GB x 8 GPUs (KV cache reserved for long context)", 192, 64),
    16. ("96GB x 16 GPUs", 96, 0),
    17. ("80GB x 20 GPUs", 80, 0),
    18. ]
    19. for label, mem, kv in scenarios:
    20. n = min_gpus(1.56, mem, kv)
    21. print(f"{label}: at least {n} GPUs required")
    22. if n == 8 and kv == 0:
    23. print(f" Weight utilization on 8 GPUs: {1.56 * 1000 / (8 * mem):.1%}")
    1. 192GB x 8 GPUs: at least 8 GPUs required
    2. Weight utilization on 8 GPUs: 97.4%
    3. 192GB x 8 GPUs (KV cache reserved for long context): at least 9 GPUs required
    4. 96GB x 16 GPUs: at least 16 GPUs required
    5. 80GB x 20 GPUs: at least 20 GPUs required

    In decimal terms, 1.56 TB of weights is about 15,600 GB, while 8 GPUs at 192 GB each provide 1,536 GB combined, so the weights alone consume 97.4% of it. Converted to base 1024, the figure is roughly 14,500 GiB and actual utilization lands around 94%. Either way, 8 GPUs is only a theoretical floor. The moment you handle long contexts or concurrent requests, memory has to be reserved for the KV cache and 8 GPUs become tight immediately. That is the real meaning of “8 GPUs to start”: fitting the weights is just the entry ticket.

    4.2 KV Cache: The Hidden Giant

    Mainstream inference frameworks manage the KV cache in blocks. The cache a single request occupies can be estimated as follows:

    1. def kv_cache_bytes(layers, kv_heads, head_dim, seq_len,
    2. bytes_per_elem=1, scale=1.0):
    3. """
    4. Estimate the KV cache size for a single request
    5. layers: number of layers
    6. kv_heads: number of KV heads (far smaller than query heads under grouped-query attention)
    7. head_dim: dimension per head
    8. seq_len: sequence length (prompt + tokens generated so far)
    9. bytes_per_elem: bytes per element, e.g. 1 for FP8, 2 for FP16
    10. scale: if historical tokens are compressed, scale by the compression ratio
    11. """
    12. elems_per_token = 2 * layers * kv_heads * head_dim # one copy each for K and V
    13. return elems_per_token * seq_len * bytes_per_elem * scale

    What each factor means:

    • Linear growth: double the sequence length and the cache doubles;
    • Precision: switching to 8-bit floating-point storage halves it;
    • Architectural differences: models using grouped-query attention or compressed KV can come in well below the theoretical value;
    • The boundary of sparse attention: as noted earlier, sparse attention does not reduce the KV cache.

    A 1M-token context is therefore an upper bound on capability. Prefilling an extremely long prompt takes minutes of compute, all intermediate results stay resident in memory, and concurrent requests stack on top of that, so memory overflow is easy. The sensible approach is to pair the model with retrieval-augmented generation (RAG) so it reads relevant material on demand, instead of stuffing a huge context in on every call.

    4.3 From “Fits” to “Runs Smoothly”

    Beyond the memory threshold, three practical constraints remain:

    • Cross-machine interconnect: 80 GB-class GPUs mean roughly 20 cards, which necessarily span multiple machines, and at that point the network becomes the throughput ceiling. Topologies with high inter-GPU bandwidth suit tensor parallelism, while plain Ethernet is a better fit for expert parallelism;
    • Low per-GPU throughput: under expert parallelism each GPU holds only some of the experts and every token must be routed, so less work is parallelizable per device. Batching requests together is necessary to raise utilization;
    • Power and datacenter: a single 8-GPU node typically requires multiple high-power CPUs and redundant power supplies, with firm demands on facility power delivery and cooling.

    4.4 Who Should Self-Host

    Use case Recommended approach Why
    Individual trials, light usage Hosted API Billed per token, zero operations, a small amount of money is enough for an initial evaluation
    Small-to-medium scale, intermittent use On-demand cloud accelerator instances No procurement cost, elasticity on demand
    Data-sensitive, long-term high-volume calls Self-hosted cluster Data stays inside your network and marginal cost is low, but upfront investment is large
    Local experience only Smaller quantized or distilled models 1.56 TB will not run on consumer hardware; use a smaller model instead

    A rough calculation: assuming a 192 GB-class cloud accelerator instance is billed at roughly 10 CNY (around USD 1.4) per hour, running around the clock for a month costs about 7,200 CNY (roughly USD 1,000) in compute alone, before storage and networking. For workloads with unstable daily call volume, calling an API directly is usually the better deal. The break-even point for self-hosting depends on three things:

    • call volume;
    • call stability;
    • data compliance requirements.

    5. Licensing: Open Weights, Not Open Source

    5.1 Nature of the License

    The repository ships a custom license, tagged as “other” on the platform rather than a standard open-source license such as MIT or Apache. The license text is word-for-word identical to the previous generation’s, with only the name changed.

    The difference between open weights and open source lies in what is delivered:

    • Open weights: the final parameters are delivered and can be used for inference, fine-tuning, and distillation; but training code, data mixture, and the full training pipeline are generally not published, so third parties cannot easily reproduce the model from scratch;
    • Open source: normally requires that code, data, and training methodology be available, reproducible, and modifiable.

    5.2 Granting Entity: A Permissive, MIT-Style Grant

    The core of the license is indeed an MIT-style grant permitting free use, copying, modification, distribution, fine-tuning, and commercial use, which is friendly to individual developers and most enterprises.

    5.3 Additional Constraints: Two Limits Aimed at Large-Scale External Business

    • Revenue threshold: companies whose primary business is offering the model as a service externally and whose cumulative revenue over 12 months exceeds USD 20 million must sign a separate agreement with the model provider before commercial use;
    • Scale threshold: when a commercial product’s monthly active users reach a certain scale, separate discussion is likewise required. Public materials are truncated at this point, so the specific figure should be taken from the official license text.

    Both limits target only businesses that “provide model capability externally at very large scale”, not commercial use in the ordinary sense. They do, however, introduce an ongoing obligation to assess:

    • Threshold determination: the 12-month cumulative revenue figure is a rolling measure spanning across calendar years; MAU is counted per specific product line or application, and a growing product may cross a line without anyone noticing;
    • Policy and commercial risk: custom terms can be revised, so products built on the model need to evaluate supply-chain risk;
    • Redistribution boundaries: when distributing modified versions of the model, how the additional terms carry over needs to be settled in advance.

    5.4 Compliance Checklist

    1. Define the use case: personal use, embedding in a product, offering MaaS externally, and redistributing the model all come with different constraints;
    2. Monitor thresholds continuously: if the business is approaching the revenue or MAU threshold, discuss licensing with the model provider early;
    3. Keep a copy of the license: record the date and version obtained, and watch for license changes;
    4. Preserve attribution notices: retain copyright and license notices as required when distributing the model or derivative works;
    5. Establish a model admission process: teams may use models under different licenses at the same time, so license review should be a fixed step in model selection.

    6. Deployment Practice: Making 1.56 TB Actually Run

    6.1 Parallelism Strategy

    Deploying an MoE model at this scale requires three kinds of parallelism working together:

    • Tensor parallelism (TP): splits each layer’s matrix operations across multiple GPUs. Communication is frequent and voluminous, so it suits intra-node parallelism where inter-GPU bandwidth is high;
    • Expert parallelism (EP): distributes the 896 experts across different GPUs, with each GPU loading only some expert weights. Communication is dominated by post-routing All-to-All;
    • Pipeline parallelism (PP): partitions by layer across nodes, suitable for cross-machine scaling.

    A common single-node 8-GPU configuration, for illustration:

    1. # Illustrative only; exact parameter names and valid ranges follow your inference framework's docs
    2. model:
    3. quantization: mxfp4 # must match the published weight format
    4. parallelism:
    5. tensor_parallel_size: 8 # tensor parallel degree, generally no more than GPUs per node
    6. expert_parallel_size: 8 # expert parallel degree, determines experts held per GPU
    7. data_parallel_size: 1 # multiple replicas for concurrency; enable once memory allows
    8. context:
    9. max_model_len: 262144 # set from actual workload; no need to open the full 1M at once
    10. cache:
    11. kv_cache_dtype: fp8 # reduce KV cache memory
    12. enable_prefix_caching: true # reuse cache for identical prefixes
    13. runtime:
    14. gpu_memory_utilization: 0.90 # leave room for the KV cache
    15. max_num_seqs: 32 # concurrent requests; tune against throughput targets

    6.2 Throughput Tuning

    • Batching: with small batches, MoE has little work to parallelize per GPU. Increasing the concurrent batch raises compute density and amortizes routing and communication overhead;
    • Chunked prefill and prefix caching: long prompts and multi-turn conversations share repeated prefixes. Chunked prefill keeps long requests from blocking decode, and prefix caching lets you reuse them directly;
    • Speculative decoding: a small draft model, or the model’s own shallow-layer prediction, guesses several tokens that the large model then verifies in one pass. Agent workloads tend to have predictable output, so the gains are usually clearer;
    • Quantized KV cache: storing the KV cache in 8-bit floating point expands the context you can serve within an acceptable accuracy loss.

    6.3 Reliability

    • Load validation: after loading sharded weights, run a fixed prompt for a consistency check to catch garbled output or abnormal repetition;
    • Tiered fallback: when self-hosted service degrades, fail over automatically to a hosted API;
    • End-to-end monitoring: track time to first token, time per output token, memory usage, request success rate, and timeout rate;
    • Capacity planning: treat “weights plus estimated KV cache” as the baseline and leave roughly 10% memory headroom.

    6.4 Example: A Minimum Viable Pipeline for Batch Agent Tasks

    Consider an automated agent pipeline that reads a repository, locates an issue, and submits a fix:

    1. # Illustrative code: batch agent tasks emitting unified JSON
    2. import json
    3. tasks = [
    4. {"repo_id": "svc-a", "issue": "no retry after API timeout"},
    5. {"repo_id": "svc-b", "issue": "pagination index out of bounds"},
    6. ]
    7. def run_agent(task, config):
    8. # 1. retrieve context: fetch only the relevant snippets, don't stuff the whole repo into the prompt
    9. context = retrieve_context(task["repo_id"], task["issue"])
    10. # 2. build the prompt: the output structure is constrained server-side
    11. prompt = (
    12. "You are a code-fixing assistant. Locate the issue from the snippets below and propose a fix.\n"
    13. f"Code snippets:\n{context}\n"
    14. f"Problem description: {task['issue']}\n"
    15. 'Output JSON only: {"root_cause": str, "patch": str, "risk": str}'
    16. )
    17. # 3. call the OpenAI-compatible endpoint exposed by the inference service
    18. # for the 8-GPU deployment, base_url points at the local service; for on-demand instances, at the cloud service
    19. resp = requests.post(
    20. f"{config['base_url']}/v1/chat/completions",
    21. json={
    22. "model": config["model"],
    23. "messages": [{"role": "user", "content": prompt}],
    24. "temperature": 0.2,
    25. "max_tokens": 2048,
    26. "response_format": {"type": "json_object"},
    27. },
    28. timeout=120,
    29. )
    30. resp.raise_for_status()
    31. return resp.json()["choices"][0]["message"]["content"]
    32. results = []
    33. for task in tasks:
    34. try:
    35. patch = json.loads(run_agent(task, config))
    36. results.append({"task": task, "patch": patch})
    37. except (requests.RequestException, json.JSONDecodeError) as e:
    38. # one failing task should not break the whole pipeline
    39. results.append({"task": task, "error": str(e)})
    40. with open("patches.json", "w", encoding="utf-8") as f:
    41. json.dump(results, f, ensure_ascii=False, indent=2)
    1. # Illustrative code: local inference service configuration
    2. config = {
    3. "base_url": "http://localhost:8000", # local inference service address
    4. "model": "locally deployed model name", # set by inference service launch arguments
    5. }
    1. # Illustrative code: service configuration on on-demand accelerated instances
    2. config = {
    3. "base_url": "https://inference-service-domain", # provided by your cloud platform
    4. "model": "cloud-deployed model name",
    5. }

    This example works with any inference service that exposes an OpenAI-compatible API. In production, API keys should be read from environment variables or a secret manager, never hardcoded. Every step of the pipeline can be swapped for a real implementation, decoupling the flow from any specific model.

    7. Conclusion: Turning a Release Into Usable Capability

    This weight release carries three kinds of value:

    1. Technical value: trillion-parameter sparse MoE combined with sparse attention and low-bit quantization offers an engineering pattern for long-context agents with controllable compute and controllable memory;
    2. Licensing value: a permissive core plus two large-scale limits is an experiment in balancing openness against commercialization, but users must read the additional terms;
    3. Engineering value: it pushes “how to evaluate, deploy, and govern very large MoE models” in front of many more teams.

    The three questions, answered directly:

    • Worth downloading: it depends on whether your business needs long-context plus tool-calling capability. Evaluate via a hosted API first, then decide whether to self-host;
    • Can it run locally: 8 GPUs is only the floor for fitting the weights. Add KV cache and concurrency headroom and you generally need more memory or higher interconnect bandwidth;
    • Safe to use commercially: the core grant is permissive, but if your business is a large-scale external service, complete the compliance check in advance.

    If the conditions are not met yet, there is no need to wait. Select models by capability rather than by brand: validate with on-demand cloud services first, and build up your evaluation sets, inference service, and monitoring along the way. Once hardware or business conditions mature, switching the backend configuration moves the capability onto a self-hosted cluster with no disruption.

    Note: all benchmark figures in this article are vendor-reported and have not been independently reproduced. Nothing here constitutes a recommendation for or against any model or service; base actual selection decisions on official information and independent evaluations.