2.8T Open-Weight Model Release: Architecture, License, and Deployment

    This article examines a 2.8T-parameter open-weight model release from a technical and practical perspective. It covers the model’s MoE architecture, 4-bit quantization, benchmark interpretation, custom license restrictions, and local deployment requirements. It also explains why eight GPUs are only a starting point and provides guidance on memory estimation, inference frameworks, long-context engineering, and license compliance for teams considering adoption.

    Introduction: Open Weights Are Where Open Use Begins

    A model becoming available through an API only shows that its capabilities can be accessed. Releasing the weights for download is what allows enterprises to control deployment location, inference cost, and data boundaries. On November 4, a Chinese model provider released the complete weights of a flagship model, referred to below as “the model,” as promised. The model has 2.8T parameters, and the weight files total approximately 1.56 TB, making it one of the largest open-weight models available. In an LLM race where development costs can reach hundreds of billions of dollars, a provider choosing to publish the weights of a 2.8T-scale model is itself a notable shift.

    However, “open weights” and “freely usable in the open-source sense” are not the same thing. The release comes with three practical barriers:

    • Size barrier: The weights are close to 1.6 TB, and running them locally typically requires eight or more accelerators.
    • License barrier: The model uses a custom license rather than a standard open-source license, so the additional terms must be reviewed before commercial use.
    • Format barrier: Only one official version was released. The weights are stored in 4-bit floating-point format and the activations in 8-bit floating-point format, with no higher-precision alternative.

    The following sections break down these issues: what the model is made of, how its benchmark results should be interpreted, where the license boundaries lie, and how to account for local deployment.


    1. What Happened: From a Release Promise to Open Weights

    Based on publicly available information, the release process can be summarized in four stages:

    1. Service launch first: On October 24, the model became available through APIs and a web interface, while the provider promised that the full weights would be released by November 4.
    2. Temporary repository visibility: A hosted community repository was created on November 2. On the morning of the release day, it was still private.
    3. Official weight release: That evening, the repository became public. Downloading required no application. There were 112 files in total, including 96 weight shards, amounting to approximately 1.56 TB.
    4. Clear positioning: According to the model card, it is a natively multimodal Agent model, continued from the previous-generation flagship and further trained.

    During this process, “capabilities first, weights later” has become a common rhythm among leading model providers. APIs gather real-world feedback first, and open weights then help expand the ecosystem. For enterprise users, the real issues are not the demonstrations at the launch event but the three facts revealed once the weights are available: whether the architecture fits their workloads, whether the license permits their business, and whether existing hardware can handle it.


    2. Architecture Breakdown: What Makes Up 2.8T

    2.1 Parameters: Only About 3.7% Are Activated, but All Weights Must Be Loaded

    The model uses a mixture-of-experts (MoE) architecture. Its core parameters are:

    Item Parameter
    Total parameters 2.8T
    Activated parameters per token 104B, about 3.7% of the total
    Routed experts 896; 16 selected per token
    Shared experts 2
    Layers 93
    Context length 1 million tokens
    Multimodal capability Natively multimodal, with a self-developed vision encoder

    In simple terms, an MoE model is like a large expert consultation center. The 896 experts specialize in different areas. When a token arrives, the router selects the 16 most relevant experts, and two shared experts also participate. This has two benefits:

    • The total parameter count can be very large, providing substantial knowledge capacity.
    • Only a small subset of parameters is used per computation, so inference compute does not need to grow linearly with total parameters.

    However, a common misconception should be corrected: MoE saves compute, not memory. Routing only determines which parameters are read for a given computation; it does not change the fact that all expert weights must remain resident in memory. In other words, memory requirements are driven by 2.8T, while compute requirements are driven by 104B.

    If we roughly estimate from activated parameters, the routed-expert portion accounts for about 98.3% of total parameters, or about 2.75T. The vast majority of GPU memory ultimately has to be allocated to expert weights. This is the starting point for the deployment calculations later in this article.

    2.2 Structural Change: Full Attention in 24 Layers Replaced by Sparse Attention

    Compared with the previous-generation model, this model makes only one structural change: replacing full attention in 24 layers with sparse attention.

    The reason is that both attention computation and cache usage grow with sequence length. In a 1-million-token context, this problem is amplified. Full attention requires attending to all previous tokens, which is costly at every layer and every decoding step. Sparse attention lets each token attend only to a carefully selected portion of prior tokens, reducing compute and bandwidth consumption in long-context scenarios.

    This means that, although ultra-long context is supported on paper, the actual experience depends on three factors:

    • Whether the inference framework correctly implements sparse attention.
    • Whether KV Cache capacity is sufficient.
    • Whether inter-GPU communication bandwidth can support the workload.

    The “engineering bill” repeatedly mentioned below comes from this point.

    2.3 Quantization: Where the 1.56 TB Comes From

    According to the official description, the model weights use 4-bit floating-point format, and activations use 8-bit floating-point format. This type of format is usually divided into small blocks, with each group of weights sharing a scaling factor. It is a common low-bit quantization approach in the industry, and the model was trained directly in low precision using quantization-aware training (QAT).

    A simple arithmetic check verifies the weight size:

    1. Theoretical value = 2.8 x 10^12 parameters x 0.5 bytes/parameter ≈ 1.40 TB decimal
    2. File size = 1.56 TB
    3. Ratio ≈ 1.11

    The additional 11% or so comes from quantization scaling factors, tensor alignment, and additional components such as the vision encoder.

    The comparison with unquantized versions is more intuitive:

    • Storing in BF16 would require about 5.6 TB for weights alone.
    • Even FP8 would require about 2.8 TB.
    • 4-bit reduces this to about 1.4 TB, roughly half of the FP8 approach.

    This is why large models can be openly released at all: without quantization, such releases would be out of reach for individual users and small-to-medium teams.

    But quantization has costs. The official release provides only this one format and does not offer BF16 or FP8 versions. For scenarios that require extremely high output quality or model architecture modification, the absence of a higher-precision “master” version is a real constraint.


    3. How to Read the Benchmarks: Vendor Tests and Third-Party Evaluations

    3.1 Vendor-Reported Results

    The vendor’s model card first presents its own benchmark data for easier comparison. The table below normalizes competitor values to 100 for reference:

    Evaluation dimension This model Competitor A relative value Competitor B relative value
    Comprehensive coding ability in a terminal environment 100 101 89
    Long-horizon software engineering tasks 100 90 Not provided
    Scientific computing terminal tasks 100 166 Not provided
    High-difficulty multidisciplinary evaluation, Diamond subset 100 155 Not provided

    Note: The original values were 65.7% for comprehensive coding in a terminal environment, 51.7% for long-horizon software engineering, 38.1% for scientific computing terminal tasks, and 39.4% for the high-difficulty multidisciplinary evaluation. The original values are included only for checking the normalization ratio. The original table did not provide Competitor B’s data for the remaining three items.

    This table shows a clear pattern: strengths and weaknesses are sharply divided.

    • Strengths: Programming and Agent-style tasks. Terminal coding is slightly below the strongest closed-source model, while long-horizon software engineering is clearly ahead.
    • Weaknesses: Scientific computing and high-difficulty reasoning. On scientific computing terminal tasks and the high-difficulty multidisciplinary evaluation, the gap with the strongest closed-source model is over 50%.

    The vendor’s own wording is also cautious: the model surpasses Competitor A on 16 out of 37 benchmarks. In other words, it does not lead on more than half of the benchmarks. “Comparable results across the board, with wins and losses by item” is a more accurate summary.

    Third-party evaluations provide another reference. One third-party evaluator gave the model a composite intelligence score of 55, ranking it fourth globally. This is consistent with the vendor’s “clear strengths and weaknesses” data: overall capability is in the first tier, but it is not a comprehensive leader.

    3.2 Can Vendor Self-Reported Data Be Trusted?

    Any vendor-published self-reported benchmark data should be viewed with caution, for three main reasons:

    1. Uneven evaluation conditions. Prompt wording, sampling parameters, tool-calling strategies, and whether multiple attempts are allowed can all significantly affect results. Self-testing parties and third parties often differ in these details.
    2. Benchmarks themselves are saturating. Top models often differ by only a few percentage points on mainstream benchmarks, which is close to the noise level of the evaluation itself. A small lead on a single benchmark has limited significance.
    3. Test-set contamination risk. If evaluation questions appear in training data, scores may be systematically inflated.

    A reasonable approach is to treat vendor data as an initial screening clue rather than a procurement decision basis. For example:

    • If an enterprise needs a coding assistant capable of long-horizon code refactoring, a lead in long-horizon software engineering is a useful signal.
    • If the core requirement is scientific computing or complex reasoning, practical testing should be emphasized, and overall rankings alone should not be trusted.

    Recommendation: Every team should build a small evaluation set that fits its own business. It should contain dozens to hundreds of real tasks, covering typical scenarios and edge cases. Using the same evaluation set to compare candidate models horizontally yields decisions with practical value.


    4. License Analysis: Open Weights Do Not Mean Open Source

    This is the part of the weight release that deserves the most separate scrutiny.

    4.1 A Custom License Is Attached

    The model repository uses a custom license. The license type shown on the hosting platform is marked as “Other,” rather than a standard open-source license such as MIT or Apache. More notably, its license terms are word-for-word identical to those of the previous-generation model, except for the name. This suggests that the licensing strategy is “change the model, not the terms,” and that the team considered large-scale commercial scenarios when designing the license.

    4.2 Main Authorization: A Permissive Base

    The main body of the license is close to MIT-style licensing. Users are explicitly allowed to:

    • Use it for free.
    • Copy, modify, and distribute it.
    • Further fine-tune based on it.
    • Use it for commercial purposes.

    This means that most use cases are unobstructed. Individual developers conducting research, enterprises building internal tools, and startups developing vertical products are generally within the scope of authorization.

    4.3 Two Additional Restrictions: Only Affecting Very Large External Services

    On top of the permissive authorization, two restrictions are added. Both point to scenarios involving very large-scale externally facing services:

    1. Revenue threshold: Entities that provide the model externally in a “model as a service” form and have cumulative revenue of more than $20 million over 12 months must sign a separate agreement with the vendor before commercial use.
    2. Monthly active user threshold: When a commercial product exceeds 100 million monthly active users, the same separate-agreement requirement is triggered. The original excerpt was truncated at this point; the exact figure should be confirmed against the official license text.

    These two restrictions send a clear signal: the vendor does not reject commercial use and encourages building applications on top of the weights. What it wants to constrain is the small number of players who simply take the model and run very large-scale businesses without contributing back. In other words, the license targets small-to-medium innovators, not very large platforms that could directly compete with the vendor.

    4.4 Scenarios That Need Attention

    Against these terms, the following situations should be checked before launch:

    Use case Trigger restriction? Explanation
    Individual research and teaching No Within the permissive authorization
    Internal enterprise tools and data analysis No Does not provide model services externally
    Small and medium team vertical products Usually no Unless revenue or monthly active users reach the thresholds
    Externally providing model API services Depends on scale A separate agreement is required if rolling 12-month revenue exceeds $20 million
    Commercial products with over 100 million monthly active users Yes A separate agreement is required
    Redistribution, including redistribution after further fine-tuning Original license must be retained Downstream users remain subject to the terms

    Two reminders:

    • Revenue is calculated on a rolling 12-month basis. Fast-growing business lines should anticipate this in advance to avoid rushing near the threshold.
    • When redistributing, the original license terms must be passed along. A modified version does not automatically escape the license constraints.

    4.5 Compliance Self-Check Checklist

    1. def need_separate_agreement(is_model_as_service, revenue_usd_12m, mau):
    2. """
    3. Determine whether a separate commercial agreement with the model provider is required.
    4. The thresholds should be verified against the official license text; this is illustrative only.
    5. """
    6. REVENUE_THRESHOLD = 20_000_000 # $20 million
    7. MAU_THRESHOLD = 100_000_000 # 100 million monthly active users; verify against official license
    8. if is_model_as_service and revenue_usd_12m > REVENUE_THRESHOLD:
    9. return True
    10. if mau > MAU_THRESHOLD:
    11. return True
    12. return False

    Implementation suggestion: include license terms in the regular model selection process, have legal and technical teams confirm them together, and record the license conclusion in technical documentation at project initiation. The cost is far lower than rework after launch.


    5. What “8 GPUs as a Starting Point” Means: The GPU Memory Bill for Local Deployment

    5.1 Why Eight GPUs?

    Back to arithmetic. Putting 1.56 TB of weights into accelerator memory is constrained most directly by whether the total memory of eight GPUs can hold them. Consider a common high-memory configuration: eight GPUs with about 140 GB of memory each.

    1. Single-node total memory ≈ 8 x 140 GB = 1120 GB
    2. Weight size ≈ 1.56 TB, about 1560 GB

    1120 GB is clearly less than 1560 GB. This comparison uses decimal capacity. If the vendor’s “1.56 TB” figure is actually 1.56 TiB, the gap becomes even larger. This shows that “8 GPUs as a starting point” does not mean that eight GPUs are just enough; it means that eight-GPU single-node deployment is already close to the physical lower limit.

    Even with eight GPUs, not all memory can be allocated to weights. It is also necessary to reserve:

    • KV Cache: With a 1-million-token context, the cache can reach tens to hundreds of GB and grows linearly with concurrency and sequence length.
    • Activations and temporary buffers: These depend on batch size and parallelism strategy.
    • Framework overhead: Communication buffers, memory fragmentation, and similar costs.

    If total memory is insufficient, some weights must be offloaded to host memory or local storage. The cost is repeated data movement during inference, significantly reduced throughput, and interactive applications becoming essentially unusable.

    5.2 How to Estimate GPU Memory

    Rather than memorizing the number “eight GPUs,” it is more useful to understand the estimation method. The core formula is:

    1. Required total memory ≈ weight size + KV Cache + activations and buffers + framework overhead
    2. Per-GPU memory requirement ≈ required total memory ÷ number of GPUs

    Illustrative code:

    1. def estimate_vram(params_b, weight_bytes=0.5, kv_cache_gb=80,
    2. buffer_gb=40, overhead_ratio=0.1):
    3. weight_gb = params_b * 1e9 * weight_bytes / 1e9 # about 1400 GB
    4. total_gb = weight_gb + kv_cache_gb + buffer_gb
    5. total_gb *= (1 + overhead_ratio)
    6. return total_gb
    7. for gpus in (8, 16, 32):
    8. total = estimate_vram(2800)
    9. print(f"{gpus} GPUs: total ≈ {total:.0f} GB, per GPU ≈ {total/gpus:.0f} GB")

    Approximate results:

    • With 8 GPUs, each GPU needs about 210 GB, exceeding mainstream single-GPU capacity.
    • With 16 GPUs, the per-GPU requirement drops to about 105 GB, entering a feasible range.
    • With 32 GPUs, the per-GPU requirement is about 52 GB, but cross-node communication becomes a new bottleneck.

    In actual selection, the following constraints also matter:

    • Inter-GPU interconnect bandwidth: Tensor parallelism exchanges activations at every step. Insufficient bandwidth directly increases latency.
    • Prefer multiple GPUs within one node over multiple GPUs across nodes: Intra-node interconnects are usually much faster than inter-node networks.
    • Context length and concurrency: KV Cache grows with both, so headroom must be reserved.

    5.3 Can Ordinary Users Run It?

    Overall:

    • Running it on a home computer or a single GPU is unrealistic.
    • Eight GPUs in a single node is barely conceivable as an option, but hardware selection is extremely demanding. It requires high-memory GPUs and high-speed interconnects, with high cost.
    • For most teams, a more practical path is API access or cloud-hosted inference.

    Individual researchers who want to experience capabilities at this scale still have the lowest-cost option of using services provided by the official platform or cloud providers, rather than building a cluster.


    6. Practical Deployment: Key Steps from Download to Production

    6.1 Download and Integrity Verification

    The weights are distributed as 96 shards, totaling about 1.56 TB. Recommended practices:

    • Use a download method that supports resumable transfers and multi-threading to avoid losing progress because of network fluctuations.
    • After downloading, verify each shard against the official checksums to confirm there is no corruption.
    • Store the weights in object storage or high-speed file storage to avoid repeated downloads from the public internet.

    6.2 Inference Framework and Parallelism Strategy

    • Prefer the inference framework version recommended by the official documentation. Support for new architectures such as sparse attention and 4-bit floating point varies significantly across frameworks.
    • Parallelism is usually based primarily on tensor parallelism (TP), supplemented by pipeline parallelism (PP) when the number of GPUs is large. MoE models can also use expert parallelism (EP) to distribute experts across different GPUs.
    • Higher parallelism is not always better. TP requires frequent communication, and cross-node TP usually sees diminishing returns.

    6.3 Quantization and Precision

    The official release provides only 4-bit weights. If further compression is desired, the following should be evaluated:

    • Re-quantizing to lower bit widths may cause accuracy loss and must be validated with a business evaluation set.
    • For accuracy-sensitive scenarios, consider intermediate options such as FP8, provided the framework supports them.
    • For models already trained with quantization-aware training, applying post-training quantization again may not be worthwhile and can sometimes perform worse than keeping the original format.

    6.4 Engineering Details for Long Context

    A 1-million-token context is not free. Key concerns include:

    • KV Cache management: Support for paged cache and prefix reuse directly affects memory efficiency in multi-turn dialogue and long-document scenarios.
    • Correctness of sparse attention: Different frameworks may implement sparse patterns differently. It is advisable to validate with long-context retrieval tasks such as needle-in-a-haystack tests.
    • Visual inputs: Images are encoded into many tokens. Context consumption for multimodal requests is often underestimated.

    6.5 Performance Acceptance

    Before launch, at least the following metrics should be load-tested:

    Metric Meaning Focus
    TTFT Time to first token Affects interactive experience, especially important with long contexts
    TPOT Time per output token Determines output speed
    Throughput Tokens per unit time Determines unit cost
    Concurrency capacity Number of requests that can be served Directly related to KV Cache

    It is recommended to load-test with request distributions close to real traffic, rather than only running a single long-text request.


    7. Selection Advice: When Is Self-Deployment Worth It?

    Your situation Recommendation
    Data must not leave your domain; private deployment is required Strong candidate, but hardware costs must be calculated
    Extremely high call volume and stable long-term demand Self-built clusters may be more cost-effective by amortizing unit cost
    Providing model services externally through APIs Check the revenue threshold first and sign early if necessary
    No GPU operations experience Consider using cloud-hosted services first to validate value
    Business focuses on scientific computing or complex reasoning Evaluate cautiously; these are not its strongest areas
    BF16/FP8 or other higher-precision versions are required Not officially provided; assess whether this is acceptable

    A general decision framework: first calculate (memory and cost), then validate (business evaluation), then ensure compliance (license review). Only after all three steps pass should deployment be discussed.

    In the cloud, this type of work can usually be supported by several common capabilities:

    • Use GPU cloud servers to quickly build validation environments.
    • Use container platforms for elastic scaling and version management.
    • Use object storage to host TB-scale weights and pull them from nearby locations.
    • Use logging services and monitoring alerts to track memory usage, latency, and error rates.

    Conclusion: From “Can It Run?” to “How Should It Be Used?”

    A 2.8T-parameter open-weight release is a step forward for the open ecosystem, but it is more like a qualification test:

    • It filters out individual users who only have a single GPU.
    • It also filters out casual users who are unwilling to read the license terms carefully.

    Three conclusions are valuable:

    1. Architecturally, it is a typical “large parameters, small activation” MoE model: memory-constrained, compute-friendly, and its long-context capability depends on engineering implementation.
    2. Licensing-wise, it combines permissive authorization with additional terms for very large-scale commercial use. Most teams can use it with confidence, while very large players should communicate in advance.
    3. Deployment-wise, eight GPUs are only the starting point. Single-GPU memory, interconnect bandwidth, and KV Cache management are the keys to the actual experience.

    For technical teams, instead of worrying about whether it can run, it is better to focus on three things: build your own evaluation set, calculate the hardware budget clearly, and complete the license review in advance. Once these three steps are solid, even a very large model can be brought into production.