Open-Source AI Reaches 2.8 Trillion Parameters: Sparse MoE Architecture and a New Commercial License Model

Open Source Community Sets a New Scale Record

The open-source large model ecosystem is undergoing rapid technical iteration. Following the release of a base model earlier this year, a Chinese AI lab fulfilled its commitment on schedule this week, opening the full weights of its flagship large model to the global developer community. With 2.8 trillion total parameters, the model is the largest dense-sparse hybrid architecture ever released in open source, setting a new ceiling for open-source model capability.

Notably, this release comes less than two weeks after the closed-source version of the same model was made available. The lab stated explicitly that open-weight distribution is part of a long-term strategy rather than a one-off action. Multiple senior AI researchers overseas have commented that this lab’s track record for honoring release timelines has become an industry consensus; across several prior model releases, delivering weights on schedule has become a consistent practice.

Architecture Deep Dive: Sparse Attention Meets MoE

The Engineering Trade-off Between Total and Active Parameters

The model uses a Mixture-of-Experts (MoE) architecture with 2.8 trillion total parameters, yet only 104 billion parameters are activated during inference. The design philosophy behind this sparse activation is straightforward: by distributing knowledge across a large number of expert subnetworks, each inference pass only routes to the most relevant subset of experts. This preserves model capacity while keeping computational overhead under control.

From an engineering standpoint, 2.8 trillion total parameters means several terabytes of GPU memory are required just to host the full weights. Even with half-precision storage, over 5 TB is needed. This places the model’s target audience primarily at enterprises operating multi-node GPU clusters, rather than individual developers.

Introduction of Sparse Attention

Compared to the previous generation, which invested heavily in full attention layers, the new version introduces sparse attention in 24 key layers. The core idea is that for each query token, attention weights are computed not against all tokens in the sequence, but only against the 2,048 most relevant tokens.

This change has significant implications for long-context processing. With a 1-million-token context window, the computational complexity of full attention grows quadratically with sequence length. Sparse attention reduces that to a near-linear scale. In practice, when processing long documents of the same length, KV cache usage and attention computation time both drop measurably after sparse attention is introduced.

Native Multimodality and Context Window

The model natively supports image input without requiring an additional vision-encoder adapter layer to handle mixed text-image understanding. The 1-million-token context window means a single pass can process roughly 750,000 English words or about 1.5 million Chinese characters—enough to cover the full text of the Three-Body Problem trilogy. For users who need to parse large code repositories, lengthy technical documents, legal contracts, or academic papers, this capability has immediate practical value.

Third-Party Benchmarks: A Widening Gap Between Performance and Cost

In an independent third-party comprehensive intelligence evaluation, the model scored 46.5 points. The previous highest score among open-source models was approximately 32.6 points, putting the new model ahead by more than 14 points. In holistic AI capability assessments, a gap of this size represents a generational difference rather than something that can be closed through fine-tuning.

Another independent evaluation institute benchmarked the model on terminal operation tasks—simulating autonomous completion of tasks in a virtual computer environment, including file operations, command execution, and script writing. The model scored 63.64%, ranking third among the 59 models tested, behind only two closed-source commercial models.

What is even more telling is the cost data. In the terminal operation test, the model’s average inference cost per question was $4.57, while the top-ranked closed-source model cost $13.20 per question—a difference of nearly three times. In other words, users can approach the capability level of leading closed-source models while paying significantly less for inference. This cost-performance advantage is especially important for developers who rely on high-frequency API calls.

A Refined Open Source License: MIT Base with Two Specific Restrictions

The license design for this release is worth examining closely. The main body follows the MIT license, meaning individuals and companies can freely use, modify, redistribute, and commercially deploy the model. However, two additional conditions are attached, each with a narrow and deliberate scope.

Restriction One: A Threshold for API Resellers

The terms state that if a company (or its affiliates) is selling API services based on this model and the group’s total revenue over any consecutive 12-month period exceeds $20 million, a separate commercial agreement must be signed with the model’s publisher before commercial deployment. The intent is clear: to prevent large cloud providers from directly commoditizing the original developer’s commercial API business.

For the vast majority of small and mid-sized development teams and startups, the $20 million annual revenue threshold is a relatively lenient standard. Even with substantial API call volumes, as long as the overall company revenue stays below that level, free commercial use remains permitted.

Restriction Two: Attribution Obligations for Super Apps

The second condition applies at the end-product application layer: if a product built on this model exceeds 100 million monthly active users, or generates more than $20 million in monthly revenue, the product interface must clearly display the model’s name. This requirement is similar to a citation convention in academic papers, aimed at ensuring brand visibility and credit for the model’s publisher.

It is important to emphasize that neither restriction applies to internal use within a company, nor to use via the model’s official products or certified inference service providers. In practice, the constraints target only large-scale API resellers and top-tier consumer applications.

Ecosystem Impact and Selection Guidance

Practical Advice for Developers

For enterprise developers planning to adopt this model, the following points deserve attention:

• Hardware planning: Given the massive total parameter count, an inference cluster of at least 8× A100/H100 GPUs (or comparable) is recommended. Quantization may also be viable to reduce memory footprint.
• Long-context use cases: If your business involves large-scale document parsing or extended dialogue history, the memory savings from sparse attention should be factored into your cost model.
• License compliance review: Carefully verify whether your current revenue and product MAU fall within the free commercial-use thresholds. If you are approaching those limits, it is advisable to contact the model provider early to discuss terms.

Where Open-Source Models Are Headed

Two trends can be observed from this release. First, the technical gap between open-source and closed-source models is shifting from “following” to “running in parallel,” and in some benchmarks, open-source models are already surpassing their closed-source counterparts. Second, licensing design is moving away from a simple permissive-versus-restrictive binary toward a more granular, scenario-based approach. This hybrid model—permissive core plus targeted restrictions—may well become the new normal for large open-source AI models.

For the broader AI technology stack, the continued emergence of ultra-large open-source models signals that the democratization of foundational model capability is still advancing. Enterprises no longer need to pretrain base models from scratch; they can build on a 2.8-trillion-parameter foundation and focus their resources on domain adaptation and application innovation. This is a strategic direction worth monitoring closely over the next twelve months.