2.8-Trillion-Parameter Open-Source LLM Released: Sparse Attention, MoE Economics, and License Terms Explained

The capability ceiling of open-source models is being pushed to a new level. A China-based AI research lab has, within the release window it previously committed to, published the complete weights of its latest flagship model: 2.8 trillion total parameters, a Mixture-of-Experts (MoE) architecture, a context window extended to 1 million tokens, and native image understanding. The release not only raises the parameter ceiling for open-source models, but also presents a well-defined commercial boundary scheme through its license design. This article analyzes the release across four dimensions—architecture, benchmark performance, cost structure, and license terms—to help developers and enterprises make informed technology decisions.

1. Architecture Evolution: The Trade-Off from Full Attention to Sparse Attention

The most notable architectural change in this release is the replacement of the attention mechanism. Compared with the previous version, the new model converts 24 layers that previously used full attention into a sparse attention scheme.

Full attention’s computational complexity grows quadratically with sequence length, causing memory usage and compute costs to rise sharply when handling long contexts. The core idea of sparse attention is simple: for each token, only a fixed number of the most relevant tokens are selected to participate in the attention computation—this model uses a threshold of 2,048 tokens. As a result, no matter how long the context grows, the attention computation cost per token stays within a controllable range.

The engineering significance of this trade-off:

• Predictable long-context costs: A 1-million-token window no longer implies unacceptable inference overhead, making scenarios such as long-document understanding and cross-file code analysis practically viable.
• Manageable memory pressure: The KV cache growth curve is significantly flattened, improving the feasibility of single-node long-context deployment.
• Accuracy trade-offs: Sparsification inevitably loses some global information interaction, so the model must compensate through training strategies. Public benchmark results suggest this compensation is effective.

It is worth noting that sparse attention is not a first for this team—it is an industry-proven, general-purpose technique. Applying it at the 2.8-trillion-parameter scale, the key challenges lie in training stability and consistency of the sparsity pattern—engineering challenges rather than algorithmic innovations.

2. MoE Activation Economics: 2.8 Trillion Parameters, 104 Billion Activated

With 2.8 trillion total parameters and only 104 billion activated per inference, the activation ratio is roughly 3.7%. Understanding how MoE activation works is key to assessing deployment costs.

MoE models split the feed-forward network into multiple experts, with a routing network selecting a small subset of experts for each token. The economics show up in three places:

1. Training: Total parameter count determines the model’s capacity ceiling, but each forward pass only computes the activated portion, so training compute is lower than for a dense model of equivalent scale.
2. Inference: Memory must hold all expert parameters (the main deployment barrier), but compute scales only with activated parameters, so throughput costs are close to those of a hundred-billion-scale dense model.
3. Load balancing: If expert routing skews, some experts overload while others sit idle, and real efficiency falls below theoretical values. This requires inference frameworks with expert parallelism and dynamic scheduling capabilities.

The practical impact for enterprises: a high memory barrier but a relatively low compute barrier. Self-hosting such a model requires a substantial accelerator cluster to host all weights, but with API access or managed inference services, the cost pressure shifts to the provider. For self-hosted deployments, evaluate whether your expert parallelism strategy and batching schedule can fully utilize the hardware.

3. Benchmarks and Cost Structure: The Capability-to-Cost Ratio

In a third-party comprehensive intelligence index, the model scored 46.5, about 14 points higher than the previous top-scoring open-source model. It is worth emphasizing that such composite indexes are weighted aggregates across multiple task categories; the score gap reflects a shift in the overall capability range rather than absolute dominance in any single task. Developers should still run targeted evaluations against their own business scenarios.

More informative are the independent results on terminal-operation tasks. On a benchmark that tests whether an AI can complete real operations in a command-line environment, the model scored 63.64%, ranking third among 59 participating models—behind two closed-source flagships, while another closed-source model ranked fifth. These benchmarks closely mirror real development work—file operations, script execution, environment configuration—so the results are directly relevant to applications such as automated operations and coding assistants.

Cost data is equally noteworthy: the model’s average cost per task is about $4.57, while a leading closed-source flagship costs $13.20—roughly three times as much. Near-frontier capability at one-third of the cost is the core argument for open-source models in cost-sensitive scenarios. That said, per-task cost varies with task complexity, retry counts, and context length, so production environments should measure against their own workloads.

4. License Design: Two Additional Boundaries on an MIT-Style Framework

The licensing strategy deserves as much scrutiny from legal and technical leads as the parameter count itself.

The license body follows the MIT license wording: individuals and companies may freely use, modify, and redistribute the model, including for commercial purposes. On top of that, two conditions are added:

Condition 1: For large-scale API resellers.
If a company or its affiliates are in the business of selling model API access, and the group’s total revenue over any rolling 12-month period exceeds $20 million, a separate agreement with the publisher is required before commercial use.

Condition 2: For super apps.
If a product built on the model exceeds 100 million monthly active users or $20 million in monthly revenue, the product must prominently display the model’s name in its interface.

The applicability boundaries of these two conditions need to be clear:

• Internal use within a company is unrestricted.
• Use through the official product or certified inference providers is unrestricted.
• In practice, only large vendors reselling APIs at scale need a separate agreement.
• For super apps, the obligation is limited to interface attribution—no fees are involved.

From an engineering and compliance perspective, the intent of this design is clear: preserve the openness of the open-source ecosystem while creating a negotiation entry point for “large-scale commercial resale of model capabilities.” For most developers and small-to-mid-sized businesses, the practical constraints are close to zero. Still, enterprises should complete two checks before launch: first, confirm whether their own or affiliated entities’ revenue touches the thresholds; second, if the product is consumer-facing with high growth expectations, assess the UI engineering cost of the attribution requirement in advance.

5. Adoption Guidance: Which Scenarios Fit

Based on the analysis above, models at this scale suit the following scenarios:

• Long-context-intensive tasks: repository-level code analysis, long-document review, and multi-turn complex conversations, where sparse attention’s cost advantages are clear;
• Cost-sensitive automation: terminal operations, batch script generation, and test case completion, where the per-task cost advantage stands out;
• Data-sensitive scenarios requiring on-premises deployment: full weight availability enables private deployment, keeping data in-domain;
• Teams building on top of the model: MIT-style terms permit modification and redistribution, making the model suitable for domain fine-tuning and product packaging.

Scenarios requiring careful evaluation include: small-to-mid-sized teams with limited GPU memory (the full-weight deployment barrier is high), tasks sensitive to sparse attention’s accuracy loss (targeted evaluation needed), and API reselling businesses (compliance confirmation required first).

Conclusion

The open-source release of a 2.8-trillion-parameter model signals that the capability gap between open-source models and closed-source flagships continues to narrow, while the cost gap remains significant. What deserves even more industry attention is the refinement of the licensing strategy: trading open access for ecosystem scale, while using two additional conditions to guard the commercial boundary. For engineering teams, the sparsification trade-off and MoE activation economics are more worth studying than the parameter number itself; for decision-makers, the license thresholds are a compliance step that must be completed before launch. With the open-source ceiling raised, the real competition will shift to engineering efficiency and ecosystem fit.