Architecture and Licensing Analysis of a 2.8T-Parameter Open-Source Model

The Open-Source Model Race Reaches a New Scale

The unit of measurement in the open-source model race is shifting from hundreds of billions of parameters to trillions. Recently, a research team released the complete weights of its latest flagship model on a public model hosting platform within its promised timeline, bringing the total parameter count to 2.8 trillion. This makes it one of the largest openly available open-source models to date. More importantly, the license attached to this release largely follows a permissive open-source framework, meaning most enterprises can use it commercially for free.

For developers and technical decision-makers, this event carries significance beyond simply having another model to choose from. It signals three simultaneous developments: the parameter ceiling for open-source models has been raised again, sparse architecture has moved from a training technique to an engineering standard, and open-source licenses are beginning to include finer-grained terms tailored for commercial scenarios. This article unpacks the model from four angles—architecture, performance, cost structure, and licensing boundaries—to provide a complete picture for evaluation and deployment.

Core Architecture: Sparse Experts + Sparse Attention

The Foundation: A Mixture-of-Experts Design

The model adopts a Mixture-of-Experts (MoE) architecture with 2.8 trillion total parameters, but only approximately 104 billion parameters are activated during each forward pass. This means GPU memory usage and computational load do not scale linearly with the total parameter count. Instead, they are governed by the number of activated parameters.

The core value of MoE lies in decoupling model capacity from inference cost: expert count expands knowledge capacity, while the routing mechanism controls per-step computation. For developers, the practical effect is straightforward: within the same inference budget, the model delivers a knowledge density significantly higher than that of a dense model with the same activation size.

Moving from Full Attention to Sparse Attention

Compared with the previous generation in the same model family, the most important architectural change in this open-source release is replacing 24 layers that previously used full attention with a sparse attention scheme. The mechanism can be summarized as follows:

• When computing attention for each token, the model no longer interacts with all tokens in the context.
• Instead, it first selects the most relevant tokens—around 2,048—and computes attention weights only for these tokens.

The logic can be represented in pseudocode as:

def sparse_attention(query, key, value, top_k=2048):
    # Compute relevance scores
    scores = relevance(query, key)
    # Keep only the top_k most relevant tokens
    indices = top_k_indices(scores, top_k)
    # Compute attention only on the filtered subset
    return attention(query, key[indices], value[indices])

The benefit of this design is especially visible in long-context scenarios: attention complexity drops from being quadratic in sequence length to being tied to the size of the filtered subset, significantly reducing compute overhead when processing contexts of up to one million tokens. The trade-off is that the selection step itself requires additional computation and places higher demands on the accuracy of relevance scoring, which is the main engineering challenge in implementing sparse attention.

Million-Token Context and Native Multimodal Support

The model supports a context window of 1 million tokens and natively accepts image inputs. Together, these two capabilities enable several typical use cases:

• Long-document understanding: Ingesting hundreds of pages of technical documentation, contracts, or research papers for cross-chapter reasoning.
• Repository-scale code analysis: Providing multiple source files as context for dependency analysis and refactoring suggestions.
• Mixed image-text tasks: Reading architecture diagrams, flowcharts, report screenshots, and acting on text instructions.

One caveat: a long context window does not equal lossless memory. At the million-token scale, retrieval quality, positional information decay, and bias in the selection mechanism can all affect downstream performance. It is advisable to run retrieval accuracy tests on critical tasks with real business data before making a final selection, rather than relying solely on the advertised context length.

Performance Positioning: General Intelligence and Agentic Tasks

According to publicly available evaluation results, the model ranks among the top open-source models on a general intelligence index, showing a clear advantage over the previous open-source leader. In an independent test using agentic task benchmarks, the model ranked in the top three out of dozens of evaluated models. The two models ahead of it were closed-source commercial offerings, and another leading closed-source model ranked fifth.

Two conclusions can be drawn from these results.

First, the capability gap between open-source and closed-source models continues to narrow on general tasks, but on certain difficult agentic operation tasks, top-tier closed-source models still hold an edge.

Second, evaluation scores must be interpreted in the context of task type. The general intelligence index focuses on broad knowledge and reasoning, while the agentic benchmark emphasizes completing multi-step operations in real command-line environments, which requires stronger tool invocation, error recovery, and state tracking. When selecting a model, developers should prioritize evaluation subsets that closely match their own task distribution rather than relying on overall rankings.

Cost Structure: Per-Task Cost Differences

Cost is a high-priority factor in model selection. Public test data shows that the model’s average per-task cost on the agentic benchmark is approximately $4.57, while a leading closed-source model in the same benchmark costs around $13.20 per task. The former is roughly one-third of the latter.

This gap stems from multiple factors:

• Activated parameter scale: Sparse activation reduces per-inference computation.
• Sparse attention: Long-context scenarios avoid unnecessary attention computation.
• Deployment autonomy: Open weights allow self-hosted inference, enabling cost reduction through batching, quantization, and speculative decoding.

For businesses with high daily call volumes, the per-task cost difference becomes substantial. When evaluating, it is useful to build a cost model along the following lines:

Per-call cost ≈ (input tokens × input price + output tokens × output price) / batching efficiency

Batching efficiency, quantization precision, and hardware utilization are all variables that can be actively optimized in a self-hosted setup. This is one of the core advantages of open weights over pure API calls: the cost curve can be shaped by engineering capability rather than dictated solely by vendor pricing.

License Breakdown: Permissive Core with Two Boundary Conditions

The licensing approach in this release deserves close attention from legal and architecture teams, arguably more than the parameter count.

Permissive Core

The main body of the license follows a permissive open-source formulation: individuals and enterprises are free to use, modify, redistribute, and commercially deploy the model. This is consistent with widely used permissive licenses and keeps compliance overhead minimal for most small and medium teams and internal projects.

Two Additional Restrictions

Beyond the permissive core, the license introduces two specific restrictions.

Restriction 1: Revenue threshold for API resellers.
If a company or its affiliates provides model API services to third parties, and the total revenue of the entire group exceeds $20 million in any consecutive 12-month period, the company must sign a separate agreement with the model developer before commercial use.

Restriction 2: Attribution requirement for super-app products.
If a product built on this model exceeds 100 million monthly active users, or generates more than $20 million in monthly revenue, the product must visibly display the model name in its interface.

Scenarios Not Subject to the Restrictions

The following use cases are exempt from the two restrictions:

• Internal use within a company.
• Use through the developer’s official product offerings.
• Use through certified inference service providers.

Practical Implications for Developers

From an engineering and compliance perspective, the license design sends a clear signal:

1. Small and mid-sized commercial use is essentially friction-free. Most startups and internal systems will not approach the $20 million revenue or 100 million MAU thresholds and can adopt the model without concern.
2. API resale is a focal point. If the business model involves packaging model capabilities into API services for external sale and the parent group has significant revenue, planning for a separate agreement should begin early.
3. Attribution is a branding obligation, not a usage restriction. At the super-app scale, the requirement is not a ban but an added visibility obligation.
4. Internal use and certified channels are the safe path. Teams that prefer to avoid additional clauses can simply use the model internally or through the developer’s certified inference channels.

Before adoption, it is recommended that enterprises run three checks: whether the business provides model APIs to external parties, whether group revenue in any consecutive 12-month period approaches the threshold, and whether the product’s MAU or monthly revenue could reach the trigger levels. If none of the three conditions apply, the model can be used under the permissive terms without further action.

Deployment Recommendations and Risk Notes

Based on the analysis above, here are several practical recommendations for engineering teams.

First, align on task types before choosing a model. Build a small evaluation set from real business data and verify long-context retrieval, multimodal input parsing, and multi-step agentic task success rates.

Second, include sparse architecture tuning in the deployment plan. The selection parameters for sparse attention, expert parallelism strategies for MoE, batch sizes, and GPU memory layout all significantly affect throughput and latency. Load testing should be completed before production launch.

Third, monitor cost and quality together. Per-task cost will fluctuate with input length, output length, and concurrency. It is advisable to track unit cost and task success rate in the same monitoring and alerting system, so that cost reduction efforts do not silently degrade critical task quality.

Fourth, address license compliance early. Define the intended usage form—internal use, external API, or end-user product—during the architecture design phase. This determines whether the additional clauses are triggered and avoids costly compliance rework after launch.

The arrival of a trillion-parameter open-source model pushes both the capability ceiling and the cost floor forward at the same time. But what ultimately determines deployment success is still the depth of architectural understanding, the rigor of evaluation, and the timing of compliance review. The more capable the tool, the more valuable sound engineering judgment becomes.