2.8T-Parameter Open-Source LLM Released: From Parameter Race to Scenario Delivery
A new open-source LLM with 2.8 trillion total parameters and a 1M-token context window has been released, ranking first among open-source models on a third-party benchmark. This article examines its likely MoE architecture, the systems engineering behind long-context support, post-training with reinforcement learning across hundreds of downstream domains, benchmark results in coding agents and 3D generation, and what the shift from parameter scale to scenario delivery means for model teams, enterprises, and developers.
2.8T-Parameter Open-Source LLM Released: From Parameter Race to Scenario Delivery
1. Background: Open-Source Model Scale Hits a New High
A Chinese AI team recently released a new-generation open-source large model with 2.8 trillion total parameters, setting another record for the parameter scale of open-source models globally. The model supports a context window of up to 1 million tokens. According to the team, the full model weights will be opened to the community about two weeks after release.
More notably, the model ranked fourth globally and first among open-source models on an authoritative third-party intelligence benchmark, with its composite score improving by 11 points over the previous generation. This result sends a clear signal: the capability gap between open-source models and closed-source flagship models is narrowing faster, and the focus of competition is shifting from “who has more parameters” to “who can actually deliver capabilities into real scenarios.”
2. Architecture: MoE Trade-offs in the Trillion-Parameter Era
A total parameter scale of 2.8 trillion is almost infeasible for inference under a pure dense architecture, so it is reasonable to infer that the model adopts a mixture-of-experts (MoE) architecture. The core idea of MoE is “large total parameters, small activated parameters”: the model contains a large number of expert subnetworks, and each token activates only a small subset of them during forward propagation.
A simplified gating and routing logic can be expressed as:
def moe_forward(x, experts, gate, top_k=8):# gate outputs a matching score for each expertscores = gate(x) # [num_experts]weights, indices = topk(softmax(scores), top_k)# Activate only the top_k experts and perform sparse computationout = sum(w * experts[i](x) for w, i in zip(weights, indices))return out
This design brings three engineering benefits:
- Controllable training and inference costs: computation scales with activated parameters rather than total parameters, making trillion-scale models feasible in practical deployment;
- Capacity and specialization at the same time: different experts can naturally specialize in different capability domains such as code, mathematics, and multilingual tasks;
- A clear expansion path: to add capacity later, the priority is to increase the number of experts rather than widening and deepening the entire model.
For users, understanding that “total parameters ≠ inference cost” is the first prerequisite for evaluating this type of model. The real deployment cost depends on the number of activated parameters, KV cache usage, and context length.
3. Million-Token Context: The Systems Engineering Behind the Capability
A 1 million-token context means the model can read hundreds of pages of technical documentation, an entire medium-sized code repository, or the complete trajectory of a multi-turn long-horizon task in one pass. However, “supporting long context” and “using long context well” are two different things. The latter depends on a full set of systems engineering work:
- Position encoding extrapolation: scaling, interpolation, or frequency-design techniques for rotary position embedding (RoPE) are needed so that the model maintains attention resolution at positions far beyond the pretraining length;
- Inference efficiency optimization: attention computation grows quadratically with long sequences, so engineering practice usually combines sparse attention, chunked computation, KV cache compression, and other methods to control latency and GPU memory;
- Long-context training strategies: gradually expanding the training window and mixing long and short samples help prevent the model from becoming “top-heavy and tail-heavy” on long texts—remembering only the beginning and end while losing middle information.
For developers, the most direct value of ultra-long context is reducing errors from RAG retrieval and stitching. Reading an entire repository directly is often more complete than chunk-based retrieval, but it is also more expensive. In practice, teams should trade off “retrieval augmentation” and “direct long-context reading” according to task characteristics.
4. Post-Training: Reinforcement Learning Extended to Hundreds of Downstream Domains
According to the official disclosure, the model is a post-training-enhanced version of the previous generation. Post-training compute has increased substantially, and reinforcement learning training has been extended to hundreds of downstream domains. To support this scale, the team rebuilt its reinforcement learning framework, long-context inference pipeline, training efficiency, sandbox infrastructure, and scheduling system, with the goal of converting every additional unit of compute into stable capability gains.
This path represents a mainstream paradigm shift in current large-model evolution: pretraining determines the capability ceiling, while post-training determines how much of that capability is delivered. Especially in scenarios with “verifiable rewards” such as code, mathematics, and tool calling, reinforcement learning with verifiable rewards (RLVR) has been shown to steadily improve a model’s reasoning depth and execution reliability. Extending RL from a few domains to hundreds of domains is difficult not because of the algorithm itself, but because of engineering:
- Large-scale parallel sandboxes are needed to safely execute model-generated code and obtain real feedback;
- An efficient scheduling system is needed to allocate GPU resources across different task types;
- The model must be prevented from “gaming” a single reward signal (reward hacking), so that generalization is preserved.
5. Benchmark Performance: Coding and 3D Capabilities Approach Top Closed-Source Models
Public test data shows that the model’s improvements are concentrated in several high-value directions:
1. Coding agent capabilities. On a benchmark that evaluates terminal coding agents, the model scored 65.7%, less than 1 percentage point behind the 66.4% of a top closed-source model. On an internal long-horizon code benchmark, its score rose from 55.95% in the previous generation to 61.2%. This means open-source models already have the potential to build complete projects from scratch. In an official demo, the model used a mainstream game engine to generate an end-to-end 3D action-adventure game featuring multiple exploration areas and multi-stage boss fights.
2. 3D modeling and creation capabilities. On a benchmark that generates CAD code and reconstructs 3D models from multi-view rendered images, the model scored 92.7%, approaching the roughly 96% of a top closed-source model. The practical significance is that, given a text description, a sketch, or a reference image, the model can infer an object’s shape, proportions, and spatial relationships and reconstruct it into a usable 3D asset. This opens a new path for “AI-assisted 3D content production” in industrial design, game development, digital twins, and other scenarios.
For developers, what matters more than the scores is the capability structure reflected by the benchmarks: long-horizon task execution, tool calling, and cross-modal generation. These are precisely the core gaps that must be closed for agents to be deployed in practice.
6. Industry Implications: From Parameter Race to Scenario Delivery
It is worth noting that after entering the new model cycle, the team has publicly stated that its strategic focus has shifted from the parameter race to synchronized delivery across three lines: technology, scenarios, and revenue. This shift is instructive for the broader industry:
- For model teams: the marginal returns from parameter scale are diminishing, and post-training quality, inference cost, and scenario fit are becoming new competitive barriers;
- For enterprise users: model selection logic should shift from “leaderboard ranking” to “task fit,” with small-scale real-world tests based on specific workloads such as code, documents, and content creation;
- For the developer ecosystem: the combination of open weights, ultra-long context, and agent capabilities lowers the barrier to building vertical applications. Enterprises can deploy open weights in private environments and use general cloud infrastructure—such as container platforms, object storage, logging, and monitoring and alerting—to build a complete chain from model inference to business integration, balancing data compliance and cost elasticity.
7. Conclusion
2.8 trillion parameters, million-token context, and open weights—together, these three keywords mark the entry of open-source large models into a new stage of “ultra-large scale plus strong delivery capability.” Parameter records will eventually be broken again, but the real dividing line is who can convert model capabilities into productivity in real scenarios stably and at low cost. For developers, now is the best window to rebuild code assistants, document analysis, and content production pipelines based on open-source flagship models.
