Open-Source Open Weights Hit a New High: 2.8T-Parameter Flagship Model Fully Released, Sets Record in Agent Real-World Benchmark

A New Benchmark for Open Weights: 2.8 Trillion Parameters Go Fully Public

The competitive focus in the large language model (LLM) space is rapidly shifting from static benchmark scores toward accessibility and real-world usability. In early November, a leading AI lab fulfilled its commitment by uploading the complete weight files of its latest flagship reasoning model to major open-source repositories. This release goes beyond merely exposing core parameters; it also includes adapted deployment toolchain configurations. As a result, any team or individual developer with sufficient hardware resources can download the weights directly for local deployment, fine-tuning, or security auditing—without relying on a proprietary API endpoint.

According to the Hugging Face repository interface, the complete model checkpoint occupies approximately 1.56 TB, with the safetensors registry reporting around 2.78 trillion total parameters. This massive parameter count is complemented by a highly efficient Mixture-of-Experts (MoE) architecture that activates only 104 billion parameters per token, striking a strong balance between inference efficiency and model capacity.

Architecture Deep Dive: KDA and Gated DSA Hybrid Attention

The model introduces notable engineering innovations at the architectural level. Its structure comprises 93 layers: the first 69 layers utilize KDA (Key-Value Distributed Attention), while the remaining 24 layers implement Gated DSA (Gated DeepSeek Sparse Attention). This design replaces the previous generation’s uniform use of Gated MLA layers, with the core objective of optimizing computational efficiency in ultra-long-context scenarios.

Specifically, Gated DSA is a sparse attention mechanism equipped with a gating mechanism. In standard Transformer layers, the computational complexity of full attention grows quadratically with sequence length. Gated DSA mitigates this by introducing sparsified routing: it performs attention computation only on a subset of critical token combinations, while a learnable gating unit dynamically determines the extent of information retention. This approach maintains long-range dependency modeling while significantly reducing GPU memory footprint and computational latency—especially critical for applications requiring long-document processing or complex multi-turn Agent tasks.

The model also upgrades its vision encoding path, integrating a 4.01-billion-parameter vision transformer module. This enables native multimodal input, allowing the model to process both text and images simultaneously, thereby expanding its applicability to a broader range of multimodal use cases.

Quantization-Aware Training: The MXFP4 Storage and MXFP8 Activation Implementation

To control model footprint and deployment costs without incurring substantial accuracy loss, the model employs cutting-edge quantization-aware training techniques. The specific scheme uses MXFP4 (4-bit floating point) for weight storage, while activation values are computed in MXFP8 (8-bit floating point). This asymmetric quantization strategy is based on a key insight: weight parameter distributions are relatively stable and can tolerate lower-bit compression, whereas activations are prone to outliers during propagation and require higher precision retention.

By integrating quantization awareness during the pretraining phase, the model fully adapts to low-bit numerical representations, effectively mitigating the sharp accuracy degradation commonly associated with post-training quantization. This approach compresses the total 2.8T-parameter model repository to approximately 1.5 TB, making multi-GPU deployment feasible. Community feedback from third-party sources confirms that mainstream inference frameworks—such as vLLM and SGLang—have already published official deployment configuration examples for this model. Developers can directly pull the corresponding model configurations and inference scripts for immediate adaptation.

Local Deployment: Hardware Requirements and Cost Assessment

The release of open weights at this scale also implies a very high barrier for local deployment. Based on preliminary estimates from the technical community, a viable high-throughput local configuration would require eight next-generation enterprise accelerator cards (e.g., NVIDIA B300) with a combined GPU memory of roughly 2.3 TB. At current market rates, the hardware procurement cost alone is estimated between $550,000 and $650,000—excluding data center power, cooling, networking, and operational staffing expenses.

However, the high hardware threshold does not diminish the value of the open-weight ecosystem. For research institutions, full weight access enables deep model interpretability analysis, safety alignment studies, and domain-specific knowledge distillation. For enterprises operating under strict data residency or compliance requirements (such as finance or healthcare where data cannot leave internal infrastructure), private cluster deployment provides a viable path for proprietary model instantiation.

For the majority of developers without access to massive compute, the official API remains the most cost-effective integration method. The API is compatible with mainstream protocols (including OpenAI and Anthropic formats) and offers high-speed inference endpoints, with the vendor claiming 2 to 3 times faster throughput compared to the standard tier. Notably, developers can control the model’s reasoning depth via the reasoning_effort parameter (options: low, high, max; default is max). In multi-turn tool-calling scenarios, it is essential to correctly pass back the previous turn’s reasoning content (reasoning_content) and tool call information (tool_calls) to maintain conversation continuity.

Agent Performance Evidence: Measured Gains in Real-World Sessions

Beyond open weights, the model’s performance in actual Agent tasks deserves close attention. Unlike traditional static benchmarks, a well-known evaluation platform assesses models based on long-horizon, real-user Agent session data. The scoring mechanism computes a “net uplift” metric by identifying multiple behavioral signals: whether users explicitly confirm task success, positive or negative user sentiment toward the model, how well the model responds after user corrections, and whether shell commands execute successfully.

The evaluation results show that the new flagship model (in its strongest reasoning mode) achieved a net uplift of +5.1% across more than 17,500 real long-horizon sessions. This places it first among all open-weight models and 13th on the overall leaderboard. In the three key categories—Work, Code, and Chat—it holds the top position among open-weight models. Among the detailed sub-metrics, the Bash command execution reliability score stands at an outstanding +7.9%, again ranking first among open-weight models.

For context, the previous generation model scored +0.91% on the same evaluation system, ranking 6th overall. The leap from +0.91% to +5.1% reflects the successful architectural transition from full-attention Gated MLA to sparse-attention Gated DSA. In the Bash reliability dimension specifically, the previous model even registered a negative score of -4.5%, while the new model jumps to +7.9%. This significant improvement is undoubtedly attributable to deeper capability optimization in code generation, instruction comprehension, and error-correction alignment.

This type of evaluation, grounded in interactive real-world data, far more accurately captures a model’s true capability in complex toolchain invocation, dynamic environment adaptation, and multi-step logical reasoning than traditional “correct answer” matching. It also provides the industry with a more meaningful evaluation paradigm.

Summary and Outlook

The full release of a 2.8-trillion-parameter open-weight model marks a decisive industry shift—from closed API black boxes toward auditable, customizable AI infrastructure. While the hardware requirements place direct deployment beyond the reach of most small and medium-sized teams, the hybrid attention architecture and quantization-aware training methodology validated by this release will serve as important engineering references for future, even larger and more efficient model development. For developers, whether through flexible API access or deep open-source customization, an era of more capable and more reliable large models has clearly arrived.