MoE Multimodal LLM Benchmarked: Near-Closed-Source Flagship Performance at a Fraction of the Cost

In AI development, competition among large language models has moved well beyond parameter counts. Engineering efficiency, inference cost, and real-world task fit now matter just as much as raw capability. A leading Chinese AI company recently released a new flagship multimodal model that claims around 3 trillion total parameters while pricing itself at roughly one-fifth the cost of top closed-source alternatives. The announcement quickly drew attention across the developer community.

Rather than simply listing benchmark scores, this article takes a closer technical look at the model from three angles: architecture, measured performance, and cost efficiency.

1. Parameter Scale and Architecture: The MoE Advantage and the Anonymous Release Strategy

First, it is important to clarify what “3 trillion parameters” actually means. The model uses a Mixture of Experts (MoE) architecture. Its total parameter count is approximately 2.8 trillion to 3 trillion, but only about 104 billion parameters are activated during a single inference pass. This is not a conventional dense model, but a large collection of smaller expert networks working together.

The key benefit of this design is a fundamental improvement in computational efficiency. Under an MoE architecture, inference cost is strongly tied to activated parameters rather than total parameters. A dense Transformer model with 3 trillion parameters would require enormous compute resources and would be commercially impractical. Through sparse activation, the model retains a very high knowledge capacity while keeping per-query compute costs comparable to a dense model with hundreds of billions of parameters. This is the underlying reason the vendor can adopt such an aggressive pricing strategy.

The model also did not appear under its official name at launch. It was first released as an anonymous entry on third-party public evaluation platforms, where developers could test it without knowing its identity. The vendor confirmed ownership only after the results were collected. This has become a common practice among Chinese LLM vendors. By separating brand reputation from benchmark results, the approach allows the model to be judged purely on code output and logical reasoning. It also signals a degree of confidence in the underlying capabilities.

2. Core Capability Benchmarks: Coding, Reasoning, and Multimodal Performance

The following analysis is based on a small set of key benchmark results published in the official technical report. The model is compared with two widely recognized closed-source frontier models, referred to here as Flagship A and Flagship B.

1. Terminal and Agentic Capabilities: Strong Tool-Calling Performance

In Terminal-Bench 4.0, which evaluates an agent’s ability to execute real terminal commands, the model achieved 65.7%, clearly outperforming Flagship A at 58.2% and coming within 0.7 percentage points of Flagship B at 66.4%.

This is a meaningful result. The benchmark is not simple question answering. It requires the model to complete complex file operations, code debugging, and environment configuration tasks in a Linux environment. The score suggests that the model’s instruction following and structured output handling, such as JSON-based tool calls, are close to the current industry ceiling. It is especially relevant for unattended automation and complex script generation scenarios.

2. Long-Horizon Coding: Early Signs of an AI Software Engineer

SWE-Marathon v1.1 simulates the process of resolving real GitHub issues. It tests the model’s ability to edit across multiple files while maintaining logical consistency. The model achieved 51.7%, a clear lead over Flagship B at 46.3%. This indicates that in real development tasks requiring dozens of function edits and consistent project-level style, the model demonstrates not only code generation capability but also the endurance and precision needed for codebase maintenance. For automated refactoring and bug fixing within enterprise codebases, this benchmark carries strong practical weight.

3. General Knowledge Breadth: Approaching the Closed-Source Leaders

On Agents’ Last Exam, which evaluates world knowledge and complex reasoning, the model scored 43.2, higher than a third-party closed-source model at 32.8 and essentially on par with Flagship B at 42.6. In pure knowledge retrieval and logical reasoning, it has entered the global first tier. However, this is not a decisive advantage by itself, because general knowledge performance is only one dimension of commercial usefulness.

4. Multimodal 3D Reconstruction: A Remaining Gap

On BenchCAD, a visual-spatial understanding task involving 3D scene reconstruction, the model achieved 92.7% accuracy. While the absolute score is high, both Flagship A and Flagship B remained around 96%. Visual-spatial reasoning is a key barrier for text-centric LLMs evolving toward world simulation. The roughly 3-point gap indicates that in higher-level visual tasks involving geometric calculation and physical spatial mapping, the model still has visible room for improvement in multimodal alignment precision.

3. Trade-offs Behind the Aggressive Pricing

The model is not without limitations. Developers should carefully assess the engineering trade-offs behind the low price.

• Rate Limits and Concurrency Constraints: Low pricing is often accompanied by stricter rate limits. In production environments that require higher concurrency, additional dynamic billing may apply. Actual cost should be validated through stress testing rather than assumed from list pricing.
• Quality Degradation in Very Long Contexts: Although the model natively supports a 1-million-token context window, attention drift near the end of extremely long inputs remains an industry-wide problem. Developers should use summarization or chunking logic for critical sections instead of feeding an entire ultra-long code repository into a single request.
• No Open Weights: The model is currently provided only through an API. For teams that require private deployment or fine-tuning on custom data, this is not an ideal option.

4. Practical Guidance for Developers

For teams evaluating this type of high-performance, cost-efficient model, the following strategies are worth considering:

1. Dual-Track Deployment: Use the model for offline batch tasks that require massive background knowledge or very long contexts, such as repository-level audits and documentation knowledge base construction.
2. Lightweight Models for Latency-Sensitive Paths: In interactive UI scenarios where response latency is critical, keep a faster distilled model as a front-end filter.
3. Build Private Benchmarks: Do not rely solely on official scores. Create internal evaluation sets from industry-specific data such as medical records or financial contracts, then compare models based on the ratio of accuracy to per-call cost before making a final decision.

Conclusion

This release reflects a shift in the high-performance LLM race from pure intelligence comparisons toward maximizing compute density per unit of cost. Even with a visible weakness in 3D visual reasoning, the model already demonstrates competitive strength in code generation and tool calling, two areas that matter most to many development teams. For teams with limited budgets but demanding requirements, it is a compelling new option. Whether it can meaningfully challenge incumbent closed-source providers will ultimately depend on long-term adoption and performance in real production environments.