Chinese LLMs: Navigating the Capability Gap and Compute Economics

I. Two Data Points, One Signal

In late October, an international evaluation agency updated its comprehensive AI intelligence leaderboard. A key methodological shift in this version was the introduction of more private test questions and real-world task scenarios, reducing reliance on public datasets. The results: the leading overseas closed-source flagship model scored around 58 points, the latest domestic closed-source flagship scored around 46 points, and two domestic open-weight models scored around 32 points.

Almost simultaneously, an overseas top-tier model lab updated its prospectus: for the first three quarters of 2026, revenue was approximately $33.2 billion, a roughly 13-fold increase year-over-year; in the same period, computing power and infrastructure expenditures were about $18.8 billion.

These two sets of data seem unrelated but are actually cause and effect. Benchmark scores measure “how strong the model is,” while capital expenditure reveals “how much it costs to become stronger.” The score gap is the result; the investment gap is the process. Looking at both together provides a complete picture of the actual situation facing domestic large models: the gap objectively exists, but its composition can be deconstructed, leaving room for targeted catch-up.

II. The “Crisis”: Two Gaps

1. The Capability Gap: From “Benchmark Gaming” to “Real-World Tasks”

First, consider the change in evaluation methodology. Public benchmarks have long suffered from two issues: first, data contamination, where test questions leak into training corpora before evaluation, inflating scores; second, saturation in differentiation, where top models converge on familiar questions, failing to separate themselves. Private question banks and real-world tasks (multi-step tool calling, long-horizon code engineering, complex document processing, prompt interference resistance, etc.) address these exact shortcomings, which is why the new leaderboard is more valuable for reference.

Under this evaluation criteria, a ~12-point gap cannot be linearly understood as “20% worse performance.” Scores in real-world tasks often exhibit a threshold effect. A model scoring 46 points might be usable for most conventional tasks, but it will frequently “give up halfway” on long-chain reasoning, self-correction, and task persistence, requiring human takeover. This directly determines the divergence in product forms: one type of model can support “unattended” automated workflows, while the other is better suited for “human-in-the-loop” assistive scenarios.

The direct takeaway for developers is that model selection cannot rely solely on aggregate scores. You must look at sub-scores across target task dimensions and, more importantly, conduct empirical testing on your own business data. Two models with similar aggregate scores might differ significantly in specific areas like coding, tool calling, and long-context handling.

2. The Capital Gap: Compute Spend as a Leading Indicator

Now look at the other set of numbers. The ~$18.8 billion in compute and infrastructure spend over nine months covers large-scale training clusters, supporting data centers and power, and resource reserves for next-generation models—this money buys “future model capability.” More noteworthy is the revenue structure: ~$33.2 billion in revenue against ~$18.8 billion in spend means the commercial flywheel is already spinning—revenue feeds back into compute, compute produces stronger models, and stronger models support higher pricing and more enterprise contracts.

Currently, the financing and revenue scale of domestic models is not in the same league. Fighting a frontal war of attrition in the most capital-intensive “frontier foundation model” arena yields a low probability of success. This is the fundamental nature of the “crisis.”

However, a sober judgment must be maintained: capital expenditure is a leading indicator, not a final verdict. Tech history repeatedly features the same script—when algorithmic efficiency experiences an order-of-magnitude leap, old heavy-asset advantages are rapidly diluted. This leads to the “opportunity” side.

III. The “Opportunity”: Three Asymmetric Paths

1. Open Weights: Trading Ecosystem Leverage for Time

The two domestic open-weight models scored around 32 points. While their absolute scores are not advantageous, their strategic value lies in three attributes:

• Reproducible: Public weights mean capabilities can be independently verified and audited. The community can conduct research and improvements on them, and those improvements can feed back into the base model itself.
• Customizable: Enterprises can fine-tune and distill on their proprietary data to obtain vertical models tailored to their business. In many industry scenarios, a “mid-tier base + deep customization” can outperform a “top-tier base + zero customization.”
• Privatizable: Data-sensitive industries like finance, healthcare, and government inherently require local deployment. Open weights provide a realistic and viable alternative path for high-end capabilities at this stage.

Open weights also create an ecosystem flywheel: the more derivative models there are and the more active the community, the deeper the base model’s industry penetration. Essentially, this is an asymmetric strategy trading “ecosystem share” for “capability catch-up time”—a lag in benchmark scores can be transformed into a lead in ecosystem adoption.

2. Engineering Efficiency: Hedging the Compute Gap with Algorithmic Leverage

Latecomers cannot compete on capital, but they can compete on “unit cost of intelligence.” For the same task quality, whoever has lower unit cost and lower latency holds the advantage. The industry-verified efficiency toolbox includes:

1. Sparsification Architecture (MoE): Only activating partial parameters during inference, drastically reducing inference costs at equivalent quality.
2. Quantization and Distillation: Compressing flagship capabilities into smaller models to handle long-tail requests.
3. Inference Runtime Optimization: KV cache reuse, speculative decoding, and continuous batching to maximize hardware utilization.
4. Request Routing: Distributing requests to different tiers of models based on task complexity.

Among these, request routing is the easiest for engineering teams to implement first. The schematic logic is as follows:

def route_model(task) -> str:
    """Route to different model tiers based on task complexity to drive down unit cost"""
    complexity = estimate_complexity(
        task,
        signals=[
            task.chain_length,    # Reasoning chain length
            task.tool_calls,      # Number of tool calls
            task.context_tokens,  # Context size
            task.risk_level,      # Business risk level
        ],
    )
    if complexity >= 0.8:
        return "flagship"     # Long-horizon reasoning, high-value tasks
    elif complexity >= 0.5:
        return "mid_tier"     # Routine business tasks
    return "lightweight"      # Light tasks like classification, extraction, rewriting

Combined with caching strategies (direct hits for repeated requests) and fallback degradation, tiered routing can reduce overall inference costs by an order of magnitude with almost no loss in premium user experience. When the primary axis of industry competition shifts from “whose model is the strongest” to “whose unit intelligence cost is the lowest,” the value of engineering capabilities will be repriced—this happens to be a traditional strength of domestic engineering systems.

3. Application Depth: Transforming the Base Model Gap into an Engineering Problem

A 12-point gap in the base model does not equate to a 12-point gap in business value. Application-layer engineering can largely flatten this gap:

• Retrieval-Augmented Generation (RAG): Letting the model “answer with reference materials” reduces reliance on parametric knowledge and mitigates hallucinations.
• Workflow Orchestration: Breaking long tasks into verifiable short steps, avoiding long-horizon reasoning shortcomings, with fallbacks at every step.
• Output Validation and Retry: Using structured constraints and automated scoring to guard the lower bound of quality.

The cloud infrastructure supporting this system is quite mature domestically. GPU container platforms handle elastic inference and auto-scaling, object storage consolidates corpora and vector data, message queues decouple inference pipelines, and logging and monitoring services ensure online quality. Domestic cloud platforms, represented by Baidu AI Cloud, generally provide these universal categories, allowing enterprises to focus their energy on data, workflows, and evaluation rather than repeatedly building infrastructure.

Domestic players also have an added layer of scenario dividends: deep industry knowledge accumulation, high proprietary data barriers, and clear localization and compliance requirements. These advantages are precisely tied to the “deploy locally, go deep into industries” path, creating a natural asymmetric competition against the “general capabilities of overseas flagships.”

IV. Three Actionable Recommendations for Enterprises and Developers

1. Build a Private Evaluation Set; Don’t Just Look at Leaderboards

Leaderboards are a sample; your business is the full picture. It is recommended to consolidate 100–500 real business use cases with clear grading standards, forming a regression-testable evaluation asset:

def evaluate(model, eval_set) -> dict:
    """Run candidate models against a private eval set and output comparable metrics"""
    report = []
    for case in eval_set:
        pred = model.invoke(case.prompt)
        report.append({
            "case": case.id,
            "pass": rubric_grade(pred, case.rubric),
            "latency_ms": pred.latency_ms,
            "cost": pred.token_count * model.unit_price,
        })
    return {
        "pass_rate": mean(r["pass"] for r in report),
        "p95_latency_ms": percentile([r["latency_ms"] for r in report], 95),
        "total_cost": sum(r["cost"] for r in report),
    }

This evaluation set will grow into one of your team’s most valuable assets. Whether switching models, upgrading versions, or modifying prompts, you can obtain comparable data under the same yardstick, turning “it feels better” into “the pass rate increased from 72% to 86%.”

2. Tiered Selection and Dynamic Routing

Give light tasks to lightweight models, hard tasks to flagship models, and sensitive tasks to open-weight models that can be privately deployed. Routing strategies transform a “capability gap” into a “cost structure problem”—this is more realistic and controllable than waiting for a universal flagship to appear.

3. Calculate the Full-Chain Cost

Token unit price is merely an explicit cost. The true cost should also factor in: the impact of latency on user experience, cache hit rates, human investment in fine-tuning and evaluation, and the rework costs of failed outputs. Making decisions based on the “fully loaded cost per effective task” often yields different conclusions than simply looking at benchmark scores.

V. Conclusion: The Gap is Dynamic

Returning to the initial data: the ~12-point benchmark gap and an order-of-magnitude capital expenditure gap are the “crises” that must be faced. However, the ecosystem leverage of open weights, the algorithmic leverage of engineering efficiency, and the scenario leverage of application depth provide three catch-up paths that avoid frontal attrition.

What determines the endgame is not the score on a single leaderboard update, but two speeds: the rate at which intelligence costs decline, and the rate at which the application ecosystem flourishes. The former is driven by algorithms and engineering, while the latter is driven by developers and scenarios—these two factors happen to be the strengths of the domestic ecosystem. For practitioners, there is no room for blind optimism or pessimism right now. The correct posture is to deconstruct the gap into specific engineering problems, optimize them item by item, and use time and ecosystem dynamics to digest the generational gap.