Beyond Parameter Count: How Domestic LLMs Are Competing on Cost-Efficiency and Engineering Practicality

Over the past year, the narrative arc of large language models has shifted noticeably: from “whose model has more parameters and tops more leaderboards” to “whose inference is cheaper and engineering more practical.” When a new model is released and community discussions are flooded with questions about “how expensive is it,” that itself signals that price has become an unavoidable dimension in any model evaluation framework. This article does not focus on specific brands; instead, it examines the engineering logic behind this round of model iteration from a general technical perspective, and how developers should rationally assess the relationship between “price cuts” and “capability.”

1. From Parameter Arms Race to Cost Per Token

Early competition among large models centered on parameter scale. From billions to hundreds of billions, and then to trillion-parameter MoE (Mixture of Experts) architectures, parameters were once treated as synonymous with capability. However, once model scale crosses a certain threshold, marginal returns begin to diminish while inference costs grow linearly or even super-linearly.

This raises a core question: inference cost per token is becoming the new benchmark for model competitiveness.

Consider a typical scenario: suppose a model charges 20 RMB per million input tokens, with output priced even higher. For an application consuming tens of millions of tokens per day, input costs alone run into hundreds of RMB daily. If a new version reduces input pricing while maintaining comparable capability, the savings for high-frequency callers are tangible.

A common misconception needs to be clarified here: a price cut does not equal a quality cut. Price reductions typically come from several sources:

• Architecture efficiency improvements: For example, more refined MoE routing strategies that activate only a subset of parameters per inference, reducing compute consumption;
• Inference optimization: Quantization, KV Cache reuse, continuous batching, and similar techniques that significantly improve GPU utilization;
• Engineering scale: As inference cluster scale grows, the marginal cost per unit of compute decreases;
• Commercial strategy: Using lower prices to drive call volume and ecosystem stickiness.

Therefore, evaluating whether a model is “expensive” cannot rely on absolute price alone—it should be assessed through a capability-to-price ratio.

2. The “Third Path” Between Open Source and Closed Source

The original discussion mentioned “the world’s first open-source 3-trillion-parameter model,” which touches on an important trend in the current LLM ecosystem: the boundary between open source and closed source is blurring.

Conventionally, open-source models are perceived as “a tier weaker in capability but self-deployable,” while closed-source models are “stronger but billed per token.” Today, however, some vendors choose to open-source extremely large models while also offering hosted API services. For developers, this model means:

1. Self-deployment verification: Developers can download weights and test capability boundaries in their own environments, avoiding “black box” anxiety;
2. Private deployment: For data-sensitive businesses, private deployment is a hard requirement;
3. Cost comparison: The cost of self-hosted inference clusters vs. calling hosted APIs can be quantitatively compared.

Note, however, that the “usability” of an open-source large model depends not only on weights but also on inference framework compatibility, VRAM footprint, concurrency throughput, and other engineering details. A 3-trillion-parameter MoE model, even if it activates only a subset of experts, still has substantial VRAM requirements and deployment barriers. For most small and mid-sized teams, hosted APIs remain the more realistic option.

3. How Should Developers Approach Model Selection?

Faced with an endless stream of new models, developers can easily fall into “evaluation anxiety.” Here is a practical selection methodology:

3.1 Define the Task Type

• Generation tasks (copywriting, code, dialogue): Focus on output quality, context length, and instruction-following capability;
• Understanding tasks (classification, extraction, summarization): Focus on accuracy, latency, and throughput;
• Multimodal tasks: Focus on image/audio understanding capability and token billing methodology.

3.2 Build Your Own Evaluation Set

Public leaderboards serve only as reference; what is truly convincing is a private evaluation set built around your business scenarios. It is recommended to sample 100–500 real examples covering both typical and edge cases, and compare multiple models using a unified prompt template.

# Pseudocode: unified evaluation framework illustration
def evaluate(model_client, test_cases, prompt_template):
    results = []
    for case in test_cases:
        prompt = prompt_template.format(input=case["input"])
        output = model_client.generate(prompt, max_tokens=512, temperature=0)
        results.append({
            "case_id": case["id"],
            "output": output,
            "latency": model_client.last_latency,
            "tokens": model_client.last_usage
        })
    return results

3.3 Calculate “Full-Pipeline Cost”

Cost is not just the token unit price. It also includes:

• Retry cost: Insufficient capability leads to repeated calls;
• Post-processing cost: Unstable output formats require additional parsing or manual verification;
• Latency cost: High latency degrades user experience and indirectly affects retention.

Therefore, a cheap but unreliable model may have a higher total cost.

3.4 Pay Attention to Context Window and Caching Mechanisms

In long-context scenarios, input tokens are often the largest cost component. If a model supports prompt caching, billing for repeated prefixes can be significantly reduced. This is especially critical in RAG (Retrieval-Augmented Generation), multi-turn dialogue, and similar scenarios.

4. General Practices for Inference Cost Optimization

Regardless of which model you choose, the following engineering techniques can help control costs:

Optimization	Principle	Applicable Scenarios
Prompt compression	Reduce redundant tokens	All scenarios
Result caching	Reuse outputs for identical inputs	High-frequency repeated queries
Tiered routing	Route simple tasks to smaller models	Mixed task flows
Streaming output	Reduce perceived latency	Conversational applications
Batching	Improve throughput utilization	Offline tasks

Among these, “tiered routing” deserves elaboration: not every request requires the largest and most capable model. Rules or a small model can perform intent recognition, routing simple requests to lightweight models and only sending complex requests to larger models. This model cascading strategy often saves 30% or more in practice.

5. A Rational View of “National Model Glory”

Discussions about whether domestic models can “rekindle national glory” are largely emotional. From a technical standpoint, the progress of domestic large models over the past year is real:

• On tasks such as Chinese-language understanding and code generation, the gap with top international models has narrowed considerably;
• In the open-source ecosystem, multiple domestic models have released weights, driving community innovation;
• On inference cost, intense competition has driven prices down continuously, benefiting developers.

But it is also important to recognize clearly that model capability is only one part of application success. Data quality, product design, engineering stability, and cost control also determine ultimate success or failure. A model may score high on leaderboards, but if its API is unstable, documentation unclear, and billing opaque, it will struggle to win long-term users.

6. Conclusion

Returning to the original question: how should one evaluate a new model? The recommendation is—spend less time watching launch events and more time on your own evaluation sets; worry less about unit price and more about full-pipeline cost.

Large models are moving from “technological spectacle” to “infrastructure.” For developers, the best era is not the moment when a particular model is crowned, but when models become cheap enough, stable enough, and interchangeable enough that we can redirect our energy back to business innovation itself.

As for “glory,” it does not belong to any single model—it belongs to those who actually put models to work and solve real problems.