Large Model Price Wars Are Over—The Cost Optimization Battle Has Just Begun

Over the past year or so, the main competitive thrust of the large model industry has undergone a silent shift: from who can offer the lowest price to who can save the most cost. This shift will shape the industry landscape for years to come and directly influence the architectural decisions of every application team. This article first reviews how the price war ended, then systematically breaks down the playbook for the cost battle.

1. How the Price War Ended

1.1 From the “One-Yuan Era” to the “Free Era”

Looking back at the timeline of this round of price cuts, the pace exceeded most practitioners’ expectations. In mid-2024, a leading model vendor was the first to push the input price per million tokens down to the one-yuan level. Within a single week, multiple mainstream vendors followed suit in succession, and API quotes were repeatedly broken. The cuts did not stop there: entering 2025, Baidu announced that its conversational product would be fully free, and the next-generation flagship model it subsequently released pushed API prices further down to roughly one-hundredth of comparable international flagship products.

The direct result of this downward pressure: industry research institutions monitoring the market showed that by the second half of 2025, the normalized average price of domestic large model token services had settled at around 6.8 yuan per million tokens. For a market that just a few years earlier charged “several yuan per call,” this number essentially declared the end of an era.

1.2 Three Signals That Prices Have Bottomed Out

Why say the price war is “over” rather than “paused”? Three signals support this judgment:

• Average prices stabilized, cuts narrowed. A market that in the second half of 2024 still saw frequent “halving”-style price adjustments—by the second half of 2025, newly released models were generally priced close to existing price bands rather than continuing to break through the floor.
• Free coverage of the long tail eliminated price differentiation. When conversational products are fully free and lightweight models are priced low enough to be negligible, “cheaper” can no longer constitute differentiation, and the weight users place on price during selection continues to decline.
• Unit prices approached inference marginal cost. At a few yuan per million tokens, after deducting depreciation of compute, electricity, bandwidth, and operations, gross margin space is extremely limited. Continuing to cut prices means losing money on every token sold—commercially unsustainable.

1.3 The Real Legacy of the Price War

The price war is over, but it thoroughly changed the industry’s cost structure and usage habits: on one hand, the barrier to invocation was lowered to the point where anyone could use it, application innovation was fully stimulated, and token consumption grew exponentially; on the other hand, once unit price no longer mattered, “total cost” began to weigh on application teams—unit prices dropped by an order of magnitude, call volumes rose by two orders of magnitude, and bills actually grew thicker.

This is precisely the starting point of the cost battle: the price war was about who dared to cut prices; the cost battle is about who can do more with the same money.

2. Cost Battle Lesson One: Get the Token Math Right

2.1 Anatomy of a Bill

Many teams’ perception of cost stops at “what’s the unit price,” but a real bill is far more complex than unit price:

• Input and output are priced differently. On mainstream platforms, output token unit prices are typically several times those of input, and the output side is precisely where neglect is most common: chain-of-thought reasoning, long-form generation, and multi-turn conversations all significantly amplify output tokens.
• Longer context costs more. In multi-turn conversations, historical messages are billed repeatedly; in long-document Q&A, concatenating retrieved fragments causes input tokens to balloon rapidly.
• Hidden costs are often left out. Failed retries, regeneration after overly long truncation, secondary calls caused by format mismatches, and manual spot-check verification all push actual costs above theoretical costs.

2.2 A Reusable Estimation Model

Breaking the bill into computable functions is the first step in cost governance:

def estimate_cost(input_tokens: int, output_tokens: int,
                  input_price: float,   # yuan / million tokens
                  output_price: float,
                  cache_hit_ratio: float = 0.0,
                  cache_price_ratio: float = 0.3) -> float:
    """Estimate per-call cost (yuan), accounting for cache hit discounts"""
    cached = input_tokens * cache_hit_ratio
    fresh = input_tokens - cached
    return (fresh * input_price
            + cached * input_price * cache_price_ratio
            + output_tokens * output_price) / 1_000_000
# Example: retrieval-augmented Q&A, 8000 input tokens (75% cacheable), 1000 output tokens
cost = estimate_cost(8000, 1000, input_price=2, output_price=8,
                     cache_hit_ratio=0.75)
print(f"Per-call cost approx. {cost:.4f} yuan")

This simple function reveals a key fact: under a pricing structure where output unit price is several times input and cache can be billed at a discount, differences in application architecture design can make the cost of the same business outcome differ by an order of magnitude.

3. Three Major Technical Battlegrounds of the Cost War

3.1 Inference Side: Squeeze Every Bit of Compute

The reason prices could fall to today’s levels is rooted in systematic advances in inference engineering over the past two years—this is also the main battleground of vendors’ cost war:

• Continuous batching and scheduling optimization. Dynamically batching different requests together and supporting edge-to-edge request insertion significantly improves GPU utilization and dilutes per-token cost.
• Fine-grained KV cache management. Paged memory management reduces fragmentation, and prefix reuse (system prompts and shared context computed only once) directly cuts redundant computation.
• Low-bit quantization. FP8, INT8, and even INT4 quantization compress memory footprint and boost throughput under controllable accuracy loss—a core means of “getting more tokens from the same card.”
• Speculative decoding. A small model drafts and a large model verifies, producing multiple tokens per forward pass and improving decoding efficiency while maintaining quality.
• Mixture-of-experts architecture. Decoupling total parameters from activated parameters allows large capacity for good quality and small activation to control per-call compute cost.

These “internal skills” determine token output per unit of compute and represent the real efficiency race among vendors.

3.2 Model Side: Let the Right Model Do the Right Job

For application teams, the biggest cost lever is often not haggling over price but not overusing large models:

• Model tiering. High-frequency simple tasks such as classification, extraction, and rewriting go to lightweight models; only complex reasoning goes to flagship models.
• Cascaded routing. Let a small model answer first and escalate to a large model only when confidence is low—most requests are absorbed at the low-cost tier:

CONF_THRESHOLD = 0.85
def cascaded_call(prompt: str) -> str:
    """Cascaded call: small model first, escalate on low confidence"""
    resp = small_model.generate(prompt)
    if resp.confidence >= CONF_THRESHOLD:
        return resp.text                       # most requests end here
    return large_model.generate(prompt).text   # a few hard cases escalate

In practice, cascaded solutions typically trade minimal quality loss for substantial cost reduction—but the prerequisite is building an evaluation set first: without a measure of “quality per yuan,” any cost-reduction action is blind tuning.

3.3 Application Side: Every Token Not Generated Is a Token Saved

The application layer often offers the largest optimization space and is the easiest to overlook:

• Prompt slimming. Remove redundant instructions, merge duplicate examples, replace natural-language piling with structured templates—input tokens can commonly be compressed by 30% or more.
• Context caching. For reusable content such as system prompts, few-shot examples, and unchanging long documents, use the context caching capabilities commonly offered by mainstream platforms (such as Baidu AI Cloud and other large model service platforms); cached portions typically enjoy significant discounts.
• Semantic caching. For high-frequency similar questions (such as customer service Q&A), perform vectorized matching and return directly on a hit, skipping the model call.
• Batch inference. Offline tasks insensitive to latency (data cleaning, content labeling, batch summarization) go through batch channels, which the industry generally prices noticeably below real-time calls.
• Output control. Explicitly limit answer length, require bullet-point output, and use structured formats instead of verbose prose to directly reduce the most expensive output tokens.

4. Enterprise-Level Token Cost Governance: Treat Cost as an Architectural Metric

Once applications scale, cost governance must upgrade from “one-off optimization” to “continuous operations”:

1. Observable. Tag every call at three levels—application, scenario, model—and build token usage and cost dashboards so cost is as visible as latency and error rate:

def call_llm(task, model_tier, **kwargs):
    resp = client.chat(model=MODEL_MAP[model_tier], **kwargs)
    cost = calc_cost(resp.usage, PRICE_TABLE[model_tier])
    log_metrics({                      # cost instrumentation: three-dimension tagging
        "app": task.app, "scene": task.scene, "model": model_tier,
        "input_tokens": resp.usage.input_tokens,
        "output_tokens": resp.usage.output_tokens,
        "cost_yuan": cost,
    })
    return resp

1. Controllable. Set budgets and quotas by team and scenario, combined with rate limiting and abnormal-usage alerts, to prevent a single looping-call defect from burning a month’s budget overnight.
2. Optimizable. Regularly audit the top-cost scenarios, rank them by ROI of “business value / cost,” and prioritize high-cost, low-value segments; validate every cost-reduction change on the evaluation set to confirm no quality regression before canary rollout.
3. Sustainable. Incorporate cost weekly reports and cost-reduction initiatives into the R&D process, making “output per yuan” a design constraint on equal footing with quality and latency.

5. Conclusion: The Cost War Tests Systems Engineering Capability

The deciding factor in the price war was capital and resolve; the deciding factor in the cost war is engineering and architecture. For vendors, the competitive focus has shifted from the “price sheet” to token output per unit of compute, cache hit rates, and tiered invocation experience; for developers, token cost must become a first-class design constraint as important as quality and latency.

As model capabilities converge and price no longer constitutes differentiation, efficiency is the new moat. Whoever can consistently deliver the same value with less compute and fewer tokens will seize the initiative in the cost war ahead.