The End of the Price War: LLM Competition Enters the Cost-Per-Task Era

1. Why the Price War Stalled: Sticker Prices No Longer Differentiate

Over the past two years, domestic LLM APIs have gone through multiple rounds of aggressive price cuts, with flagship models’ per-million-token prices falling repeatedly—often by multiples. But the price curve has reached a point where marginal effects are unmistakable. On one hand, hard inference costs—compute depreciation, electricity, bandwidth, and operations—set a floor below which prices cannot endlessly fall. On the other hand, as general capability gaps between models keep narrowing, “slightly cheaper than the other guy” is no longer a compelling reason for customers to migrate.

This has produced an intriguing pattern: a recent wave of new-generation models launched at prices largely unchanged from their predecessors, with only a few vendors continuing to cut rates. Third-party testing showed that the one vendor advertising nominal cuts saw essentially no change in real cost per task—longer outputs and heavier context offset the per-token concessions. Meanwhile, more vendors are choosing to keep sticker prices flat while driving down the amount customers actually spend to complete each task.

This means the industry’s unit of competition has fundamentally shifted: from “how much per million tokens” (the sticker-price war) to “how much per successfully delivered task” (the cost war).

2. Why Sticker Prices Mislead: Deconstructing Cost Per Task

A real business task is rarely completed by a single model call. A typical agent task usually includes task planning, multiple rounds of tool calls, long-document reading, intermediate reasoning, and final generation. Its actual cost can be written as:

C_task = Σᵢ [ (inᵢ − cachedᵢ) × p_in + outᵢ × p_out ] ÷ success rate

Variable definitions:

• inᵢ / outᵢ: input and output token counts for the i-th call;
• cachedᵢ: input tokens that hit the cache and are billed at a lower price (or free);
• p_in / p_out: unit prices for the model tier used in that call;
• success rate: the share of tasks passing on the first attempt; failure and retry costs must be amortized into successful tasks.

This formula explains why sticker prices mislead:

1. Context inflation: multi-turn dialogue and long documents make input tokens far exceed output tokens, so input-side engineering (how much to send, how to send it) affects the total bill more than output-side unit price;
2. Call count: an agent loop of 10 rounds versus 3 rounds can differ several-fold in cost even at identical unit prices;
3. Failures and retries: rate cards have no line item for “failure,” but invoices do. Two solutions with 90% and 70% success rates differ by roughly 30% in cost per successful task, all else being equal;
4. Model tier mix: which steps use small models and which require flagship models determines the weighted unit price.

Consider an illustrative comparison (figures are hypothetical): Solution A has a lower unit price but averages 5 calls per task with an 85% success rate; Solution B has a slightly higher unit price but averages 2 calls with a 97% success rate. On the total bill, Solution B very likely has the lower cost per successful task. This is the watershed between the sticker-price war and the cost war: customers pay for task completion, not for token consumption itself.

3. Six Technical Levers of the Cost War

1. Context Engineering: Spend Every Token Where It Counts

• History compression: in multi-turn dialogue, summarize stale turns into short memories and keep only the most recent few turns verbatim;
• Structured prompts: replace verbose natural-language preamble with field-based templates;
• On-demand retrieval: in RAG scenarios, control the number and length of injected chunks and use reranking to pass in only the most relevant content;
• Tool result trimming: filter fields and truncate large JSON blocks returned by tools before sending them back to the model.

def build_context(history, recent_k=3, budget=2000):
    recent = history[-recent_k:]
    older = history[:-recent_k]
    summary = summarize(older) if older else ""
    prompt = f"[History summary]{summary}\n" + "\n".join(recent)
    return clip_to_budget(prompt, budget)

2. Cache Reuse: Stop Paying Twice for Repeated Input

• Prefix caching: stable prefixes such as system prompts and knowledge-base documents reuse server-side KV computation, with cached portions billed at a significantly lower rate. In engineering terms, place stable content first and dynamic content last to improve hit rates;
• Semantic caching: for Q&A workloads, semantically similar requests reuse historical answers directly, bypassing model calls at near-zero marginal cost.

3. Model Tiering and Routing: The Right Model for the Right Job

• Cascaded scheduling: a small model answers first, escalating to a flagship model when confidence is insufficient or the task is complex;
• Step-level tier mix: use flagship models for planning and final generation, and delegate deterministic steps such as format conversion, information extraction, and intent classification to small models;
• Distilled small models: small models distilled from flagship models handle high-frequency simple tasks, pushing unit cost down by another order of magnitude.

def route(task):
    draft = small_model(task)
    if confidence(draft, task) >= 0.9:
        return draft
    return flagship_model(task)

4. Output Constraints: Control Output Tokens at the Source

• Set reasonable max_tokens limits to avoid rambling long answers;
• Use structured output constraints (e.g., JSON Schema) to cut explanatory filler;
• Trim few-shot examples—examples themselves are billed as input.

5. Agent Orchestration: Eliminate Needless Loop Rounds

• Plan first: produce an execution plan once, then execute step by step, avoiding step-by-step trial and error;
• Parallel execution: launch independent subtasks in parallel to shorten the chain and reduce repeated passing of intermediate context;
• Tool consolidation: merge multiple fine-grained calls into a single batch call to reduce model rounds;
• Cost circuit breakers: set round and budget caps per task, terminating abnormal loops promptly.

6. Inference-Side Optimization: Where Vendor Concessions Really Come From

What lets vendors drive down cost per task while holding sticker prices flat is progress in inference engineering: Mixture-of-Experts (MoE) architectures reduce activated parameters per forward pass; quantization and speculative decoding raise throughput per unit of compute; continuous batching and KV cache management improve GPU utilization. These optimizations compress vendors’ unit inference costs, creating room—without moving per-token prices—to shrink customers’ actual bills by consuming fewer tokens per task and shortening the chain.

4. How to Measure Properly: Build a Unit-Economics Ledger

When evaluating models, enterprises should abandon the habit of reading rate cards and instead run the numbers on their own real task sets:

1. Define a standard task set: sample from production traffic, covering typical difficulty and length distributions;
2. Log call details: model tier, input/output tokens, cached tokens, and final success status for each call;
3. Compute core metrics: cost per successful task = total spend ÷ number of successful tasks; average calls and token consumption per task; cache hit rate, small-model share, and failure cost ratio.

@dataclass
class Call:
    tier: str              # "flagship" / "small"
    input_tokens: int
    output_tokens: int
    cached_tokens: int = 0
    success: bool = True
def cost_per_success(calls, price):
    spend = sum(
        (c.input_tokens - c.cached_tokens) * price[c.tier]["input"]
        + c.output_tokens * price[c.tier]["output"]
        for c in calls
    )
    ok = sum(c.success for c in calls)
    return spend / max(ok, 1)

At the implementation level, mainstream cloud providers’ LLM platforms generally offer call details, usage statistics, and supporting capabilities such as logging and monitoring alerts (Baidu AI Cloud is one such example). Enterprises can build cost dashboards on these capabilities and manage cost per successful task as an SLA metric on par with latency and accuracy.

In addition, some industries have hard requirements around data residency. Private deployment or dedicated instances may not offer the lowest nominal unit price, but once compliance risk, data rework, and migration costs are folded into total cost of ownership (TCO), the overall equation may favor them. The cost war plays out not only in unit prices but also in the choice of deployment model and compliance architecture.

5. Three Takeaways for Enterprise Model Selection and Architecture

1. Budget by task, not by token: when scoping projects and reporting, switch the budget unit from “how many tokens” to “how much to handle one inquiry, review one contract, generate one report.” Only then do technical and business language align, and only then does cost reduction have a business anchor;
2. Make cost reduction an engineering loop: context engineering, caching, routing, and output constraints are not one-off actions. Use cost per successful task as the north-star metric, pair it with canary experiments and continuous regression testing, and prevent costs from quietly creeping back after model or prompt upgrades;
3. Put hidden costs on the balance sheet: failure retries, human fallbacks, compliance rework, and vendor lock-in all land in TCO. The lowest unit price is not necessarily the lowest total cost.

6. Outlook: From Selling Tokens to Selling Outcomes

The cost war has only just begun, and its endgame may be a further evolution of billing: once task pipelines and success rates are standardized enough, per-task billing—and even per-outcome billing—will become viable options, and the model-serving business will move from “selling compute” to “selling delivery.” For enterprise customers, this means more predictable cost structures; for the industry, it means competition shifting from capital-driven price subsidies back to head-to-head comparison of engineering efficiency and product capability.

The price war taught the market that LLMs are cheap; the cost war will teach the market that LLMs are worth it. The distance between cheap and worth it is exactly the battleground where every player will be competing for the foreseeable future.