The LLM Price War Is Over: Why Falling Token Prices Don't Lower Your Bill

    The headline price war in large language models has ended, but the real fight is over the cost of each completed task. This article traces how LLM pricing moved from across-the-board cuts to a tiered market, explains why falling list prices no longer translate into falling bills, and breaks down the true anatomy of task cost — token mix, context growth, reasoning overhead, agent round trips, retries, and tokenization effects. It then shows how to instrument cost per task with code, and gives seven practical levers — context governance, caching, tiered routing, output budgets, batching, agent loop limits, and multi-model routing — plus a four-phase rollout plan and a cost dashboard to keep optimization from degrading quality.

    I. Introduction: The war over list prices is over. The war over bills has just started.

    In May 2024, a Chinese LLM vendor pushed its input price down to RMB 1 per million tokens. Within a week, several cloud providers and leading model companies followed. For the next two years, price cuts barely stopped. In February 2025, Baidu announced that its public-facing AI chat product would become fully free starting in April; in March, a new-generation model pushed API pricing down to roughly one-hundredth of what overseas flagship models charged.

    By the second half of 2025, however, two developments broke the intuition that “a price cut equals a cost cut”:

    • List prices fell, but the average price went back up. According to monitoring by China’s Academy of Information and Communications Technology (CAICT), the normalized average price of domestic LLM token services landed at RMB 6.8 in H2 2025. The average for H1 2025 under the same methodology was RMB 13.46 — an increase of about 97.9%. The two figures cover different periods, but one down and one up already shows the point: list prices in some tiers are falling while the overall structure of transactions moves upward. “Same methodology” here means both measurements used consistent normalization rules.
    • List prices were cut, but task cost barely moved. In October 2025, several vendors released new models within ten days, and the visible price war disappeared. Only one of them cut list prices, and after third-party testers measured actual usage, its unit cost on typical tasks had hardly changed.

    Hence the dispute: list prices are falling, so why aren’t bills falling? The answer is that the basic unit of LLM cost has shifted from “the price per million tokens” to “the money actually spent to complete one task.” This article first reviews how the price war ended, then dissects the components of task cost from an engineering perspective, and finally lays out a practical playbook for cutting costs.

    II. How the price war ended: from across-the-board cuts to tiering

    2.1 Three phases

    Phase 1 (from May 2024): the value tier drops collectively.
    One vendor was first to push input pricing to RMB 1 per million tokens, and peers followed within a week. The cuts focused on general-purpose chat models, with the goal of lowering the barrier to adoption and competing for developers.

    Phase 2 (H1 2025): cuts and increases happen at the same time.

    • In May, a hardware vendor announced a permanent price cut across its in-house model family, with reductions of up to 99%.
    • In June, several general-purpose chat models saw cuts of more than 90%.
    • At the same time, the flagship tier started moving up together:
      • One Chinese LLM company raised prices three times within the year. Its CEO said the goal was to return prices to a normal level of commercial value.
      • An open-source-oriented vendor introduced peak/off-peak pricing for its new-generation model, raising the output price during peak hours from RMB 6 to RMB 27.
      • In July, a vendor known for long context priced its new flagship’s output at RMB 100. No one followed, and its revenue more than tripled within three months.

    Phase 3 (October 2025): the comparison shifts from list price to task cost.
    Between October 15 and October 24, five vendors released new models one after another. The product form was still a model plus an API:

    Vendor type Pricing move Cost claim
    Open-source-oriented vendor Unchanged from previous generation Per-task cost cut from $0.40 to $0.26
    Multimodal vendor Unchanged from previous generation Agent task cost reduced by roughly 70%
    Vendor with speech technology roots Unchanged from previous generation Emphasized keeping data on-premises
    General model vendor Unchanged from previous generation Per-task cost at about one-eighth of overseas flagship models
    Long-text vendor The only one to cut list price Third-party tests showed per-task cost barely changed

    Overall, price cuts concentrated in low- and mid-tier general models, while flagship models raised prices steadily. This is typical market stratification: base capabilities are commoditizing, and competition is shifting toward the capability ceiling and unit task cost.

    2.2 The limits of the normalized average price: it measures list price, not bills

    One clarification is needed: the RMB 6.8 average is derived by normalizing each model’s published list price at release. It is not the average of what users actually pay. Using it as a cost baseline introduces at least the following biases:

    1. Input and output prices are asymmetric. For most models, the output price is several times the input price, and a normalized average cannot reflect each model’s input/output token mix.
    2. Cache pricing is ignored. For models that support prefix caching, billed cost for cache hits may be only a small fraction of the original input price — something the list price does not show.
    3. Reasoning models introduce thinking tokens. The thinking process is billed like output, and this is not broken out separately in the list price.
    4. Context lengths differ enormously. Context windows range from tens of thousands to hundreds of thousands of tokens, and how much long context a real task actually consumes directly changes bill structure.

    So for enterprises, the normalized average is useful for observing market trends, but it cannot predict your own bill.

    III. List prices fell, so why didn’t bills fall? The anatomy of task cost

    3.1 The basic task cost formula

    Broken down, the cost of one complete task can be written as:

    1. Task cost = Σ(model call fees) + Σ(tool/retrieval call fees)
    2. + retry overhead from failures + infrastructure overhead
    3. Single model call fee = (uncached input tokens × input unit price)
    4. + (cache-hit tokens × cache unit price)
    5. + (output tokens × output unit price)

    The unit price is just a coefficient in the formula. What actually determines the bill is the number of tokens and how they are distributed across billing items. If you halve the list price but token usage doubles, the bill goes up instead.

    3.2 Five cost amplifiers that are easy to overlook

    (1) Context accumulates across turns.
    In chat-style applications, the nth request usually has to carry the content of all previous turns. Assume 500 new input tokens per turn and 300 output tokens per turn. After 20 turns of conversation:

    • Cumulative input tokens: 500 × (1 + 2 + … + 20) = 105,000;
    • Cumulative output tokens: 6,000;
    • Input volume is roughly 17.5 times the output volume.

    The longer the context, the more quadratically the input side grows. This is where “input price cuts” pay off most and “output price cuts” matter least. Yet many teams do the opposite and only negotiate on output pricing.

    (2) Chain-of-thought and reasoning overhead.
    Reasoning models generate a large amount of intermediate thinking before giving the final answer, and that content is billed the same as ordinary output. A classification task that normally takes a few hundred tokens can balloon to thousands of output tokens once deep reasoning is enabled. If the reasoning level is turned on across the board, cost scales up by orders of magnitude.

    (3) Tool-call round trips in agents.
    Every time an agent calls a tool, it has to write the tool’s result back into the context and request the model again. A single task involving search, file reads, and running tests commonly involves a dozen or more round trips — and each round trip re-pays the input cost of the entire context.

    (4) Retries and invalid output.
    Truncated output, JSON parsing failures, failing unit tests, and invalid tool parameters all trigger retries. This cost is completely invisible in a cost model that only looks at list prices, but it often accounts for a substantial share in agent-style workloads.

    (5) The normalization factor.
    Chinese, code, and English can differ considerably in token count for the same semantic content, and the exact ratio depends on the tokenizer. Extrapolating cost linearly from token counts easily leads to wrong conclusions.

    IV. Measuring task cost: get the accounting right first

    Cost reduction starts with measurement. Many teams can only see the total bill and cannot answer “which business line, which type of task, which single call is burning the most money.” The recommended approach is to build task-centric cost accounting along the following steps.

    4.1 Define the task boundary

    First define what “one task” means. There are two common definitions:

    • One user request, from initiation to return;
    • One agent, from receiving a goal to producing a result.

    Whether multiple requests within the same session count as one task must be specified in advance, otherwise data from different teams cannot be compared.

    4.2 Full-chain instrumentation

    For every model call, tool call, and retrieval, record the following fields and link them through trace_id and task_id:

    • model: the model tier or name;
    • input_tokens, cached_input_tokens, output_tokens, reasoning_tokens: usage of each token type (use the usage actually returned by the provider);
    • latency_ms: elapsed time;
    • status: success, truncated, format error, timeout, and so on;
    • billing price snapshot: record it at call time, to avoid historical data being distorted by later price changes.

    Illustrative code follows (the code only shows structure and does not represent any specific API):

    1. # Illustrative code: units are RMB per million tokens, values are examples only
    2. PRICE_TABLE = {
    3. "small": {"input": 0.8, "cached_input": 0.08, "output": 2.0},
    4. "mid": {"input": 2.0, "cached_input": 0.20, "output": 8.0},
    5. "flagship": {"input": 4.0, "cached_input": 0.40, "output": 16.0},
    6. }
    7. class TaskCostLedger:
    8. """Aggregate cost per task, returning the actual fee and token structure."""
    9. def __init__(self, task_id: str, max_cost: float = 0.5):
    10. self.task_id = task_id
    11. self.max_cost = max_cost
    12. self.cost = 0.0
    13. self.calls = 0
    14. self.input_tokens = 0
    15. self.cached_input_tokens = 0
    16. self.output_tokens = 0
    17. def record(self, model: str, usage: dict):
    18. p = PRICE_TABLE[model]
    19. uncached = usage["input_tokens"] - usage.get("cached_input_tokens", 0)
    20. fee = (
    21. uncached * p["input"]
    22. + usage.get("cached_input_tokens", 0) * p["cached_input"]
    23. + usage["output_tokens"] * p["output"]
    24. ) / 1_000_000
    25. self.cost += fee
    26. self.calls += 1
    27. self.input_tokens += usage["input_tokens"]
    28. self.cached_input_tokens += usage.get("cached_input_tokens", 0)
    29. self.output_tokens += usage["output_tokens"]
    30. def budget_left(self) -> float:
    31. return self.max_cost - self.cost
    32. def summary(self) -> dict:
    33. return {
    34. "task_id": self.task_id,
    35. "cost": round(self.cost, 5),
    36. "calls": self.calls,
    37. "input_tokens": self.input_tokens,
    38. "cached_input_tokens": self.cached_input_tokens,
    39. "output_tokens": self.output_tokens,
    40. "cached_ratio": round(
    41. self.cached_input_tokens / max(self.input_tokens, 1), 3
    42. ),
    43. }

    4.3 Quality gates and task success rate

    Cutting cost without regard for quality just shifts the cost onto rework, complaints, and manual fallback. Before and after each optimization, measure two categories of metrics on the same task benchmark:

    • Quality metrics: task success rate, for example unit test pass rate, structured output parse success rate, and human spot-check pass rate;
    • Cost metrics: average cost per task, P95 cost per task, and retry rate.

    4.4 Using regression cases to hold the line on “lower cost, same quality”

    Run the same batch of cases before and after the optimization. The pseudocode below shows a minimal quality-versus-cost comparison framework:

    1. def compare_runs(baseline: dict, candidate: dict, cost_tolerance=0.05) -> bool:
    2. """Allow the new approach to ship only if quality holds and cost improves clearly."""
    3. quality_ok = candidate["pass_rate"] >= baseline["pass_rate"]
    4. quality_gap_ok = (candidate["pass_rate"] - baseline["pass_rate"]) >= -0.01
    5. p95_ok = candidate["p95_cost"] <= baseline["p95_cost"] * 1.1
    6. cost_ok = candidate["avg_cost"] <= baseline["avg_cost"] * (1 - cost_tolerance)
    7. return quality_ok and quality_gap_ok and p95_ok and cost_ok

    V. Cost levers: seven handles that go from list price to bill

    5.1 Context governance: the main battlefield of the cost war

    For long conversations and agent tasks, cutting repeated input is often more effective than pushing unit prices down. Common techniques include:

    • Sliding window and summarization: compress early conversation into a structured summary that keeps only the task goal, constraints, and key conclusions;
    • Retrieval instead of full injection: replace “stuffing an entire document into the prompt” with on-demand retrieval of relevant fragments, and control the returned length;
    • Tool result trimming: keep only the key fields from successful calls; write large logs and full files to disk first, then let the model read fragments on demand.

    The key point: every token in the context is being paid for on the bill, and a good portion of it is content the model never uses.

    5.2 Caching: pay once for a repeated prefix

    A common industry practice is to put stable system prompts, tool definitions, few-shot examples, and long documents at the front of the prompt. That way they hit the prefix cache, and the repeated portion usually costs only a fraction of the original input price; meanwhile the context stays stable and avoids frequent cache invalidation.

    1. import hashlib
    2. def cached_chat(system: str, documents: list[str], question: str, ledger):
    3. # Stable prefix: system prompt and reference material concatenated in a fixed order,
    4. # guaranteeing the prefix is byte-for-byte identical.
    5. prefix = system + "\n" + "\n".join(documents)
    6. prefix_hash = hashlib.sha256(prefix.encode()).hexdigest()
    7. # Note: this only illustrates structure. In practice, store prefix content in object
    8. # storage or a vector store and fetch the cached result on a hit, avoiding repeated
    9. # retrieval and assembly.
    10. result = call_model(prefix + "\n\nQuestion: " + question, ledger=ledger)
    11. return prefix_hash, result

    In conversational scenarios, for repeated requests of the form “system prompt + high-frequency question,” you can add an application-level result cache. Two cautions apply:

    • Exact caching is only suitable for highly deterministic scenarios, such as fixed-template Q&A and static document extraction;
    • For semantically similar questions, when using fuzzy matching, set a similarity threshold and an expiration time to prevent incorrect reuse.

    5.3 Tiered routing: let the right model do the right job

    Not every request needs a flagship model. The typical pattern is “try a small model first, escalate when necessary”:

    1. def answer_with_routing(task, ledger):
    2. # 1. Try the small model first, with output constrained to a structured format
    3. # that is easy for programs to validate.
    4. cheap = call_model(
    5. prompt=task.prompt, model="small", ledger=ledger, json_schema=task.schema
    6. )
    7. if cheap.valid and cheap.confidence >= 0.9:
    8. return cheap # most tasks end here
    9. # 2. If the budget is too low, skip escalation and return the fallback directly.
    10. if ledger.budget_left() < 0.1:
    11. return cheap
    12. # 3. Escalate with the small model's draft attached, reducing the flagship's workload.
    13. strong = call_model(
    14. prompt=task.prompt + "\nReference draft: " + cheap.content,
    15. model="flagship",
    16. ledger=ledger,
    17. )
    18. return strong if strong.valid else cheap

    The key to tiered routing is a reliable criterion for “does this need escalation.” Usable signals include: whether structured output validation passes, whether unit tests pass, whether confidence falls below a threshold, and whether there is ambiguity or a need for multi-hop reasoning. Also, every time you switch models, keep the prompt order and cache keys fixed, otherwise you will repeatedly invalidate the cache.

    5.4 Output budgets and reasoning levels

    • Set a max_tokens (maximum generated tokens) ceiling per task type to prevent individual requests from running away;
    • For reasoning models, choose the reasoning level according to task difficulty — low levels for simple tasks, high levels only for complex ones;
    • Prefer structured output constraints (such as JSON mode) to reduce retries caused by parsing failures;
    • Streaming responses can deliver the first segment of the result early, and partial valid output is preserved even when a timeout interrupts the request.

    5.5 Batching and scheduling: move deferrable workloads to the valley

    For workloads that are not time-sensitive — daily report generation, offline data labeling, batch document processing — asynchronous batch processing works well. Some platforms offer price discounts for batch jobs. In scenarios with peak/off-peak pricing, running off-peak directly takes advantage of the lower valley-hour price. Recommended practices:

    • Convert synchronous calls into message-queue-driven asynchronous tasks to smooth traffic peaks;
    • Use a container platform to elastically scale inference and orchestration services;
    • Set token consumption and cost thresholds in monitoring and alerts, and degrade or throttle automatically on anomalies.

    5.6 Agent loops: put brakes on autonomy

    The cost challenge with agents is that the number of turns is unpredictable. Common constraints include:

    • Loop caps: limit the maximum number of model calls and tool calls per task;
    • Budget circuit breaker: stop when cumulative cost for a task exceeds a threshold, and return the best current result or hand off to a human;
    • Deduplication and idempotency: execute the same search or read with the same parameters only once, and reuse the cached result;
    • Subtask isolation: split long tasks into independent subtasks, meter each one separately, and avoid unbounded context growth;
    • Failure strategy: distinguish retryable errors (such as rate limits and timeouts) from non-retryable ones (such as invalid parameters). Return directly on the latter to avoid wasted retries.

    Illustrative code:

    1. MAX_STEPS = 12 # maximum number of turns per task
    2. def run_agent(task, ledger, tool):
    3. history = []
    4. for step in range(MAX_STEPS):
    5. if ledger.budget_left() <= 0:
    6. return {"status": "budget_exhausted", "partial": history}
    7. reply = call_model(prompt=build_prompt(task, history), ledger=ledger)
    8. history.append(reply)
    9. if reply.done:
    10. return {"status": "done", "result": reply.content}
    11. tool_result = tool(reply.tool_call) # trims and deduplicates results internally
    12. history.append({"role": "tool", "content": tool_result})
    13. return {"status": "step_limit", "partial": history}

    5.7 Multi-model portfolio: allocate budget by task characteristics

    Different models have different strengths in code, math, long-text understanding, and other directions. Routing requests to models that are good at and appropriately priced for the task is more economical than “switching everything to one model.” Two points matter in practice:

    • Build a unified evaluation benchmark to avoid choosing models based on subjective impressions;
    • Keep a configuration switch per model that allows fast switching, to handle price changes or capacity fluctuations.

    VI. Rollout roadmap: four phases to run the cost war

    Phase 1: Make it visible (1–2 weeks)
    Connect a unified model call gateway, complete full-chain instrumentation, and break cost reports down by business line and task type to find the top 20% of scenarios by cost share.

    Phase 2: Make it accurate (2–4 weeks)
    Sample production traffic, build a task benchmark set covering typical scenarios, establish baseline data on both the quality and cost dimensions, and add it to CI regression.

    Phase 3: Make it come down (continuous iteration)
    Roll out context governance, caching, tiered routing, output budgets, and batching in order of return on investment, running the regression cases every time one goes live.

    Phase 4: Keep it under control (long term)
    Establish a quota and budget system, allocate monthly token budgets by team and business line, combine monitoring and alerts for anomaly detection, and periodically re-evaluate model selection.

    The illustrative cost dashboard metrics are as follows:

    Metric Meaning Optimization direction
    Average cost per task Average spend to complete one task Core objective
    P95 cost per task Cost of the most expensive 5% of tasks Govern the long tail and runaway loops
    Cache hit rate Share of input tokens served from cache Improve prompt structure and stability
    Retry rate Share of calls that need retrying Structured output, parameter validation
    Task success rate Share of tasks passing quality checks Prevent cost cuts from degrading quality
    Cost per successful task Total spend ÷ successful tasks Overall efficiency measure

    VII. Conclusion

    Over the past two years, competition in the LLM industry revolved around “how much per million tokens,” and list prices fell by tens of times. But once prices hit bottom, the market quickly split into a value tier and a flagship tier: the value tier trades scale for efficiency, and the flagship tier trades capability for premium pricing. The clustered releases in October 2025 showed that vendors are no longer competing on list price, but on what customers actually spend to complete one task.

    For developers and technical decision-makers, this means the center of gravity of cost optimization must shift:

    • From “choose the cheapest model” to “use the most suitable model to complete the target task”;
    • From “watch the unit price” to “manage token flow”;
    • From “a one-time model selection” to “continuous measurement and continuous tuning.”

    Only by doing context governance, cache reuse, tiered routing, and budget control on the demand side can the upstream price dividend actually turn into a lower bill. Price wars will reignite periodically, but the cost war is an engineering practice that needs to be managed over the long term.