The LLM Price War Is Ending. The Cost-per-Task War Is Just Beginning

    The generative AI market is moving beyond headline token prices. As model pricing becomes increasingly segmented, enterprises are focusing on the real cost of completing a successful task—including retries, tool calls, storage, operations, and quality-related rework. This article explains why list-price competition is giving way to task-level cost optimization and outlines the engineering practices needed to measure and reduce that cost.

    The LLM Price War Is Ending. The Cost-per-Task War Is Just Beginning

    Over the past two years, the most visible metric in the large language model market has been the price per million tokens. Input prices started at roughly one currency unit per million tokens. Consumer chat products were offered for free, while API prices were cut to as little as one percent of comparable flagship benchmarks.

    Industry monitoring once reported a normalized average of approximately RMB 6.8 per million tokens. However, this figure was calculated from normalized list prices at the time models were released. It was not the average amount customers actually paid. In other words, it reflected the market’s “sticker price,” not the real cost of completing a task.

    Once list prices fell to a certain level, the market did not continue moving downward uniformly. Instead, it split by model tier: mass-market offerings continued to maintain low prices, while flagship models collectively moved upward.

    Under a common public monitoring methodology, the normalized average price at one point rose to approximately RMB 13.46, nearly doubling. The increase was driven primarily by flagship models. Some providers raised prices several times to better reflect commercial value. Others introduced peak and off-peak pricing, with output prices rising significantly during high-demand periods. Some providers raised output prices to levels that made their models among the most expensive domestic offerings at the time, yet their annual recurring revenue continued to grow rather than triggering broad market-wide price increases.

    More recently, several providers released new models within a ten-day period. Their products remained essentially the same combination of model and API, and most kept pricing unchanged from the previous generation. Only one provider reduced its listed prices.

    At the same time, providers repeatedly highlighted metrics such as cost per task, AI agent cost, data locality, and the actual amount spent to complete each task. One provider reduced its cost per task from $0.40 to $0.26. Another cut agent costs by 70 percent. A third reduced the cost per task to one-eighth of a flagship benchmark. The only provider to lower its list price saw almost no change in its cost per task in third-party testing.

    This suggests that the price war has not simply ended. Instead, competition has shifted from broad list-price cuts to cost competition that is more closely tied to customers’ actual bills.

    1. Why the Price War Ended in Market Segmentation

    The early price war followed a straightforward logic: use low list prices to acquire usage quickly and build economies of scale.

    However, LLM services are not standardized commodities like bandwidth or storage. Task quality, latency, context length, output length, reasoning steps, and tool-call frequency can all change the final cost. A single low price may attract price-sensitive users with simple workloads. For complex tasks, however, a low-priced model may require more retries, longer prompts, or additional human review. The final bill may therefore be higher, not lower.

    The market naturally segmented as a result:

    • Mass-market tiers use low prices to cover frequent, lightweight workloads.
    • Flagship tiers charge more for complex reasoning, long-context processing, and high-reliability tasks.

    The fact that flagship price increases did not trigger broad matching increases—and that some providers continued to grow revenue—shows that customers are not paying only for the lowest list price. They are paying for task value.

    The price war ended through structural differentiation. Low prices still exist, but they are no longer the only competitive dimension.

    2. Why List Prices Are Losing Relevance

    Task-level cost can be defined as the total resource cost required to complete one verifiable task, divided by the number of successfully completed tasks.

    At a minimum, it should include:

    • Input token costs: Separate uncached input tokens from cached input tokens.
    • Output token costs: Output is usually more expensive, making long-output workloads particularly sensitive.
    • Context and caching costs: Long contexts increase compute and storage pressure, while cache hits can significantly reduce costs.
    • Reasoning and retry costs: Multi-step reasoning, failed retries, and self-verification can multiply consumption.
    • Tool-call and orchestration costs: External tools, function calls, and workflow orchestration create additional expenses.
    • Storage, logging, and monitoring costs: Task records, audit logs, and metric collection are not free.
    • Deployment and operations costs: Elastic scaling, peak/off-peak scheduling, incident response, and compliance audits must also be included.
    • Quality costs: Human review and rework caused by low-quality outputs are among the most frequently overlooked expenses.

    The following pseudocode illustrates a task-level accounting model:

    1. def task_cost(records):
    2. total = 0
    3. for r in records:
    4. cached = min(r.cached_tokens, r.input_tokens)
    5. uncached = r.input_tokens - cached
    6. cost = (uncached / 1_000_000) * price_input
    7. cost += (cached / 1_000_000) * price_cache
    8. cost += (r.output_tokens / 1_000_000) * price_output
    9. cost += r.retry_count * r.retry_cost
    10. cost += r.tool_call_cost + r.storage_log_cost
    11. total += cost
    12. return total / len(records)

    The important point is not the formula itself. It is the decision to break list prices down into observable task-level dimensions.

    Without task-level accounting, an enterprise can see only its monthly aggregate bill. It cannot determine whether the cost comes from long inputs, long outputs, retries, or failed tasks.

    3. Engineering Levers for the Cost War

    3.1 Build a Task Benchmark Set

    Organize workloads by business scenario, such as summarization, retrieval-augmented question answering, code generation, document review, and AI agent orchestration.

    For each task category, define:

    • A quality threshold
    • A latency target
    • A success criterion

    Only the cost per successful task measured on the same benchmark set can support a meaningful comparison.

    3.2 Use Model Tiering and Routing

    Route lightweight tasks to mass-market models and complex tasks to flagship models. Use caching for requests with predictable inputs, batch processing for bulk workloads, and off-peak scheduling for businesses with significant demand fluctuations.

    The routing objective should be the lowest cost after quality requirements have been met—not simply the lowest list price.

    3.3 Optimize Prompts and Context

    Removing redundant instructions, splitting long documents, improving cache-hit rates, controlling output length, and reducing unnecessary retries can often lower costs more effectively than switching models.

    For agent workloads, set a maximum number of steps, impose budget limits, and quickly terminate failure paths when appropriate.

    3.4 Build Cost Observability

    General-purpose cloud infrastructure can support this work. Object storage can archive task records, logging and monitoring services can track cost anomalies, and container platforms can support elastic deployment of inference services.

    Bring the following metrics into a single dashboard:

    • Token consumption
    • Cache-hit rate
    • Retry rate
    • Task success rate
    • Cost per task

    This makes cost optimization measurable and allows teams to verify whether improvements persist over time.

    3.5 Treat Provider-Side Cost Reduction as a Systems Problem

    For model providers, inference efficiency depends on techniques such as batching, continuous batching, cache reuse, quantization, distillation, sparsification, and heterogeneous hardware scheduling.

    On the commercial side, providers can use mass-market tiers to maintain scale, flagship tiers to recover value, and peak/off-peak pricing to balance supply and demand.

    Free does not mean cost-free, and a low list price does not necessarily mean a low total cost. The key metric in the next stage of competition will not be the lowest price per million tokens. It will be the lowest cost per completed task while meeting quality, latency, and compliance requirements.

    The end of the price war means that relying on a single low price to acquire customers is no longer enough. The beginning of the cost war means that developers and enterprises must use engineering methods to measure, optimize, and govern their LLM bills.

    The providers and users that make task-level costs transparent, controllable, and sustainable will gain the real cost advantage in the next phase.