Beyond the LLM Price War: Shifting from Token Pricing to Task Cost

1. How the Price War Reached Its End

1. From Freefall to Bottoming Out

Over the past two years, the unit price of LLM API tokens experienced an unprecedented drop. By mid-2024, some providers pushed input prices down to near-zero levels per million tokens, with competitors following suit within weeks. By 2025, leading vendors announced free access to their conversational products (such as Baidu making its chat product entirely free), and flagship model API prices were driven down to 1% of their international equivalents. Industry monitoring data indicates that in the second half of 2025, the normalized average price for domestic LLM token services fell into single-digit territory (per million tokens).

It is crucial to clarify the statistical caliber of this “normalized average price”: it reflects the mean calculated by converting the list prices of different models at their release times into a unified unit. It represents a supply-side pricing trend, not the actual average user bill. Distinguishing between these two metrics is key to understanding subsequent market shifts.

2. Market Divergence by Model Tier

Once the average price bottomed out, the market stopped its unilateral decline and segmented by model tier:

• Budget tier continues to drop: In the first half of 2025, some vendors permanently reduced prices for general-purpose conversational models, with certain models seeing price cuts of over 90%. The positioning of low-cost, broad accessibility was explicitly retained.
• Flagship tier trends upward: Several leading teams raised prices multiple times within a year, citing a return to “normal commercial value.” A few flagship models pushed output prices up to the hundred-yuan level per million tokens, setting a new price high for domestic models.
• Billing mechanism divergence: Peak/valley pricing and tiered pricing mechanisms emerged. Some vendors introduced peak-hour surcharges for flagship models, multiplying the output unit price compared to the baseline.

A counterintuitive data point: due to the collective price hike in the flagship tier, the normalized average price for the first half of the year actually increased rather than decreased, nearly doubling. For the first time, structural divergence appeared in market pricing. Notably, this round of price hikes did not trigger retaliatory price cuts; in fact, the recurring revenue of some vendors raising prices continued to grow. This indicates that customers are willing to pay a premium for stronger capabilities. The era of acquiring customers solely through low prices is over.

3. New Model Releases No Longer Compete on Price

In October, multiple leading teams released new-generation models within ten days. Across five launch events, not a single one marketed “lower price” as a selling point. Only one vendor lowered its list price, while the rest maintained their previous generation’s pricing. Ironically, while list prices remained almost static, the actual changes to customer bills were larger than at any point in the past two years. Some teams reduced actual single-task costs by over 30% without changing the list price; others claimed to bring single-task costs down to an eighth of international flagship models, or cut agent-scenario task costs by more than half. Some teams stopped comparing prices altogether, shifting the focus to local data privacy and stable output per unit time. Even the one vendor that lowered its list price saw almost no change in third-party measured single-task costs. What happened behind the scenes?

2. Why “List Price” is Failing: From Token Pricing to Task Cost

1. List Price is Unit Price; Bill is Total Volume Multiplied by Unit Price

The basic formula for API billing is: Bill = Input Tokens × Input Unit Price + Output Tokens × Output Unit Price. The price war only affected the “unit price” variable, but the bill also depends on “total token volume,” which is dictated by invocation behavior: context length, conversational turns, failure retries, cache hit rates, and the number of loops an agent must execute. Two models with identical list prices can yield bills that differ by several multiples in the same real-world business scenario.

2. Agents Amplify the Weight of “Total Volume”

With the popularization of Agent architectures, a single task often involves a multi-round loop of planning, tool calling, observation, and replanning. Each round must carry an ever-growing historical context. The total token volume of a task expands rapidly with the number of turns, diminishing the influence of the unit price on the final bill. Conversely, engineering variables like inference efficiency, context management, and cache hit rates have drastically increased in importance.

3. The Industry’s Unit of Measurement Has Shifted

This is exactly what happened in October: the metric of competition shifted from “list price per million tokens” to “actual money spent by the customer to complete a task.” Price reduction is no longer a single concept—even with a static list price, task costs can still drop significantly. The path to this reduction lies in the technical variables outlined below.

3. What Determines Single-Task Cost: Four Technical Variables

For developers, understanding the composition of single-task cost is far more valuable than memorizing any list price. Broken down, the cost of a task is roughly determined by four categories of variables:

1. Inference-Side Efficiency

• Prefix Caching and KV Cache: When repeated prefixes like system prompts and few-shot examples hit the cache, they are typically billed at a lower unit price. This saves money and reduces time-to-first-token.
• Continuous Batching and Speculative Decoding: Server-side scheduling efficiency determines the vendor’s cost baseline, dictating how low a list price they can afford to offer.
• Quantization and Distillation: Trading smaller deployment density for lower unit costs is the technical prerequisite that makes budget-tier pricing viable.

2. Context Engineering

• Prompt Streamlining: Removing redundant instructions and repeated examples directly reduces input tokens.
• History Compression: Summarizing multi-turn conversations instead of carrying the full history yields significant savings in long-session scenarios.
• Cache-Friendly Structure: Placing stable content at the prefix and volatile content at the tail maximizes cache hit rates.

3. Invocation Architecture

• Model Routing: Downgrading simple subtasks to smaller models. If a small model’s unit price is 10% of the flagship model and it handles 60% of invocations, the overall cost can be reduced by roughly half.
• Batch Processing Channels: Routing latency-insensitive offline tasks through batch interfaces usually unlocks discounts.
• Retry and Timeout Strategies: Failure retries are an invisible cost amplifier; setting limits and ensuring idempotency is critical.

4. Billing Mechanisms

• Peak/Valley Pricing: Shifting deferrable invocations to off-peak hours.
• Tiered Pricing: Segmenting by throughput commitments and concurrency levels. Enterprise customers often receive effective unit prices different from the catalog price.
• Cache Discounts: Prefix-hit portions are billed at a discounted rate.

4. Enterprise Practices: Building Cost Metrics at the “Task” Level

1. Define the “Task” First, Then Instrument

The first step in cost optimization is not cutting prices, but aligning the measurement metric with the business: how many tokens, how many calls, and how much money does a single retrieval Q&A, a generated report, or a processed ticket consume? Creating a unified wrapper at the invocation layer to log the model, token count, cache hits, latency, and success status for every request forms the foundation for all subsequent analysis. Mainstream model service platforms (such as Baidu AI Cloud and others) generally provide common capabilities like usage statistics, logging services, and monitoring alerts, which can be directly reused to avoid building an entire instrumentation system from scratch.

2. A Practical Cost Accounting Example

# Single-task cost accounting illustration: Bill breakdown for a 3-step Agent task
# Prices are illustrative; refer to official platform prices for actual values
PRICE_IN    = 4.0    # Input token price: CNY / million tokens
PRICE_OUT   = 16.0   # Output token price: CNY / million tokens
PRICE_CACHE = 1.0    # Prefix cache hit input price: CNY / million tokens
def step_cost(prompt_tk, cached_tk, output_tk):
    uncached = max(prompt_tk - cached_tk, 0)
    return (uncached * PRICE_IN
            + cached_tk * PRICE_CACHE
            + output_tk * PRICE_OUT) / 1_000_000
steps = [
    # (input_tokens, cached_tokens, output_tokens)
    (12_000,  8_000,   800),   # Task planning
    ( 9_000,  7_000, 1_200),   # Tool call and observation
    (15_000, 11_000,   600),   # Result summarization
]
total = sum(step_cost(*s) for s in steps)
print(f"Single task cost ≈ {total:.4f} CNY")   # Approx. 0.11 CNY

3. Replace “Raw Cost” with “Quality-Weighted Cost”

The effective cost metric is the single-task cost divided by the task success rate: Cost per Successful Task = Single Task Cost / Task Success Rate. If the single-task cost is 0.1 CNY and the success rate is 80%, the cost per successful task is 0.125 CNY; increasing the success rate to 90% is equivalent to an 11% cost reduction. This also explains why flagship models can retain customers despite price hikes—if a stronger model significantly improves success rates and latency, the quality-weighted cost might actually be lower.

4. Governance Mechanisms

• Budgets and Alerts: Set budgets by project and scenario, with immediate alerts for anomalous usage.
• Attribution Analysis: Regularly use logs to break down cost composition and identify “token hogs.”
• A/B Validation: Any changes to routing, compression, or caching should be evaluated against dual metrics: quality and cost.

5. Trends and Implications

The essence of the cost war is shifting competition from the marketing department’s price list to systemic engineering and algorithmic efficiency. Predictable trends include: pricing granularity will continue to refine, with explorations into task-based or outcome-based billing; peak/valley, tiered, and cache discount mechanisms will become standard; and model selection will shift from “looking at benchmarks” to “calculating quality-weighted costs on real tasks.”

For developers, the implication is straightforward: in this new phase of competition, token list price is merely one variable in the cost formula. Context engineering, invocation architecture, and caching strategies are the most controllable levers for cost reduction. Establishing a task-level cost measurement system early on is far more reliable than waiting for the next price cut in a price war.