The LLM Price War Is Ending: Why Cost per Successful Task Matters More Than Price per Million Tokens
List prices for large language model services have fallen sharply, but lower token prices do not automatically translate into lower business costs. This article explains why organizations should measure cost per successful task rather than price per million tokens. It examines the main cost drivers across inference efficiency, context management, call-chain design, and observability; introduces a task-level cost model; and covers engineering optimizations such as model routing, caching, context trimming, structured output, and quality regression. It also discusses governance practices for budgets, quotas, cost attribution, and responsibility boundaries, and concludes with selection criteria based on effective cost and service-level requirements.
Over the past year, list prices for large language model (LLM) services have continued to decline. The competitive focus is shifting from the quoted price per million tokens to a more practical question: how much does it cost to complete a business task?
Since 2024, one model vendor has reduced its input price to RMB 1 per million tokens, and multiple cloud providers and model providers followed within a short period. In 2025, Baidu announced that a conversational product would be fully free at a specified time and pushed LLM API pricing down to 1% of the price of a mainstream international model. Industry monitoring data showed that in the second half of 2025, the normalized average price of LLM token services in China was approximately RMB 6.8 per million tokens. Importantly, this average was normalized from the prices published at model release; it is not the average price on users’ actual bills. In other words, the price war at the list-price level is entering its final stage, while the cost war that truly determines enterprise spending has only just begun.
1. Lower Prices Do Not Mean Lower Costs
On the surface, a lower token unit price directly reduces invocation fees. In real business systems, however, the bill is determined by multiple variables: the ratio of input to output tokens, context length, cache hit rate, failed retries, tool calls, additional requests introduced by retrieval-augmented generation, concurrency and SLA requirements, dedicated instances and elastic resource costs, and investment in logging, monitoring, evaluation, and operations.
A model with a lower unit price may end up costing more per task if it produces longer outputs, requires more retries, or is more likely to cause context bloat. Enterprises therefore should not compare only “price per million tokens.” They should establish a “cost per successful task” metric. A successful task is a request that meets quality, latency, and stability requirements, is accepted by the business system, and completes the full workflow. Only by putting failed retries, human fallback, tool calls, and retrieval costs into the same ledger can a team determine whether cost optimization is genuinely effective.
2. Four Main Battlegrounds in the Cost War
Compute and Inference Efficiency
Model inference cost is closely related to model size, precision, batching strategy, memory usage, KV cache management, quantization, and distillation. Long contexts can significantly increase cache and memory pressure, while high concurrency requires better batching and scheduling. Enterprises should focus on throughput, P95 latency, and output per unit of compute, not only the quoted price of a single invocation.
Context Governance
Many applications lose control of cost not because the model unit price is high, but because context keeps accumulating: multi-turn conversations retain the entire history, retrieval results are concatenated in full, tool responses are not trimmed, and prompt templates contain redundancy. The longer the context, the more input tokens are consumed and the greater the cache and compute pressure. Teams should control context through summarization, segmented retrieval, retaining only necessary fields, limiting the number of historical turns, and using compressed representations for structured data.
Call-Chain Governance
A single business request may pass through intent recognition, retrieval, reranking, main model generation, result validation, tool calls, and other steps. Every additional call adds cost. Teams should clarify which steps must use an LLM and which can be handled by rules, classifiers, or lightweight models. Low-risk requests can use tiered routing, while only high-value requests need to be sent to high-quality models.
Operations and Observability
Without measurement, there is no optimization. Enterprises should record input and output token distributions, cache hit rate, retry rate, timeout rate, number of tool calls, context-length percentiles, and cost per thousand successful requests. Without these metrics, budget alerts can show only the total amount and cannot identify where costs originate.
3. Establish a Unified Cost Measurement Standard
Normalized list prices are suitable for comparing published model prices, but they cannot be used directly for financial budgeting. An actual bill should include at least: input token fees, output token fees, cache fees or discounts, batch-processing discounts, dedicated resource fees, network and storage fees, retrieval and vector service fees, failed retry costs, and monitoring and logging costs. A task-level cost model can be built using pseudocode such as the following:
def task_cost(request):input_tokens = count_tokens(request.prompt)output_tokens = count_tokens(request.expected_answer)cache_hit = estimate_cache_hit(request)retries = estimate_retries(request)base = input_tokens * (1 - cache_hit) * price_input+ input_tokens * cache_hit * price_cache+ output_tokens * price_outputreturn base * (1 + retries)+ retrieval_cost(request)+ tool_call_cost(request)+ observability_cost(request)
The point of this pseudocode is not the specific prices, but the reminder that cost is a task-level function rather than a single unit price. Enterprises should connect the cost model to application-side metering, allocate costs by business line, tenant, functional module, and request type, and form a traceable cost ledger.
4. Engineering Optimization: From Unit Price to Effective Cost
Practical optimizations generally include:
- Tiered model routing: Assign simple classification, extraction, and formatting tasks to lightweight models.
- Semantic and result caching: Reuse results for repeated questions, while setting expiration and consistency checks.
- Context trimming: Use rolling summaries for conversation history and rerank and truncate retrieval results by paragraph.
- Structured output: Reduce parsing failures and retries.
- Asynchronous batch processing: Process non-real-time tasks in batches.
- Output-length control: Use templates, field constraints, and stopping conditions to reduce unnecessary generation.
- Quality regression: Use evaluation sets to monitor accuracy after optimization and avoid saving cost at the expense of quality.
Caching should distinguish between exact caching and semantic caching. Exact caching carries lower risk and is suitable for stable question answering. Semantic caching requires similarity thresholds and manual spot checks to avoid incorrectly reusing an answer for a similar but different question. Routing strategies should also support fallback: when a lightweight model has low confidence or validation fails, the request should be escalated to a high-quality model.
5. Enterprise Governance: Budgets, Quotas, and Responsibility Boundaries
The cost war is not only an engineering problem; it is also a governance problem. Enterprises should establish budget alerts, tenant quotas, rate limits, cost tags, and regular review mechanisms. Budget alerts should cover daily, weekly, and monthly granularity and automatically trace abnormal growth to applications, endpoints, and request types. Quotas should be allocated according to business value to prevent low-value requests from crowding out resources needed by high-value requests. Cost tags should run through development, testing, and production environments to prevent test traffic from appearing on production bills.
Within the general capabilities of Baidu AI Cloud, budget alerts, logging services, monitoring alerts, and elastic scaling can be used for cost governance; specific configuration should be based on actual product documentation and business architecture. The key is not to stack tools, but to clearly define responsibility boundaries: who requests resources, who bears the budget, and who is responsible for optimization.
6. Selection Advice: Look at SLAs and Effective Cost
When selecting a model, teams should compare at least six types of metrics: cost per successful task, P95 latency, output stability, context limit and long-text performance, failure and retry rate, and service level and elasticity. For conversational products, multi-turn context cost also matters. For extraction tasks, structured output success rate is important. For high-concurrency scenarios, throughput and rate-limiting strategy matter. A price list is only the starting point; effective cost and business success rate are the destination.
7. Conclusion
Falling list prices are good for the industry because they lower the barrier to experimentation. But the cost war has only just begun. Future competitiveness will not be determined by who can quote the lowest price per million tokens, but by who can reliably complete more business tasks with less compute, shorter contexts, more stable call chains, and clearer governance mechanisms. For enterprises, the most practical action is to establish task-level cost measurement as soon as possible and upgrade optimization from “switching to a cheaper model” to “designing a cheaper system.”
