The Token Cost Paradox: Why Stronger LLMs Can Still Burn Your Budget

1. A Counterintuitive Industry Phenomenon

Over the past year, the iteration speed of large language models (LLMs) has been visible to the naked eye: new versions are released successively, capability leaderboards are frequently refreshed, and reasoning and coding abilities continue to break through. At the same time, API prices continue to drop, with “price reduction” becoming a standard feature of almost every model upgrade. Ordinarily, stronger capabilities and lower prices should elate users. However, a thought-provoking voice has emerged in the developer community: the models are indeed better, but the bills haven’t gotten lighter. Some subscription users even report that “the unit price dropped, but the quota is consumed much faster.”

This coexistence of “price drops” and “increasing costs” is not an isolated case of a specific vendor, but an inevitable result of the collision between LLM technological evolution and business models. To understand it, we need to break down “price” into two variables: price per token and token consumption per task. The true cost is the product of the two, divided by the task success rate. Focusing on only one variable can easily mislead you with surface-level numbers.

2. Why the Unit Price Keeps Dropping

From both technical and market dimensions, the decline in unit price has its internal logic:

1. Continuous optimization on the inference side: Model compression techniques like quantization and distillation migrate large model capabilities to smaller inference carriers. Engineering methods such as KV cache optimization, operator fusion, and continuous batching continuously improve hardware throughput. More efficient attention mechanisms also reduce the computational overhead of long contexts. Together, these advancements lower the compute cost per token.
2. Market-driven pricing strategies: The rise of open-weight models has lowered the industry’s price anchor. Under homogeneous competition, pricing becomes the most direct means of customer acquisition. Top vendors trade price for volume, pushing the overall price band downward.
3. Economies of scale: Rising call volumes dilute the fixed costs of cluster construction and maintenance, providing room for price reductions.

3. Why Token Consumption Keeps Rising

While the unit price drops, the token consumption per task is increasing, driven by clear technical factors:

1. Chain-of-Thought and test-time compute scaling: New-generation models generally emphasize “deep thinking”: generating large amounts of intermediate reasoning text before providing a final answer. These reasoning tokens are billed just the same, yet they are a crucial source of answer quality—more thorough reasoning leads to more reliable conclusions, and greater consumption. This is a classic case of “trading computation for quality.”
2. Reflection and self-correction: Stronger models often exhibit a behavioral pattern of “plan first, execute, then check.” A single task might involve rewriting code multiple times or verifying results across several rounds, with each step generating new tokens.
3. Multi-turn loops in Agentic tasks: When a model acts as an agent calling tools, a single task expands into a “plan-call-observe-replan” loop. The results returned by tools re-enter the model as context, causing the context to continuously expand. Meanwhile, the attention computational cost for long contexts grows with their length.
4. Long context and multimodal inputs: Document-level comprehension and codebase-level Q&A have become common scenarios, meaning the base volume of input tokens itself is growing.

In other words, the improvement in model capabilities is largely the result of being “willing to think and compute more.” Rising consumption is not an anomaly, but an integral part of where capabilities come from.

4. The Effective Cost Formula: Redefining “Expensive” and “Cheap”

For developers, the real concern is not how much it costs per million tokens, but “how much does it cost to complete my task?” This can be described using a simplified effective cost model:

Effective Cost ≈ Unit Price × Token Consumption per Task ÷ Task Success Rate

The interplay of these three variables explains many counterintuitive phenomena:

• Model A has a lower unit price, but requires multiple retries to achieve the same quality, resulting in a higher effective cost.
• Model B has a higher unit price, but offers a high first-pass success rate and a concise reasoning chain, potentially resulting in a lower overall cost.
• If the unit price is reduced by one-third, but consumption rises by more than 30%, the total cost increases instead of decreasing.

The industry feedback stating that “a new model’s unit price dropped by about 30%, but the consumption per task increased significantly, so the calculated total cost is about the same” is a direct reflection of this formula. The price war competes on the first variable, technological evolution rewrites the second, and the user’s actual perception is determined by both.

5. Subscription vs. API: Two Ledgers, Two Algorithms

API users pay per token, making price reductions a direct benefit. Subscription users, however, face a completely different ledger:

1. Under quota systems, consumption directly determines “how long it lasts”: As models become stronger and consume more, the number of tasks that can be completed with the same subscription fee decreases. Users naturally feel it has become “more expensive.”
2. The vendor’s dilemma: If vendors adapt to the new model using old quotas, heavy users will consume massive amounts of reasoning tokens, putting pressure on gross margins. If they tighten quotas, they hurt user experience and trigger public backlash. There are cases in the industry where vendors publicly apologized due to quota and billing disputes—this is the externalization of the contradiction. The stronger the model, the higher the per-service cost, making it harder to balance subscription pricing and quota design.
3. The fragility of average pricing: Subscription models are built on the assumption of “average usage.” However, the improvement in reasoning capabilities widens the variance in usage among users: light users feel nothing, while heavy users see costs skyrocket. The pricing model is exposed to structurally amplified risks.

6. A Cost Evaluation Framework for Developers

Facing rapidly iterating models, it is advisable to establish a task-level cost baseline rather than staring at official unit prices:

1. Fixed task set: Select representative real-world tasks (code repair, document summarization, data extraction, etc.) and freeze the inputs and evaluation criteria.
2. Three key metrics: Token consumption per task (tracking input and output separately), first-pass success rate, and manual correction cost.
3. Calculate the effective cost and horizontally compare old and new models.

Below is a sample code snippet illustrating the evaluation logic (the values are purely hypothetical and do not represent any real product):

# Task-level cost estimation example
def estimate_cost(price_in, price_out, tokens_in, tokens_out,
                  success_rate, retries):
    attempt_cost = (tokens_in * price_in + tokens_out * price_out) / 1e6
    expected_attempts = 1 / success_rate \
                        + retries * (1 - success_rate) / success_rate
    return attempt_cost * expected_attempts
# Old model: higher unit price, lower consumption, average pass rate
old = estimate_cost(15, 80, 8000, 4000, 0.7, 1)
# New model: lower unit price, higher consumption, higher pass rate
new = estimate_cost(10, 55, 12000, 9000, 0.9, 1)
print(old, new)  # Results may be very close: price drops are offset by increased consumption

By placing unit price, consumption, and success rate into the same formula, you can clearly see the true benefits of each model upgrade.

7. A Practical Checklist for Cost Optimization

Given that model-side pricing cannot be controlled, there is still clear room for optimization on the application side:

1. Model tiered routing: Assign simple tasks to smaller models or low-cost tiers, and only enable flagship models for complex tasks, dynamically selecting based on task complexity.
2. Context management: Prune historical conversations, compress summaries, and inject only necessary document fragments to directly reduce input tokens.
3. Prompt engineering: Use clear instructions and structured output constraints to reduce irrelevant outputs and repetitive explanations, lowering output tokens.
4. Caching and reuse: Utilize context caching capabilities for fixed prefixes (system prompts, long documents). Hit portions are typically billed at a lower rate.
5. Batch processing: Route offline tasks through batch channels, which usually offer discounts.
6. Retry governance: Set budget limits and fallback strategies for failure retries to avoid cascading consumption triggered by low-quality outputs.
7. Usage monitoring and alerting: Mainstream LLM service platforms, such as Baidu AI Cloud, generally provide token usage statistics and budget alert capabilities. Integrating these allows for timely intervention when costs behave anomalously.

8. The Commercial Dilemma: Structural Challenges from the Vendor’s Perspective

Shifting the perspective back to vendors, “stronger models, harder business” is not an exaggeration but a structural reality: inference costs rise with the depth of thinking, and deep thinking is the very source of capability—cutting costs may directly harm capabilities. Once the industry’s price anchor drops, it is difficult to raise again; reducing prices is easy, raising them is hard. Subscription prices are locked in monthly, while inference costs dynamically rise with model upgrades, leaving vendors to absorb the difference. Furthermore, any adjustment to quotas or billing rules can trigger user backlash.

There are roughly three directions to break the deadlock: First, continuously reducing unit inference costs through architectural innovations, such as more efficient attention mechanisms, Mixture-of-Experts (MoE) structures, and speculative decoding. Second, exploring value-aligned billing models, such as billing by task outcomes or a hybrid of usage and seat-based billing. Third, building differentiation through enterprise-grade platform capabilities (private deployment, dedicated compute, usage governance tools) to break free from a pure token price war.

9. Conclusion: Establishing a Task-Centric Cost Perspective

The capability dividends brought by model upgrades are real, and so are the changes in your bill. For developers and enterprise users, rather than fixating on “which one is cheaper,” it is better to return to first principles: measure the effective cost of your own tasks, and make the benefits of each model upgrade quantifiable and verifiable. When the industry shifts from “competing on parameters and leaderboards” to “competing on task quality per unit cost,” this game over tokens will move toward a healthier equilibrium—both for vendors and users.