Why Stronger Models Burn More Tokens: Understanding the LLM Token Cost Trap

A flagship model from a leading model vendor went live, and within less than twelve hours, controversy followed: users reported that their subscription quota was approaching its limit shortly after tasks began running, while questions were also raised about the compliance of the vendor’s marketing practices. Two days later, the vendor issued a public apology, announced a one-time compensation of partial monthly quota for all subscribers, and promised to optimize invocation pipelines and reduce unnecessary sub-agent calls and verbose outputs. This was already the vendor’s third public apology within six months, yet “quota anxiety” has not gone away.

Notably, this is not an isolated case. “The model got upgraded, but the quota runs out faster than ever” is becoming a common experience among heavy AI users. This article does not judge the right or wrong of specific vendors. Instead, it attempts to answer three more fundamental questions: Why do stronger models burn more tokens? Why do API unit prices keep falling while the actual cost for users to complete tasks does not? And how should developers and enterprise users govern their own token costs?

Quota Anxiety: A Spreading Industry Phenomenon

Before discussing technical root causes, it is worth understanding where the complaints come from. For ordinary users, an LLM may be just a chat tool they open occasionally. But for heavy users, it is a productivity product embedded in daily workflows—writing code, conducting research, running automation tasks—invoked dozens or hundreds of times a day. These users are highly sensitive to quota consumption, and any change in billing metrics will be quickly noticed.

In technical community discussions, comparisons like “with the same budget, another provider lasts a month, but here it only lasts a day” are not uncommon. Some users have also concluded after doing the math: the new model’s unit price has indeed dropped, but the tokens consumed to complete a single task have increased, so the total bill is almost unchanged. When this “felt experience” erupts collectively after a new model launch, apologies and compensation can only calm the immediate public opinion. The real contradiction—the tension between capability upgrades and cost control—remains.

Stronger Models, More Tokens Burned: Three Technical Root Causes

Capability upgrades are never free. The acceleration of token consumption in new-generation LLMs mainly comes from three mutually reinforcing mechanisms.

1. The “Thinking Tax” of Reasoning Models

Current high-capability models generally introduce explicit reasoning mechanisms: before giving a final answer, the model first unfolds internal thinking—decomposing the problem, trying multiple paths, and self-verifying. This thinking content is counted as reasoning tokens in consumption, but is often invisible or semi-visible to users.

The stronger the capability, the longer the thinking chain typically is. A request that appears to only need “a few lines of code changed” may be accompanied by thousands of tokens of reasoning overhead. The user’s perception that “the quota is nearly exhausted shortly after the task starts” is largely driven by this hidden consumption.

2. The Multiplication Effect of Multi-Agent Orchestration

To complete long-horizon tasks, new-generation models commonly adopt multi-agent architectures: a main agent handles task decomposition, while sub-components handle retrieval, execution, and summarization respectively. This architecture significantly improves the quality of complex task completion, but also brings a multiplication effect in consumption:

• Each sub-call typically re-carries a copy of the task context;
• Intermediate results passed between sub-components generate large amounts of duplicated input tokens;
• The finer the task decomposition and the deeper the call hierarchy, the more context is repeatedly transmitted.

A rough illustrative estimate: if a task is split into 5 sub-steps, each carrying 2,000 tokens of context, the duplicated context transmission alone amounts to 10,000 tokens; if each step also unfolds its own reasoning chain, consumption is further amplified. This is why vendors almost always mention “reducing unnecessary sub-agent calls” in their apology announcements.

3. The Dual Lift from Long Context and Task Complexity

On one hand, long-context capability has become table stakes for competition, with context windows expanding from thousands of tokens to hundreds of thousands. The longer the window, the more users tend to stuff complete conversation histories, entire documents, or even entire codebases into requests, raising the baseline of input tokens across the board.

On the other hand, higher capability ceilings naturally push users toward heavier tasks: where they once used it to write emails, they now use it for long-horizon planning, complex code generation, and 3D modeling assistance. The more complex the task, the higher the total tokens per task. Benchmark scores are rising, and so are users’ bills—these are essentially the same thing.

Unit Prices Have Fallen, So Why Hasn’t Total Cost?

From publicly available information, LLM API unit prices have continued to decline over the past year. It is not uncommon for new-generation models to be priced more than 30% lower than their predecessors, and the industry generally uses “per million tokens” as the pricing unit. But the user’s real cost structure is:

Effective task cost = unit price × total tokens per task

Unit prices have fallen, but total tokens per task are rising. Suppose the unit price drops by one-third while tokens per task rise by 50%:

• Original cost: 1.00 × 1.00 = 1.00
• New cost: 0.67 × 1.50 ≈ 1.00

The two are almost identical. This is exactly the real experience of many users: the unit price is cheaper, but the model consumes more to complete the same task, so the total bill is unchanged. And under subscription plans, where quota is issued in the form of credits or usage counts, accelerated consumption is more intuitively perceived as “quota not lasting as long.”

This paradox reveals the industry’s current predicament: price cuts are an inevitability of competition, and capability upgrades are an inevitability of survival. The combined result is a shift of cost pressure from “unit price” to “usage volume.”

The Business Dilemma of the Subscription Model

Once the technical root causes are understood, it becomes clear why vendors are struggling. The key characteristic of subscriptions is “average-cost pricing”: vendors use the low usage of light users to subsidize the high usage of heavy users. When capability upgrades cause a general rise in per-task consumption, the consumption curve of heavy users accelerates upward, and the subsidy model rapidly deteriorates: light users continue to pay monthly with little noticeable impact; heavy users exhaust their quota quickly and either upgrade to a higher-tier plan—further increasing service costs—or churn entirely.

More troublesome is the trust problem. The hidden consumption of reasoning tokens and the internal calls of sub-components are a black box to users. When the speed of quota consumption deviates noticeably from users’ intuitive experience, and the billing details cannot fully explain the deviation, complaints and public opinion crises are almost inevitable. The “three apologies in six months” mentioned earlier is essentially the repeated eruption of the same contradiction. The stronger the model and the more complex the consumption structure, the harder the business becomes—this is not an operational failure, but a structural threshold that the business model must cross.

The Developer’s Response: A Four-Step Token Cost Governance Framework

Rather than passively waiting for vendors to optimize, developers and enterprise users should treat tokens as a resource that requires active governance—managing it the way you would manage database connection pools and memory.

1. Establish Token Budgets and Observability

The first principle of cost governance: consumption you cannot see cannot be optimized. It is recommended to encapsulate a unified model invocation gateway at the application layer, recording the input tokens, output tokens, latency, and associated task for each request, and connecting it to logging services and monitoring/alerting systems (whether through a self-built gateway or general-purpose logging and monitoring categories offered by cloud platforms such as Baidu AI Cloud). Example:

class TokenBudget:
    """A simple guard that controls total token consumption per task"""
    def __init__(self, limit: int):
        self.limit = limit
        self.used = 0
    def acquire(self, estimated: int):
        if self.used + estimated > self.limit:
            raise RuntimeError(
                f"Token budget insufficient: used {self.used}, "
                f"estimated {estimated} this time, limit {self.limit}"
            )
        self.used += estimated
budget = TokenBudget(limit=200_000)
for step in task_plan:
    est = estimate_tokens(step.context) + step.max_output
    budget.acquire(est)            # Circuit-break when over budget to prevent runaway tasks
    result = invoke_model(step)
    budget.used += result.usage    # Correct estimates with actual usage

With this layer in place, teams can clearly answer: which feature is burning tokens, what the unit cost is for each type of task, and what happened to the cost curve after a model version upgrade.

2. Context Governance: Compress, Retrieve, Keep Only What’s Necessary

• Sliding history window: keep only the most recent N turns of raw text, and compress earlier content into structured summaries;
• Document retrieval: do not stuff large files into context wholesale; chunk first, retrieve, and inject relevant fragments on demand;
• Slim down intermediate results: when passing state between sub-components, pass only conclusions and necessary parameters, not the full reasoning process.

Teams that practice this well can compress the context consumption of long tasks by an order of magnitude, with almost no impact on task quality.

3. Agent Call Governance: Set Rules for Sub-Tasks

• Limit the call depth of sub-components and the total number of calls per task;
• Set an independent output cap for each sub-component, prohibiting open-ended long outputs;
• Add caching for reusable results (retrieval, code interpretation, etc.) to avoid repeated calls;
• Add necessity checks at the orchestration layer: if it can be done in one step, don’t split it into multiple steps.

4. Model Tiering and Output Control

• Task tiering: use lightweight models with short outputs for simple tasks, and escalate to flagship models only for complex tasks;
• Explicitly set maximum output length and require the model to produce structured, concise results rather than lengthy explanations;
• Trim system prompts—the prompt itself is a fixed overhead billed on every request.

Implications for Vendors and Platforms

First, billing transparency. Presenting the consumption structure of reasoning tokens, context tokens, and sub-component calls to users in itemized form is the lowest-cost way to rebuild trust.

Second, optimize default behavior. A model’s default output length and invocation strategy should strike an engineering balance between “quality” and “consumption,” rather than shifting the entire cost of choice onto users.

Third, tier the quota model. Differentiate pricing between lightweight tasks and heavy tasks, and design sustainable high-usage plans for productivity users, rather than pushing heavy users toward alternatives with one-size-fits-all plans.

Fourth, manage expectations. Capability upgrades inevitably change the consumption structure. Proactive advance notice is far better than after-the-fact apologies and compensation.

Conclusion

The claim that “the stronger the model, the harder the business” may be an exaggeration, but it points to a real turning point for the industry: the first half of LLM competition was about capability; the second half is about cost per unit of capability. For vendors, this is a test of business models and cost structures. For developers and enterprise users, it is a clear engineering signal—tokens are no longer freely available tap water, but a scarce resource that requires budgeting, monitoring, and optimization. Whoever builds token cost governance capabilities first will find their own balance point between the capability dividend and cost reality.