LLM Token Costs: Why Stronger Models Burn More Tokens and How to Optimize Them

Over the past year, the LLM industry has replayed a familiar scene: a new model launches, benchmark scores climb, and market reception is enthusiastic. Soon after, users flood support channels with complaints about “abnormal quota consumption” and “hitting the ceiling minutes into a task.” The provider then issues a public apology, hands out compensation credits, and promises to rein in inference overhead — and some providers have been through this cycle more than once.

On the surface, these look like a string of operational incidents. Underneath lies a hard problem in technical economics: models keep getting stronger, tokens keep burning faster, and the business keeps getting harder to run. This article skips the drama and dissects two things: where tokens actually leak, and what platforms and users can each do about it.

1. The “Quota Black Hole”: An Underestimated Structural Inflation

1.1 Thinking Is Billed: The Reasoning-Chain Amplifier

To solve complex problems, the latest generation of models commonly “thinks before answering” with extended reasoning. Users see only the final answer, but the intermediate reasoning chain — decomposing the problem, comparing approaches, self-checking — is metered and billed entirely as output tokens. The “deeper” the thinking, the faster the bill climbs, and this consumption is nearly invisible in the chat interface. That’s why so many post-incident pledges from providers include phrases like “controlling reasoning depth” and “adaptive thinking based on question difficulty.”

1.2 Multi-Agent Orchestration: The Multiplication Effect

Advanced scenarios such as automated coding and deep research lean heavily on multi-agent pipelines of plan → decompose → execute → synthesize. Each sub-agent independently loads the system prompt and task context, then produces its own intermediate results. If the shared context is 50,000 tokens and one orchestration spawns four sub-agents, context reloading alone can inflate input consumption 4–5x. Add parallel calls and result aggregation, and what the user perceives as “one task” is, on the billing ledger, a miniature job cluster.

1.3 Context and Output: Expanding on Both Ends

Long-context processing has become a key selling point for modern models, but it has also changed user habits: entire documents and whole codebases get pasted straight into the conversation. Input is billed by length, so the more you stuff in, the faster it burns. The output side works the same way — newer models are more “thorough”: code arrives with comments, conclusions come with tables, and answers include self-check explanations. Output length can easily double.

1.4 Usage Patterns Have Shifted

For heavy users, LLMs stopped being an occasional chat tool long ago; they are productivity engines embedded in daily workflows — writing code, editing documents, running automation. Once call frequency goes from “a few times a day” to “several times a minute,” any uptick in per-call consumption is multiplied on the monthly bill. These users feel the squeeze first, and they are the loudest about it.

2. Running the Numbers: Why Unit Prices Fall While Bills Rise

Start with the general formula:

Task cost = Σ over steps (input tokens × input price + output tokens × output price)

Now look at a set of illustrative figures (prices and consumption are hypothetical, shown only to convey the order of magnitude):

Dimension	Previous-generation model	New-generation model
Input price (CNY per million tokens)	20	14
Output price (CNY per million tokens)	90	60
Input tokens per task	50,000	120,000
Output tokens per task	10,000	40,000
Cost per task	~1.9 CNY	~4.1 CNY

Unit prices drop by roughly 30%, yet the cost per task more than doubles. The root of the paradox: unit price is a linear variable, while consumption is a structural variable. A capability upgrade changes not only “how much each token costs” but “how many tokens the same task requires” — longer thinking, more sub-tasks, fuller outputs. Multiplied together, they can swamp any proportional price cut. This is why the real cost of a model service must be evaluated at the task level, not by list price.

3. Platform-Side Quota Governance: Four Engineering Levers

1. Make usage transparent. Provide real-time usage dashboards, task-level consumption breakdowns, and anomaly alerts. What users fear usually isn’t paying more — it’s not knowing why they’re paying more. Surface a breakdown like “this task consumed 40,000 tokens: 12,000 for thinking, 18,000 for sub-agents,” and most disputes dissolve before they start.
2. Design tiered rate limits. Distinguish burst quotas from steady-state quotas, apply limits by plan tier, and warn users before they hit a cap. One-size-fits-all hard throttling is the fastest way to trigger a trust crisis, while guaranteed steady throughput for heavy productivity users is what drives renewals.
3. Converge inference costs. There are at least three points of convergence: cut unnecessary sub-agent calls and merge steps that can be merged; enable prefix caching for high-confidence repeated prefixes so context isn’t billed at full price over and over; adaptively control reasoning depth and output length by task difficulty so simple questions never trigger “deep thinking.” These optimizations don’t lower the capability ceiling, but they can flatten the cost curve significantly.
4. Repair trust with observability by default. One-time compensation only stops the bleeding; transparent billing rules and advance notice are the foundation of long-term trust. Mainstream cloud platforms — Baidu AI Cloud among them — typically bundle model services with logging, monitoring, and alerting capabilities, so providers are well positioned to make usage observability a default product experience rather than a post-incident talking point.

4. User-Side Practices: A Token-Saving Checklist

1. Match the model tier to the task. Configure different model tiers for different steps in an automated pipeline — route retrieval, formatting, and simple rewrites to lightweight models, and reserve the flagship model for complex planning and core generation. This is the single highest-leverage move for cost-effectiveness.
2. Practice context engineering. Instead of pasting full documents, use retrieval-augmented generation (RAG) to feed only relevant excerpts; periodically summarize and compress conversation history; keep system prompts lean and stable — which also improves output quality.
3. Constrain output. Explicitly cap output length, prefer structured output (e.g., JSON), and specify “skip the explanation” to cut pleasantries and redundant self-checks. For high-frequency tasks, every token trimmed from output is real money.
4. Reuse prefix caches. Put fixed, unchanging instructions at the start of the message. On platforms that support prompt prefix caching, billing for the repeated portion can drop substantially.
5. Set budgets and monitor. Assign a token budget to each task category and alert when it’s crossed. For a rough first estimate, use the common rule of thumb — one Chinese character is roughly one token, and four English characters are roughly one token — to size the order of magnitude.

The following pseudocode sketch shows the core logic of cost estimation and budget alerting:

RATES = { "input": 14, "output": 60 }   # CNY per million tokens (example rates)
def task_cost(in_tok, out_tok):
    return in_tok / 1e6 * RATES["input"] + out_tok / 1e6 * RATES["output"]
# Single agent: shared context loaded once
single = task_cost(50_000, 10_000)        # ~1.30 CNY
# Multi-agent orchestration: shared context reloaded by sub-tasks
multi  = task_cost(50_000 * 4, 40_000)     # ~5.20 CNY
def budget_guard(used, budget, threshold=0.8):
    ratio = used / budget
    if ratio > threshold:
        emit_alert(...)   # can be wired to logging, monitoring, and alerting services
    return ratio

Same task, same result — only the orchestration differs, and the cost differs by 4x. Building this estimation logic into the task pipeline, so every orchestration passes a cost check first, is the most direct way to keep the bill under control.

5. Going Deeper: Why Stronger Models Make the Business Harder

1. The capability–cost–price triangle. Capability upgrades push up the cost of each inference, while market competition pushes list prices down. Someone has to absorb the gap: either the platform eats it, or it is passed to users through rate limits and quota design. The “tight quotas” users feel are exactly what this triangle looks like when it’s squeezed.
2. When cross-subsidies break down. The classic subscription logic is “light users subsidize heavy users.” When heavy users embed the model deep into their productivity workflows, their consumption blows past what the plan was designed for, and the subsidy structure collapses: raising prices drives away light users; tightening quotas enrages heavy users. Both directions hurt — and heavy users are precisely the core of word-of-mouth and renewals.
3. The hidden cost of differentiation. Long context and deep reasoning are competitive assets — and token black holes. The sharper the differentiation, the higher the per-task consumption and the heavier the quota pressure. Platforms face a built-in tension between advertising capability ceilings and controlling average consumption.
4. Trust is the most expensive asset. Compensation can close the books on one incident, but users permanently switch to “accounting-mode usage” — weighing the quota before every task and skipping calls they would otherwise have made. Once users start calculating carefully, both the product experience and the commercial upside take a discount. Compensation is easy; expectations are hard to repair. That may be the real cost behind repeated public apologies.

6. Conclusion: Turn the “Quota Problem” into an “Engineering Problem”

The quota dispute is, at its core, a rebalancing of capability, cost, and business model — and the lever for that rebalancing is turning every token of consumption into an engineering metric that can be observed, attributed, and optimized. For platforms, transparent billing, a leaner inference architecture, and tiered rate limiting retain users far better than after-the-fact compensation. For developers and enterprise users, building task-level cost models and sticking to tiered calling and context engineering is the reliable path through the “unit-price illusion.” Models will keep getting stronger. But what determines whether this business lasts is whether every call comes with an account that can be calculated clearly and explained plainly.