LLM Pricing: From Per-Token Rates to Real Task Cost and Subscription Leverage
This article offers a practical framework for evaluating LLM pricing beyond headline per-token rates. It breaks down input, output, and cache-hit billing, introduces the leverage ratio for comparing subscription plans, and identifies cache hit rate, tokens consumed per task, and output share as the three variables that actually determine spend. It also includes a cost-estimation code snippet, guidance on choosing between pay-as-you-go and subscriptions, and advice on observability, governance, and common pricing misconceptions.
Why Per-Token Prices Rarely Tell the Full Story
Pricing for a new generation of LLM services has been a hot topic lately. Reports on one next-generation model list an output price of CNY 100 per million tokens, an input price of CNY 20, and just CNY 1 for cache hits. That is roughly a 5x increase over the previous generation, and the conversation quickly narrowed to a single question: is it worth using?
Three arguments dominate that conversation:
- Benchmark rankings. The model is stronger, so the higher price is justified.
- Leverage versus overseas plans. Compared with subscription plans from overseas providers, the value for money is clearly worse.
- Plan pricing as a signal. Subscription prices should reflect the underlying cost structure, and that is how you judge whether a product can compete internationally.
Each argument touches a real issue, but stopping at unit price or leaderboard scores is not enough. To judge whether a pricing scheme is reasonable, look past the headline rate to the three variables that actually shape your bill: cache hit rate, tokens consumed per task, and plan credit relative to plan price. The rest of this article breaks each of them down.
How LLM API Billing Actually Works
Billing for LLM APIs generally has three components:
- Input (prompt) tokens — billed for the text the user sends or that appears in the context.
- Output (completion) tokens — billed for the text the model generates.
- Cache hit tokens — when the request prefix matches an existing cache entry, that portion of the input is billed at a steep discount. Some services also charge a one-time fee for writing to the cache.
Using the published rates of a next-generation model as an example (illustrative figures, not a real quote from any vendor):
| Billing item | Price (CNY per million tokens) |
|---|---|
| Input | 20 |
| Cache hit | 1 |
| Output | 100 |
This structure tells you three things:
- Output is usually several times more expensive than input, because generation involves more compute.
- The cache hit discount is enormous — often one to two orders of magnitude — making it the single most valuable line item to optimize.
- The input-to-output ratio determines how much of the total bill comes from the expensive half, which is exactly why agent-style long tasks and simple question answering can differ so dramatically in cost.
Leverage Ratio: The Right Way to Compare Subscription Plans
When you move from pay-as-you-go to a subscription, the comparison method has to change too. What determines value for money is not the monthly fee but the ratio of usable credit value inside the plan to the plan price — call it the leverage ratio.
A typical scenario (hypothetical numbers, shown only to illustrate the calculation):
| Plan tier | Monthly fee (CNY) | Usable credit (CNY) | Leverage ratio |
|---|---|---|---|
| Base tier | 199 | 1,200–2,800 | ~6–14x |
| Higher tier | 99 | 2,000–5,000 | ~20–50x |
When the base tier’s leverage ratio is still in single digits, plans from overseas providers in the same price range often reach 20x or more. A higher tier only becomes genuinely competitive once it clears 20x as well. The monthly fee is not the point; the point is how much usable token credit each unit of subscription spend buys.
Keep in mind that the leverage ratio only means something alongside rate limits, concurrency caps, and cache hit rate. A plan with a large nominal credit but strict throttling can cost far more in practice than the headline suggests, while a plan with a modest leverage ratio but stable cache hits can end up cheaper overall.
The Three Variables That Actually Drive Cost
1. Cache hit rate: the first lever in long-context work
Cache hits take the input price from CNY 20 to CNY 1, a 95% reduction. But cache hits do not happen automatically. They depend on two conditions:
- Prefix stability. Put everything that never changes — system prompts, tool definitions, long-term knowledge, fixed examples — at the very front of the request, and everything that changes at the end.
- Server-side cache policy support. Not every endpoint enables caching by default. Confirm whether the API supports it, how long the cache window lasts, and what invalidates it.
On the engineering side, the recommended approach is a request template built as a stable prefix plus a variable suffix:
[fixed] system prompt + tool definitions + long document excerpts + few-shot examples[variable] current user input, incremental conversation, timestamp / session ID
With this pattern, every turn in the same conversation reuses the same prefix, and hit rates improve substantially.
2. Tokens per task: the hidden cost of agent workloads
An agent task typically involves several rounds of read file → call tool → reflect → call again, and each round resends the full context to the model. A task that looks simple can easily consume tens of times more input tokens than it produces as output.
The cost formula simplifies to:
task cost ≈ number of rounds × (input tokens × input price + output tokens × output price)
Since cache hits sharply reduce the input price, the real cost amplifiers are the number of rounds and the number of input tokens. This is also why the same model can cost several times more or less depending on how it is used.
3. Output share: why “expensive” usually means expensive generation
Output tokens usually cost several times more than input tokens. When a task is mostly long-form generation — code generation, long report writing — the output side dominates the bill. When the task is mostly retrieval QA or structured extraction, the input side dominates. The optimal model choice is completely different for the two.
Cost Estimation: Turning Intuition into a Formula
Here is a calculation you can drop straight into a codebase (illustrative):
def task_cost(input_tok: int, cache_hit_tok: int, output_tok: int,p_in=20.0, p_cache=1.0, p_out=100.0):"""Estimate per-task cost in CNY from per-million-token prices."""uncached = input_tok - cache_hit_tokcost = (uncached * p_in + cache_hit_tok * p_cache + output_tok * p_out) / 1_000_000return round(cost, 4)# Example: 8,000 input tokens, 6,000 of them cache hits, 500 output tokensprint(task_cost(8000, 6000, 500)) # about CNY 0.096
In the same scenario, raising the cache hit rate from 0% to 75% drops the per-task cost from about CNY 0.21 to about CNY 0.096 — more than half. At 10,000 tasks per month, cache optimization alone saves hundreds of CNY per month.
Practical steps:
- Collect real call logs and record input, cache-hit, and output tokens for every request.
- Group by task type and compute P50 and P95 per-task cost.
- Use the formula above to back out the maximum task volume that fits a target monthly budget.
- Make cache hit rate and average rounds per task observable metrics, and track them in day-to-day monitoring.
Pay-as-you-go or Subscription: Where Each Model Fits
- Pay-as-you-go suits teams with volatile traffic, varied task types, and a need for flexible experimentation. Cost scales linearly with usage: controllable, but you give up plan discounts.
- Subscription suits teams with steady usage dominated by high-frequency interaction or agent work. When the leverage ratio is high, unit cost falls well below pay-as-you-go — but watch credit burn rate, rate limits, and concurrency caps.
- Hybrid — a subscription for everyday development and debugging, pay-as-you-go plus caching for high-concurrency production or batch jobs — is often the cheapest overall.
Three questions decide the choice: is monthly token consumption predictable? Are there obvious peaks? Does the team already have cache reuse and request governance in place? If any answer is no, start with pay-as-you-go and fix the governance first.
Observability and Governance
Cost control depends on observability. Build three layers in your own application:
- Metering. Wrap model calls in a single client that automatically records tokens and cost for every request.
- Alerting. Set daily and monthly budget thresholds, per-task cost anomaly alerts, and cache hit rate drop alerts.
- Governance. Downgrade high-frequency low-value calls to smaller models, cache results for duplicate requests, and circuit-break abnormal retries.
On the cloud side, general-purpose cloud products can support this stack: a container platform for inference and gateway services, object storage for request logs and evaluation data, a message queue to smooth out batch job spikes, and logging plus monitoring and alerting to attribute cost. General-purpose capabilities from cloud providers such as Baidu AI Cloud can serve as the foundation for this kind of governance setup.
Common Misconceptions
- Looking only at unit price. Cache hit rate and task structure sit between the unit price and the total bill. Unit price means little without knowing how you use the model.
- Looking only at benchmark scores. Leaderboards measure capability ceilings, not cost per task. Judge with your own business evaluation set.
- Ignoring the leverage ratio. Plan price is not cost. The ratio of credit to price is the real value for money.
- Assuming caching is automatic. Cache hits require deliberately designing request structure. Passive use often means a 0% hit rate.
- Ignoring output share. The best model choice for long-form generation and for retrieval QA may be entirely different.
Conclusion: A Three-Step Decision Process
When pricing changes, work in this order:
- Measure the structure. Determine the input/output ratio and cache hit potential in your own workloads.
- Compute the cost. Use the task cost formula and the leverage ratio to get your real monthly spend.
- Make the tradeoff. As long as capability is sufficient, prefer the option with high cache gains, a good leverage ratio, and governance friendliness.
Pricing is never an isolated product question; it is the combined result of capability, structure, and governance. Building a sound evaluation method is worth more than arguing about whether something is expensive.
