LLM API Cost Breakdown: Why Identical Unit Prices Can Produce Very Different Bills
The per-token price of an LLM API is only one input to your final bill. This article breaks down the billing dimensions that actually determine cost—separate input/output pricing, prefix caching, batch offline channels, subscription plans, and time/region differences—then provides a unified estimation formula and ready-to-use Python code. You will learn how to structure prompts for cache hits, calculate the break-even point between pay-as-you-go and subscription plans, avoid common billing mistakes, and build realistic monthly cost estimates from real logs.
1. Same Unit Price, Different Bills
The pay-as-you-go unit price is just one input to your bill. The actual cost of a single API call is determined by multiple factors working together: input token count, output token count, whether prefix caching is hit, whether the call goes through an offline batch channel, whether a subscription plan applies, and when and where the call happens. Two teams calling the same model can end up with very different month-end bills if they differ on just one or two of these factors.
So before comparing prices, you need to align the estimation basis. Looking only at the unit price makes it easy to conclude “every provider costs the same,” while ignoring the billing rules that truly determine cost. This article breaks down each billing dimension one by one, and ends with estimation code you can modify and use directly.
All price figures in this article are hypothetical examples used to demonstrate calculation methods. Actual costs depend on each platform’s price list and billing documentation, and platform rules may change.
2. Breaking Down the Main Billing Dimensions
First an overview, then details for each item.
| Billing Dimension | Main Impact | Optimizable? | Applicable Scenarios |
|---|---|---|---|
| Separate input/output pricing | Output is usually priced higher | Yes—control output length and context | All calls |
| Prefix caching | Write and hit fees for repeated prefixes | Yes—adjust prompt structure | Fixed system prompts, fixed knowledge segments |
| Batch offline channel | Commonly discounted, results returned with delay | Yes—convert to offline jobs | Data labeling, report generation, batch translation |
| Subscription plans | Fixed monthly fee plus a quota | Yes—choose based on usage | Stable, predictable usage |
| Time and region | Prices or routing may differ by time or region | Platform-dependent | Latency-insensitive tasks |
2.1 Separate Pricing for Input and Output
Most LLM APIs price per million tokens, with different rates for input and output. Output is typically priced significantly higher than input. Even with the same 100,000 calls, a business generating long-form answers has a completely different cost structure from one doing intent classification.
- Input cost: system prompts, conversation history, retrieved reference material
- Output cost: the model’s generated responses, structured results, and reasoning content that some platforms count as output
There are two optimization directions: control output length, and remove context irrelevant to the current task. The former usually pays off faster because output costs more per token.
2.2 The Three Types of Prefix Caching Fees
When multiple requests share the same prefix, some platforms enable caching for that prefix. Cache-related fees typically fall into three categories:
- Cache write: the fee incurred when a prefix is first written to cache. The rate may be higher than regular input.
- Cache hit: the fee for reusing the cache in subsequent requests. Usually lower than regular input.
- Cache TTL (time to live): how long the cache is retained—commonly 5 minutes or 1 hour. Whether TTL affects the write price, and by how much, depends on the platform. Check the price list rather than assuming a universal rule.
Two practical points matter. First, put stable content (role definitions, tool descriptions, fixed knowledge segments) at the very beginning of the prompt, and put the user input that changes each time at the end—this maximizes prefix reuse. Second, if the interval between two requests exceeds the TTL, the cache expires and the next request pays the write fee again.
The recommended prompt structure is:
[Fixed system prompt: cacheable][Fixed knowledge segment: cacheable][Current user input: changes every time]
A simple example: a customer service bot has a system prompt of about 4,000 tokens, and each conversation only adds a few dozen new tokens. If the user’s question is placed before the system prompt, the prefix differs every time, the cache almost never hits, and all 4,000 tokens are billed as regular input on every call. If the system prompt comes first, the same prefix is reused repeatedly within the TTL, and most of the input is billed at the hit rate. The content is identical in both versions, but the bills can differ substantially.
2.3 Batch Offline Channels
For tasks that don’t need immediate results—data labeling, overnight report generation, batch translation, offline summarization—some platforms offer batch APIs. These are commonly priced at a discount; the exact discount and completion time window depend on the platform’s documentation.
Use this checklist to decide whether the batch channel fits:
- Latency tolerance: can the business accept completion times of minutes or even hours?
- Failure retries: when an individual request fails, can it be retried separately, and does the retry cause duplicate billing?
- Result validity: how long are result files retained, and can they be retrieved again after expiration?
- Result verification: does the number of returned results match the number submitted, and are failed items clearly marked?
- Request–result mapping: does the return order match the submission order? If not, use custom IDs to link requests to results.
Real-time interactive features—online customer support, instant completion—are not suitable for the batch channel. Focusing only on the discount without considering these boundaries often leads to failed jobs and rework, which actually raises the real cost.
2.4 The Break-Even Point Between Subscription Plans and Pay-as-You-Go
Plans typically provide a quota for a fixed monthly fee, with an overage rate applied beyond the quota. Whether a plan is worthwhile depends on total monthly usage.
Let P be the pay-as-you-go price (yuan per million tokens), F the monthly plan fee, Q the included quota (in millions of tokens), and O the overage rate (yuan per million tokens). For a monthly usage of U (in millions of tokens):
- Pay-as-you-go cost: U × P
- Plan cost: F + max(0, U − Q) × O
If usage stays within the quota, the plan cost is simply F. The break-even usage is U = F / P. When usage exceeds U but not Q, the plan is cheaper. Beyond Q, compare F + (U − Q) × O against U × P.
Hypothetical example: P = 10 yuan/million tokens, F = 3,000 yuan, Q = 500 million tokens, O = 6 yuan/million tokens. Break-even usage U* = 3000 / 10 = 300 million tokens. At 400 million tokens of monthly usage, pay-as-you-go costs 4,000 yuan versus 3,000 yuan for the plan—the plan wins. At 550 million tokens, the plan costs 3000 + 50 × 6 = 3,300 yuan versus 5,500 yuan pay-as-you-go—still cheaper.
Before choosing a plan, confirm these frequently overlooked points:
- Does the quota reset monthly, and can unused quota roll over?
- Do concurrency limits cover peak demand?
- Are cache-hit tokens counted against the quota? Conventions differ across platforms.
- Is the overage rate the same as the standard rate? Sometimes it’s significantly higher.
- What are the validity period and auto-renewal rules?
2.5 Time and Region Differences
Some platforms offer different prices by time of day or region, or route requests to service nodes in different regions. These differences directly affect the bill, but practices vary widely between platforms.
- Time-based differences: some platforms offer off-peak discounts or charge more during peak hours. Whether this exists, which models it applies to, and the discount size all need to be checked in the platform’s documentation.
- Regional differences: per-region pricing may differ, and data residency requirements may restrict which regions you can use. If your business has compliance requirements, confirm where data is processed before comparing prices.
- Routing differences: some platforms provide automatic routing that selects nodes based on load. The routing result can affect latency and cache hit rates, which in turn affect cost.
For latency-insensitive tasks like overnight batch processing, consider scheduling during off-peak hours and evaluating the batch channel at the same time. For online services, prioritize response speed—don’t sacrifice user experience to save money.
3. Cost Estimation Method and Sample Code
3.1 The Total Cost Formula
The cost of each call has four components:
Cost per call = uncached input tokens × input rate + cache-write tokens × write rate + cache-hit tokens × hit rate + output tokens × output rate
All rates are in yuan per million tokens, so divide the result by 1,000,000.
Monthly cost = cost per call × monthly call count × batch factor (1 for real-time calls)
If you use a subscription plan, compare the plan cost against pay-as-you-go using the break-even formula from Section 2.4 and take the lower value.
Three cautions for estimation. First, derive cache-write and cache-hit token counts from historical logs, not idealized assumptions. Second, when averaging output tokens, also look at the long-tail distribution—a small number of long responses can account for most of the cost. Third, estimate with a week of real logs before projecting a monthly budget.
3.2 Sample Code
The following code runs as-is. Prices are hypothetical and only demonstrate the calculation method.
from dataclasses import dataclass@dataclassclass Price:"""Rates in yuan per million tokens. Values are examples only;refer to the platform's price list for actual rates."""input_per_m: floatcache_write_per_m: floatcache_hit_per_m: floatoutput_per_m: floatbatch_factor: float = 1.0 # Batch channel factor; 1.0 means no discount@dataclassclass CallProfile:"""Average token counts per call."""input_tokens: int # Input not served from cachecache_write_tokens: int # Prefix written to cache in this callcache_hit_tokens: int # Prefix served from cache in this calloutput_tokens: int # Outputdef cost_per_call(price: Price, profile: CallProfile) -> float:total = (profile.input_tokens * price.input_per_m+ profile.cache_write_tokens * price.cache_write_per_m+ profile.cache_hit_tokens * price.cache_hit_per_m+ profile.output_tokens * price.output_per_m)return total / 1_000_000def monthly_cost(price: Price, profile: CallProfile, calls: int,batch: bool = False) -> float:cost = cost_per_call(price, profile) * callsreturn cost * price.batch_factor if batch else costdef plan_monthly_cost(total_tokens_m: float, plan_fee: float,quota_m: float, overage_per_m: float) -> float:"""Monthly plan cost. total_tokens_m is the month's total usagecounted against the quota, in millions of tokens."""overage = max(0.0, total_tokens_m - quota_m)return plan_fee + overage * overage_per_mif __name__ == "__main__":price = Price(input_per_m=2.0, cache_write_per_m=2.5,cache_hit_per_m=0.2, output_per_m=8.0, batch_factor=0.5)profile = CallProfile(input_tokens=1000, cache_write_tokens=0,cache_hit_tokens=4000, output_tokens=500)calls = 100_000realtime = monthly_cost(price, profile, calls)batch = monthly_cost(price, profile, calls, batch=True)print(f"Real-time monthly cost: {realtime:,.2f} yuan")print(f"Batch channel monthly cost: {batch:,.2f} yuan")# Total usage counted against the quota: input + cache hits + output,# converted to millions of tokenstotal_tokens_m = (profile.input_tokens + profile.cache_hit_tokens+ profile.output_tokens) * calls / 1_000_000plan = plan_monthly_cost(total_tokens_m, plan_fee=3000,quota_m=500, overage_per_m=6.0)print(f"Total monthly usage: {total_tokens_m:.0f}M tokens, "f"plan cost: {plan:,.2f} yuan")
In the output, the real-time monthly cost is 680 yuan, the batch channel cost is 340 yuan, and total monthly usage is 550 million tokens. The plan parameters in the code and the pay-as-you-go prices above are two independent example sets used to demonstrate the functions—they cannot be compared directly. To compare, plug the same set of prices into both.
3.3 Estimation Steps
- From logs, compute the average input, cache-write, cache-hit, and output token counts for each task type.
- Use the functions in Section 3.2 to compute cost per call, then multiply by monthly call volume.
- For offline-capable tasks, also compute the batch channel cost.
- Compare plan versus pay-as-you-go using Section 2.4, and choose the lower one.
- Leave headroom in the budget for usage fluctuations and price changes.
4. Selection Checklist
Go through this checklist when evaluating options:
- Is there an upper bound on output length, and do responses contain only necessary content?
- Does the prompt place stable content at the front, and have you confirmed the prefix is reused within the TTL?
- Is the interval between requests shorter than the cache TTL? If it frequently exceeds the TTL, consider adjusting call cadence.
- Have offline tasks been migrated to the batch channel, with retry and result-retention rules confirmed?
- Is monthly usage stable enough for a plan to cover most of it?
- Have you confirmed the plan’s quota accounting method, reset rules, and overage rate?
- Do off-peak or regional differences apply to your business, and are compliance requirements met?
- Have all prices and rules been verified against the latest price list?
5. Common Mistakes
Looking only at the unit price. With identical unit prices, different output lengths, cache structures, and call patterns can produce very different bills. Align the estimation basis before comparing.
Misestimating cache TTL. Many teams assume the cache stays valid indefinitely. In reality, once the interval between two requests exceeds the TTL, the cache expires and must be rewritten. Estimate hit rates using real request intervals.
Plan overage. Beyond the quota, the overage rate may exceed the standard rate. Looking only at the monthly fee without the overage rules underestimates cost.
Duplicate billing in batch jobs. If failed requests are resubmitted as a whole batch, the successful ones may be billed twice. Retry per item and deduplicate with custom IDs.
Ignoring output length. Output is usually priced higher. A few unnecessary sentences multiplied by a large call volume can cost more than the input you tried to trim.
Conclusion
Differences in LLM API bills come mainly from the combination of billing rules, not from the unit price itself. Break down input/output pricing, caching, batch channels, plans, and time/region differences; estimate with a unified formula; then choose based on your latency requirements and usage stability. This lets you control cost without sacrificing service quality. Always verify prices and rules against the platform’s latest documentation, and review them regularly.
