LLM API Cost Breakdown: Why Identical Unit Prices Can Produce Very Different Bills

    The per-token price of an LLM API is only one input to your final bill. This article breaks down the billing dimensions that actually determine cost—separate input/output pricing, prefix caching, batch offline channels, subscription plans, and time/region differences—then provides a unified estimation formula and ready-to-use Python code. You will learn how to structure prompts for cache hits, calculate the break-even point between pay-as-you-go and subscription plans, avoid common billing mistakes, and build realistic monthly cost estimates from real logs.

    1. Same Unit Price, Different Bills

    The pay-as-you-go unit price is just one input to your bill. The actual cost of a single API call is determined by multiple factors working together: input token count, output token count, whether prefix caching is hit, whether the call goes through an offline batch channel, whether a subscription plan applies, and when and where the call happens. Two teams calling the same model can end up with very different month-end bills if they differ on just one or two of these factors.

    So before comparing prices, you need to align the estimation basis. Looking only at the unit price makes it easy to conclude “every provider costs the same,” while ignoring the billing rules that truly determine cost. This article breaks down each billing dimension one by one, and ends with estimation code you can modify and use directly.

    All price figures in this article are hypothetical examples used to demonstrate calculation methods. Actual costs depend on each platform’s price list and billing documentation, and platform rules may change.

    2. Breaking Down the Main Billing Dimensions

    First an overview, then details for each item.

    Billing Dimension Main Impact Optimizable? Applicable Scenarios
    Separate input/output pricing Output is usually priced higher Yes—control output length and context All calls
    Prefix caching Write and hit fees for repeated prefixes Yes—adjust prompt structure Fixed system prompts, fixed knowledge segments
    Batch offline channel Commonly discounted, results returned with delay Yes—convert to offline jobs Data labeling, report generation, batch translation
    Subscription plans Fixed monthly fee plus a quota Yes—choose based on usage Stable, predictable usage
    Time and region Prices or routing may differ by time or region Platform-dependent Latency-insensitive tasks

    2.1 Separate Pricing for Input and Output

    Most LLM APIs price per million tokens, with different rates for input and output. Output is typically priced significantly higher than input. Even with the same 100,000 calls, a business generating long-form answers has a completely different cost structure from one doing intent classification.

    • Input cost: system prompts, conversation history, retrieved reference material
    • Output cost: the model’s generated responses, structured results, and reasoning content that some platforms count as output

    There are two optimization directions: control output length, and remove context irrelevant to the current task. The former usually pays off faster because output costs more per token.

    2.2 The Three Types of Prefix Caching Fees

    When multiple requests share the same prefix, some platforms enable caching for that prefix. Cache-related fees typically fall into three categories:

    1. Cache write: the fee incurred when a prefix is first written to cache. The rate may be higher than regular input.
    2. Cache hit: the fee for reusing the cache in subsequent requests. Usually lower than regular input.
    3. Cache TTL (time to live): how long the cache is retained—commonly 5 minutes or 1 hour. Whether TTL affects the write price, and by how much, depends on the platform. Check the price list rather than assuming a universal rule.

    Two practical points matter. First, put stable content (role definitions, tool descriptions, fixed knowledge segments) at the very beginning of the prompt, and put the user input that changes each time at the end—this maximizes prefix reuse. Second, if the interval between two requests exceeds the TTL, the cache expires and the next request pays the write fee again.

    The recommended prompt structure is:

    1. [Fixed system prompt: cacheable]
    2. [Fixed knowledge segment: cacheable]
    3. [Current user input: changes every time]

    A simple example: a customer service bot has a system prompt of about 4,000 tokens, and each conversation only adds a few dozen new tokens. If the user’s question is placed before the system prompt, the prefix differs every time, the cache almost never hits, and all 4,000 tokens are billed as regular input on every call. If the system prompt comes first, the same prefix is reused repeatedly within the TTL, and most of the input is billed at the hit rate. The content is identical in both versions, but the bills can differ substantially.

    2.3 Batch Offline Channels

    For tasks that don’t need immediate results—data labeling, overnight report generation, batch translation, offline summarization—some platforms offer batch APIs. These are commonly priced at a discount; the exact discount and completion time window depend on the platform’s documentation.

    Use this checklist to decide whether the batch channel fits:

    • Latency tolerance: can the business accept completion times of minutes or even hours?
    • Failure retries: when an individual request fails, can it be retried separately, and does the retry cause duplicate billing?
    • Result validity: how long are result files retained, and can they be retrieved again after expiration?
    • Result verification: does the number of returned results match the number submitted, and are failed items clearly marked?
    • Request–result mapping: does the return order match the submission order? If not, use custom IDs to link requests to results.

    Real-time interactive features—online customer support, instant completion—are not suitable for the batch channel. Focusing only on the discount without considering these boundaries often leads to failed jobs and rework, which actually raises the real cost.

    2.4 The Break-Even Point Between Subscription Plans and Pay-as-You-Go

    Plans typically provide a quota for a fixed monthly fee, with an overage rate applied beyond the quota. Whether a plan is worthwhile depends on total monthly usage.

    Let P be the pay-as-you-go price (yuan per million tokens), F the monthly plan fee, Q the included quota (in millions of tokens), and O the overage rate (yuan per million tokens). For a monthly usage of U (in millions of tokens):

    • Pay-as-you-go cost: U × P
    • Plan cost: F + max(0, U − Q) × O

    If usage stays within the quota, the plan cost is simply F. The break-even usage is U = F / P. When usage exceeds U but not Q, the plan is cheaper. Beyond Q, compare F + (U − Q) × O against U × P.

    Hypothetical example: P = 10 yuan/million tokens, F = 3,000 yuan, Q = 500 million tokens, O = 6 yuan/million tokens. Break-even usage U* = 3000 / 10 = 300 million tokens. At 400 million tokens of monthly usage, pay-as-you-go costs 4,000 yuan versus 3,000 yuan for the plan—the plan wins. At 550 million tokens, the plan costs 3000 + 50 × 6 = 3,300 yuan versus 5,500 yuan pay-as-you-go—still cheaper.

    Before choosing a plan, confirm these frequently overlooked points:

    • Does the quota reset monthly, and can unused quota roll over?
    • Do concurrency limits cover peak demand?
    • Are cache-hit tokens counted against the quota? Conventions differ across platforms.
    • Is the overage rate the same as the standard rate? Sometimes it’s significantly higher.
    • What are the validity period and auto-renewal rules?

    2.5 Time and Region Differences

    Some platforms offer different prices by time of day or region, or route requests to service nodes in different regions. These differences directly affect the bill, but practices vary widely between platforms.

    • Time-based differences: some platforms offer off-peak discounts or charge more during peak hours. Whether this exists, which models it applies to, and the discount size all need to be checked in the platform’s documentation.
    • Regional differences: per-region pricing may differ, and data residency requirements may restrict which regions you can use. If your business has compliance requirements, confirm where data is processed before comparing prices.
    • Routing differences: some platforms provide automatic routing that selects nodes based on load. The routing result can affect latency and cache hit rates, which in turn affect cost.

    For latency-insensitive tasks like overnight batch processing, consider scheduling during off-peak hours and evaluating the batch channel at the same time. For online services, prioritize response speed—don’t sacrifice user experience to save money.

    3. Cost Estimation Method and Sample Code

    3.1 The Total Cost Formula

    The cost of each call has four components:

    Cost per call = uncached input tokens × input rate + cache-write tokens × write rate + cache-hit tokens × hit rate + output tokens × output rate

    All rates are in yuan per million tokens, so divide the result by 1,000,000.

    Monthly cost = cost per call × monthly call count × batch factor (1 for real-time calls)

    If you use a subscription plan, compare the plan cost against pay-as-you-go using the break-even formula from Section 2.4 and take the lower value.

    Three cautions for estimation. First, derive cache-write and cache-hit token counts from historical logs, not idealized assumptions. Second, when averaging output tokens, also look at the long-tail distribution—a small number of long responses can account for most of the cost. Third, estimate with a week of real logs before projecting a monthly budget.

    3.2 Sample Code

    The following code runs as-is. Prices are hypothetical and only demonstrate the calculation method.

    1. from dataclasses import dataclass
    2. @dataclass
    3. class Price:
    4. """Rates in yuan per million tokens. Values are examples only;
    5. refer to the platform's price list for actual rates."""
    6. input_per_m: float
    7. cache_write_per_m: float
    8. cache_hit_per_m: float
    9. output_per_m: float
    10. batch_factor: float = 1.0 # Batch channel factor; 1.0 means no discount
    11. @dataclass
    12. class CallProfile:
    13. """Average token counts per call."""
    14. input_tokens: int # Input not served from cache
    15. cache_write_tokens: int # Prefix written to cache in this call
    16. cache_hit_tokens: int # Prefix served from cache in this call
    17. output_tokens: int # Output
    18. def cost_per_call(price: Price, profile: CallProfile) -> float:
    19. total = (
    20. profile.input_tokens * price.input_per_m
    21. + profile.cache_write_tokens * price.cache_write_per_m
    22. + profile.cache_hit_tokens * price.cache_hit_per_m
    23. + profile.output_tokens * price.output_per_m
    24. )
    25. return total / 1_000_000
    26. def monthly_cost(price: Price, profile: CallProfile, calls: int,
    27. batch: bool = False) -> float:
    28. cost = cost_per_call(price, profile) * calls
    29. return cost * price.batch_factor if batch else cost
    30. def plan_monthly_cost(total_tokens_m: float, plan_fee: float,
    31. quota_m: float, overage_per_m: float) -> float:
    32. """Monthly plan cost. total_tokens_m is the month's total usage
    33. counted against the quota, in millions of tokens."""
    34. overage = max(0.0, total_tokens_m - quota_m)
    35. return plan_fee + overage * overage_per_m
    36. if __name__ == "__main__":
    37. price = Price(input_per_m=2.0, cache_write_per_m=2.5,
    38. cache_hit_per_m=0.2, output_per_m=8.0, batch_factor=0.5)
    39. profile = CallProfile(input_tokens=1000, cache_write_tokens=0,
    40. cache_hit_tokens=4000, output_tokens=500)
    41. calls = 100_000
    42. realtime = monthly_cost(price, profile, calls)
    43. batch = monthly_cost(price, profile, calls, batch=True)
    44. print(f"Real-time monthly cost: {realtime:,.2f} yuan")
    45. print(f"Batch channel monthly cost: {batch:,.2f} yuan")
    46. # Total usage counted against the quota: input + cache hits + output,
    47. # converted to millions of tokens
    48. total_tokens_m = (profile.input_tokens + profile.cache_hit_tokens
    49. + profile.output_tokens) * calls / 1_000_000
    50. plan = plan_monthly_cost(total_tokens_m, plan_fee=3000,
    51. quota_m=500, overage_per_m=6.0)
    52. print(f"Total monthly usage: {total_tokens_m:.0f}M tokens, "
    53. f"plan cost: {plan:,.2f} yuan")

    In the output, the real-time monthly cost is 680 yuan, the batch channel cost is 340 yuan, and total monthly usage is 550 million tokens. The plan parameters in the code and the pay-as-you-go prices above are two independent example sets used to demonstrate the functions—they cannot be compared directly. To compare, plug the same set of prices into both.

    3.3 Estimation Steps

    1. From logs, compute the average input, cache-write, cache-hit, and output token counts for each task type.
    2. Use the functions in Section 3.2 to compute cost per call, then multiply by monthly call volume.
    3. For offline-capable tasks, also compute the batch channel cost.
    4. Compare plan versus pay-as-you-go using Section 2.4, and choose the lower one.
    5. Leave headroom in the budget for usage fluctuations and price changes.

    4. Selection Checklist

    Go through this checklist when evaluating options:

    • Is there an upper bound on output length, and do responses contain only necessary content?
    • Does the prompt place stable content at the front, and have you confirmed the prefix is reused within the TTL?
    • Is the interval between requests shorter than the cache TTL? If it frequently exceeds the TTL, consider adjusting call cadence.
    • Have offline tasks been migrated to the batch channel, with retry and result-retention rules confirmed?
    • Is monthly usage stable enough for a plan to cover most of it?
    • Have you confirmed the plan’s quota accounting method, reset rules, and overage rate?
    • Do off-peak or regional differences apply to your business, and are compliance requirements met?
    • Have all prices and rules been verified against the latest price list?

    5. Common Mistakes

    Looking only at the unit price. With identical unit prices, different output lengths, cache structures, and call patterns can produce very different bills. Align the estimation basis before comparing.

    Misestimating cache TTL. Many teams assume the cache stays valid indefinitely. In reality, once the interval between two requests exceeds the TTL, the cache expires and must be rewritten. Estimate hit rates using real request intervals.

    Plan overage. Beyond the quota, the overage rate may exceed the standard rate. Looking only at the monthly fee without the overage rules underestimates cost.

    Duplicate billing in batch jobs. If failed requests are resubmitted as a whole batch, the successful ones may be billed twice. Retry per item and deduplicate with custom IDs.

    Ignoring output length. Output is usually priced higher. A few unnecessary sentences multiplied by a large call volume can cost more than the input you tried to trim.

    Conclusion

    Differences in LLM API bills come mainly from the combination of billing rules, not from the unit price itself. Break down input/output pricing, caching, batch channels, plans, and time/region differences; estimate with a unified formula; then choose based on your latency requirements and usage stability. This lets you control cost without sacrificing service quality. Always verify prices and rules against the platform’s latest documentation, and review them regularly.