LLM API Cost: Why Monthly Bills Can Differ by Nearly Half at the Same Unit Prices

    This article explains why identical LLM API unit prices for input, cache hits, and output do not guarantee identical monthly bills. It breaks cost into usage mix, cache rules, batch discounts, free credits, subscription and idle-time rules, regional pricing, legacy model migration, and engineering overhead such as retries and context waste. Readers get a general accounting model, a provider comparison checklist, pseudocode for cache and monthly cost estimation, and a practical process for sampling real traffic, building a cost model, and validating it with shadow bills.

    When teams procure LLM services, they often start by comparing three core unit prices: input, cache hit, and output. If several providers offer exactly the same prices on these three tiers, the intuitive conclusion is that cost should be the same. In practice, a real monthly bill is not simply “unit price × usage.” It is the combined result of usage mix, billing rules, discounts, and credits. At the same unit prices, monthly bills can still differ by nearly half. The reasons usually lie not on the pricing page but in billing definitions, cache strategy, batch-processing share, free credits, subscription rules, regional price differences, and engineering-side retries and context waste.

    This article does not recommend any specific provider. Instead, it breaks “same prices, different bills” into a general accounting model that developers and enterprises can reuse for cost comparison.

    1. The Illusion of Identical Unit Prices: A Bill Is a Function of Usage Mix and Billing Rules

    Consider a simplified formula:

    1. monthly bill ≈ uncached input tokens × input unit price
    2. + cache-hit tokens × cache-hit unit price
    3. + cache-write tokens × cache-write unit price
    4. + output tokens × output unit price
    5. then multiplied by batch/idle-time/region discount factors
    6. then minus free credits, subscription credits, and similar offsets

    In this formula, unit prices determine only the linear part. What really separates bills is four types of variables:

    1. Usage mix: the ratio of input to output, the share of repeated prefixes, cache hit rate, and the share of batch tasks.
    2. Cache rules: whether caching is enabled automatically, how cache writes are billed, TTL length, and whether expired prefixes are written again.
    3. Discounts and credits: batch discounts, free credits for new models, subscription plans, idle-time rules, tiered pricing, or promotional pricing.
    4. Engineering amplifiers: failed retries, timeout resends, improper context truncation, log replay, duplicate requests, and peak concurrency.

    For example, two applications use the same model at the same unit prices. Application A is output-heavy and has a low cache hit rate. Application B relies on a fixed system prompt and repeated document Q&A, so it has a high cache hit rate. Even with identical unit prices, Application A’s bill will be noticeably higher because output unit prices are usually higher than input and cache-hit unit prices. In another case, two providers list the same three unit prices, but one bills cache writes separately while the other includes cache writes under the input definition. One uses a short default TTL, while the other offers a more expensive long TTL. The bill difference then comes from “invisible rules.”

    Therefore, comparing providers should not stop at a unit-price table. Teams should build a three-dimensional comparison table of unit price, rules, and usage mix. Below is an illustrative checklist framework that does not point to any specific provider:

    Check item Provider A Provider B Provider C Provider D
    Input/cache-hit/output unit prices Same Same Same Same
    Cache writes billed separately To confirm To confirm To confirm To confirm
    Default TTL and long-TTL pricing To confirm To confirm To confirm To confirm
    Batch discount and stacking rules To confirm To confirm To confirm To confirm
    Scope of free credits To confirm To confirm To confirm To confirm
    Subscription/idle-time rules To confirm To confirm To confirm To confirm
    Cross-region price differences To confirm To confirm To confirm To confirm

    The point of this table is not to fill in “yes/no.” It is to turn every “to confirm” into a verifiable billing parameter.

    2. Caching: The Most Underestimated Variable

    Caching is the first major source of “same prices, different bills.” Many teams know that cache hits are billed only at the hit price, but they overlook three details.

    First, whether repeated prefixes automatically trigger caching. If system prompts, tool descriptions, and fixed knowledge snippets are placed at the front of requests and remain stable over time, the cache hit rate will be higher. If every request puts timestamps, random IDs, or dynamic user information at the front, the prefix keeps changing and caching becomes difficult. From an engineering perspective, stable content should be placed first, dynamic content later, and stable prompt templates should be designed for different business scenarios.

    Second, whether cache writes are billed separately. Cache-hit prices are usually lower than regular input prices, but cache writes may be billed at the input price, at a write price, or separately by TTL. If the workload contains many first-time requests and prefixes change frequently, cache-write cost can offset the benefit of hits. Teams should track the ratio of cache-write volume to cache-hit volume, not just the hit rate.

    Third, TTL and expired rewrites. Short TTLs suit high-frequency, time-sensitive scenarios. Long TTLs suit stable knowledge bases and fixed system prompts, but the write price for long TTLs may be higher. If the TTL is too short, the cache expires frequently and the system rewrites repeatedly. If the TTL is too long, write cost rises while hit benefits may not grow correspondingly. Choose TTL based on the request-interval distribution rather than using the default value uniformly.

    The following pseudocode can be used to estimate cache-related cost:

    1. function cache_cost(usage, price, ttlPolicy):
    2. hitCost = usage.cacheHitTokens * price.cacheHit
    3. writeCost = usage.cacheWriteTokens * price.cacheWrite(ttlPolicy)
    4. missCost = usage.uncachedInputTokens * price.input
    5. expiredRewriteCost = usage.expiredPrefixTokens * price.cacheWrite(ttlPolicy)
    6. return hitCost + writeCost + missCost + expiredRewriteCost

    If two providers have the same three unit prices, but Provider A has lower cache-write cost and cheaper short TTLs while Provider B charges more for long-TTL writes, Provider A may have a significantly lower monthly bill in high-repeat-prefix scenarios. Conversely, in low-repeat, long-context scenarios, cache benefits are limited and the difference narrows again.

    3. Batch Processing, Free Credits, and Subscription Idle Time: Different Discount Definitions

    Batch processing is the second common difference. Non-real-time tasks such as offline summarization, batch classification, data cleaning, and report generation can usually tolerate higher latency and may therefore qualify for batch discounts. However, teams need to confirm: Does the batch discount stack with cache hits? Does it stack with free credits? Is it limited to certain models, task durations, concurrency levels, or submission methods? If batch tasks account for a high share of usage, the discount definition can significantly change the monthly bill. If the business is mainly real-time conversation, batch discounts are almost irrelevant.

    Free credits are also easy to misread. New models or new users may receive one-time free credits, but credits usually have scope, expiration, whether they offset cache writes, and whether they are limited to specific regions. Free credits are suitable for validation and cold start, but they should not be included in long-term unit cost. When budgeting, remove free credits from the “long-term average price” and list them separately as a one-time offset.

    Subscription and idle-time rules also need to be broken down. A subscription plan may cover some models or a certain amount of usage, with overage billed on demand. Idle-time rules may apply only during specific hours and may impose constraints on task type, concurrency, and response time. If business traffic is concentrated during peak hours, the actual coverage of idle-time discounts may be very low. The right approach is to amortize the fixed subscription fee over the usage it can cover and then compare it with a pure pay-as-you-go plan:

    1. subscription equivalent unit price = (monthly subscription fee + overage pay-as-you-go fee) / total effective monthly tokens
    2. pure pay-as-you-go unit price = pay-as-you-go bill / total effective monthly tokens

    Only by including fixed fees, coverage scope, idle-time coverage, and overage rules can you judge whether a subscription is truly cheaper.

    4. Legacy Model Retirement and Migration Cost: Hidden Expenses Beyond Unit Price

    Legacy model retirement is another cost item that is often overlooked. After an older model stops being available to new users or is scheduled for retirement, teams need to migrate to a newer model. Migration is not just changing a model identifier: prompt templates, output length, cache-hit behavior, tool-call formats, evaluation sets, regression tests, canary releases, and rollback plans all need to be revalidated. The new model may have the same unit price as the old one, but differences in output style, truncation behavior, and retry rate can still change the actual bill.

    Therefore, provider comparisons should include migration cost: evaluation and regression labor, dual-running cost during canary periods, extra tokens from failed retries, and monitoring and alerting changes. If an old model still has a large volume of existing calls, a shorter migration window increases engineering risk. Teams should build a model abstraction layer in advance and make model identifiers, billing parameters, cache strategy, and fallback strategy configurable, so migration does not require large-scale code changes.

    5. Regions and Networks: Additional Items Beyond the Unit-Price Table

    The same service may have different unit prices in different regions. Cross-region deployments often involve higher resource, compliance, and network costs, so unit prices may be higher than in the primary region. If the business serves users in multiple regions, calculate a weighted unit price based on traffic distribution rather than looking only at the primary-region price. Cross-region calls may also bring additional network latency, data transfer fees, cross-border log storage, and compliance review costs. For latency-sensitive businesses, choosing a closer region may reduce retry rates and therefore indirectly lower the bill. For offline tasks, regional price differences can be incorporated into batch scheduling strategy.

    6. Engineering-Side Amplifiers: Retries, Context, and Failed Requests

    Many bill differences do not come from the provider but from the caller’s engineering implementation. Common amplifiers include:

    • Failed retries: timeouts, rate limits, and network jitter cause retries. If failed requests are billed or partially billed, cost rises.
    • Excessive context: putting irrelevant history, full logs, and repeated documents into requests causes input tokens to grow rapidly.
    • Improper truncation: after output is truncated, another request is made, producing duplicate input and duplicate output.
    • Unstable cache prefixes: dynamic fields placed at the front reduce cache hit rate.
    • Peak concurrency: retries after peak-hour rate limiting create additional traffic.
    • Lack of cost attribution: without tags by business line, tenant, and task type, high-consumption sources cannot be located.

    Applications should record input tokens, cache-hit tokens, cache-write tokens, output tokens, retry count, task type, and region for each request, then aggregate daily. Only with the real usage mix can a unit-price table be turned into a credible monthly bill forecast.

    7. Practical Accounting Method and Checklist

    An executable comparison process is as follows:

    1. Sample real requests: draw representative samples from production traffic, covering peak/off-peak, short/long context, and real-time/offline tasks.
    2. Measure usage mix: calculate the shares of uncached input, cache hits, cache writes, output, retries, and batch processing.
    3. Collect billing rules: confirm cache-write billing, TTL, batch stacking, free credits, subscription idle time, regional price differences, minimum billing granularity, and failed-request billing.
    4. Build a cost model: use a unified formula to calculate each provider’s monthly bill and perform sensitivity analysis.
    5. Run small-traffic shadow bills: send the same batch of requests through different providers, record actual bills, and calibrate model parameters.
    6. Set budget alerts: set daily/weekly budgets by business line, along with alerts for cache hit rate, retry rate, and output length.

    An illustrative cost model is shown below:

    1. function monthly_cost(usage, price, rules):
    2. subtotal = usage.input_uncached * price.input
    3. + usage.cache_hit * price.cache_hit
    4. + usage.cache_write * price.cache_write(usage.ttl)
    5. + usage.output * price.output
    6. if usage.is_batch:
    7. subtotal = subtotal * rules.batch_discount
    8. subtotal = subtotal * rules.region_multiplier
    9. subtotal = subtotal - min(rules.free_credit, subtotal)
    10. if rules.subscription_covers(usage):
    11. subtotal = subtotal + rules.subscription_fee - rules.subscription_credit
    12. return subtotal

    The checklist should include at least: whether the three unit prices include tax; whether cache writes are listed separately; default TTL and long-TTL pricing; cache-hit determination conditions; whether batch processing stacks with caching; free-credit validity period and scope; subscription coverage and overage unit prices; idle-time windows and task restrictions; cross-region price differences; whether failed requests, retries, and truncation are billed; minimum billing unit; and legacy model retirement dates and migration guidance.

    8. Conclusion: Same Price Is Only the Starting Point; Rules and Usage Mix Determine the Bill

    When several providers have exactly the same three unit prices, monthly bills can still differ by nearly half. The root cause is different billing definitions and usage mix. Cache writes and TTL, batch discounts, free credits, subscription idle time, regional price differences, legacy model migration, and engineering-side retry waste can all amplify or reduce cost beyond the unit price. When selecting a provider, teams should not stop at the pricing page. They should build an accounting system based on “unit price + rules + real usage”: first model with formulas, then calibrate with small-traffic shadow bills, and finally include cache hit rate, retry rate, output length, and batch-processing share in continuous monitoring. Only by calculating the real cost can teams avoid being misled by “same price” and make more robust technical and budget decisions.