Why Identical LLM API Prices Produce Very Different Bills: A Field Guide to Token Billing Rules

    LLM API pricing goes far beyond the per-token rate on a price sheet. Monthly costs are shaped by three factors: unit prices for each token type, the composition of your usage (input, output, and cache hits), and platform-level rules such as offline discounts, subscription plans, and free tiers. This article breaks down the core dimensions of token-based billing—including cache write fees, time-of-day coefficients, and batch discounts—walks through a concrete cost calculation example with code, and provides practical guidance on choosing between real-time and offline APIs, designing prompts for cache efficiency, evaluating subscription versus pay-as-you-go plans, and building a cost-monitoring system. A final checklist helps you audit any platform’s billing rules before committing to a purchase.

    Why the Same Unit Price Doesn’t Mean the Same Bill

    The per-token unit price is only one part of an LLM API bill. Your monthly cost is determined by three things: the unit price for each token category, the structure of your usage (how much is input, output, or cache-hit), and any rules the platform applies on top of the base price—such as offline discounts, off-peak pricing, subscription credits, or free allowances.

    This means two platforms can quote the exact same per-token price for the same model, yet produce very different invoices for identical usage—if one of them offers discount rules and the other doesn’t. When estimating costs, start by reading the price list, then systematically review the rules page, and finally run calculations based on your own real usage patterns.

    The Four Core Dimensions of Pay-as-You-Go Billing

    1. Input, Cache Hits, and Output Are Priced Separately

    Most platforms divide tokens into three billing categories:

    • Uncached input tokens: Billed at the standard input rate, typically one of the highest per-token prices.
    • Cache-hit input tokens: When a previously cached prefix is reused, these tokens are billed at a much lower rate than standard input.
    • Output tokens: Tokens generated by the model. These are usually priced significantly higher than input tokens.

    Output tokens are often the dominant cost driver. For workloads that generate long documents or large blocks of code, reducing output length saves more money than compressing the input.

    2. Cache Writes and Cache TTL

    Caching is one of the most frequently overlooked cost factors. The typical mechanism works as follows:

    • If the beginning of a request exactly matches a historical request, the system can reuse the computed result—this is a cache hit.
    • Caches have a time-to-live (TTL), often ranging from a few minutes to an hour. Once the TTL expires, the prefix must be written to the cache again.
    • Some platforms charge separately for cache writes, and differentiate pricing based on TTL length. A longer TTL costs more per write but enables reuse over a longer time window.

    Cache savings therefore depend on two conditions: how often a prefix is reused, and whether the interval between reuses is shorter than the TTL. If your request intervals frequently exceed the TTL, write costs can erase the savings from cache hits.

    3. Offline Batch Discounts

    For workloads that don’t require immediate responses—such as data labeling, document summarization, or batch evaluation—some platforms offer offline batch endpoints at a reduced rate, typically between 50% and 70% of the standard price. The trade-off is delayed results, often delivered within a few hours or by the end of the day.

    Whether offline APIs are viable depends on whether your business can tolerate asynchronous results. Converting a real-time request flow to offline processing requires reworking your task queue and result-writeback logic, and that engineering cost should be factored into your evaluation.

    4. Subscriptions and Overages

    Subscription plans typically bundle a fixed monthly fee with a set quota. Usage beyond the quota is billed at an overage rate. The break-even point of such a plan needs to be calculated separately. Subscriptions make sense for workloads with stable, high-volume usage; pay-as-you-go is better for spiky or early-stage workloads.

    Some platforms also apply different billing coefficients depending on the time of day—for example, discounts during certain hours. These rules mean the same workload can produce different costs at different times, so accurate accounting requires logging usage on an hourly basis.

    An Example: Deconstructing a Bill

    To demonstrate the calculation method, let’s use hypothetical prices. These numbers do not represent any real platform’s pricing and are used only to illustrate the calculation structure.

    Assume the following monthly usage:

    • Input tokens: 50 million total, of which 40 million are cache hits and 10 million are uncached;
    • Output tokens: 5 million;
    • Example prices: uncached input at ¥10 per million, cache-hit input at ¥1 per million, output at ¥40 per million.

    Pay-as-you-go cost:

    • Uncached input: 10M × ¥10/M = ¥100
    • Cache-hit input: 40M × ¥1/M = ¥40
    • Output: 5M × ¥40/M = ¥200
    • Total: ¥340

    If the platform offers a 60% discount for offline batch processing, the same workload through the offline endpoint costs 340 × 0.6 = ¥204.

    Putting this calculation into a function makes it easy to compare multiple plans:

    1. from dataclasses import dataclass
    2. @dataclass
    3. class Pricing:
    4. input_price: float # uncached input, currency per million tokens
    5. cache_hit_price: float # cache-hit input, currency per million tokens
    6. output_price: float # output, currency per million tokens
    7. offline_factor: float = 1.0 # offline batch coefficient, e.g., 0.6 for 60%
    8. def monthly_cost(p: Pricing, uncached_in_m: float, cached_in_m: float,
    9. output_m: float, offline: bool = False) -> float:
    10. """Usage in millions of tokens; returns monthly cost. Does not include cache write fees."""
    11. factor = p.offline_factor if offline else 1.0
    12. total = (uncached_in_m * p.input_price
    13. + cached_in_m * p.cache_hit_price
    14. + output_m * p.output_price)
    15. return round(total * factor, 2)
    16. pricing = Pricing(input_price=10, cache_hit_price=1, output_price=40,
    17. offline_factor=0.6)
    18. print(monthly_cost(pricing, 10, 40, 5)) # real-time: 340.0
    19. print(monthly_cost(pricing, 10, 40, 5, offline=True)) # offline: 204.0

    Note that this example function does not include cache write fees. In a real cost analysis, you’ll need to add that line item based on the number of writes and the per-write price.

    Real-Time vs. Offline: How to Choose

    The choice comes down to three questions:

    1. Does the result need to be returned immediately? User-facing conversations, online search, and interactive features must use real-time endpoints.
    2. Can the task be batched? Document processing and offline evaluation can typically be collected and submitted in batches, making them good candidates for offline APIs.
    3. Can the downstream system tolerate latency? Offline results may take hours, so you need to confirm your pipeline can accept that time window.

    In practice, it’s common to separate real-time and offline traffic. Online services stay responsive, while offline jobs are centrally scheduled—this keeps costs controlled and makes monitoring simpler.

    Engineering Practices for Cache-Efficient Prompt Design

    To make caching actually reduce costs, prompt structure needs careful design:

    • Put stable content at the front: System instructions, role definitions, fixed examples, and other invariant parts should be placed at the beginning of the request to ensure a consistent prefix.
    • Put dynamic content at the back: User queries, retrieved context, timestamps, and other variable parts belong at the end, so they don’t break prefix consistency.
    • Avoid meaningless variation: If a request contains random IDs or per-request timestamps, the prefix will break and caching will never hit.
    • Choose TTL based on reuse intervals: For workloads with short reuse intervals, a shorter TTL reduces write costs. For long-interval reuse, you’ll need to evaluate whether the write cost of a longer TTL is justified.

    Use the following metrics to verify the effectiveness of caching: the ratio of cache-hit tokens to total input tokens, the number of cache writes, and the average input cost per request. These metrics should be tracked separately by feature module—overall averages can hide problems that are concentrated in specific flows.

    Subscription vs. Pay-as-You-Go: Finding the Break-Even Point

    Comparing subscription and pay-as-you-go plans comes down to a single equation. Let F be the monthly fee, Q be the included quota (in millions of tokens), p be the overage price per million tokens, and U be your monthly usage (in millions of tokens). The subscription cost is:

    Subscription cost = F + max(0, U − Q) × p

    The pay-as-you-go cost is calculated by your actual metered usage. The break-even point is where these two values are equal.

    1. def subscription_cost(fee: float, quota_m: float,
    2. overage_price: float, usage_m: float) -> float:
    3. overage = max(0.0, usage_m - quota_m)
    4. return round(fee + overage * overage_price, 2)

    Two points deserve attention when making this decision. First, usage forecasts should include peaks, not just averages. Second, subscription quotas usually have a validity period—unused quota generally doesn’t roll over, so costs in low-usage months can be higher than expected.

    The Risk of Model Version Iteration

    LLM versions evolve quickly. Older versions may stop being offered to new users and eventually be deprecated entirely. This creates three types of risk:

    • Forced migration: When a version shuts down, services depending on it must switch, and the migration window may be tight.
    • Output differences: Newer versions may differ in response style, formatting adherence, and reasoning capability—your existing prompts may not carry over unchanged.
    • Cost changes: New versions may carry a higher or lower unit price, changing your bill structure after migration.

    Mitigations include: managing model names centrally in configuration rather than hard-coding them; maintaining a fixed evaluation set for core use cases and running regression tests before every migration; and tracking version release and deprecation dates so you can plan migration windows in advance.

    Setting Up Cost Monitoring

    Cost governance starts with traceable usage data. The following practices are recommended:

    1. Log input, cache-hit, and output token counts for every request, along with the model and request type.
    2. Tag requests by feature module, business line, or tenant to pinpoint where costs originate.
    3. Set budget thresholds—trigger alerts when daily usage or monthly costs approach predefined limits.
    4. Periodically reconcile platform invoices with your internal statistics to confirm the billing dimensions match.

    These data points can be written to a standard logging service and monitored through your existing alerting system. Cost anomalies often stem from accidentally inflated prompts, cache invalidation, or runaway loops. Early alerting keeps losses contained.

    A Selection Checklist

    Before committing to a platform or plan, run through the following checks:

    • Confirm whether the price list quotes per million tokens or per thousand tokens, and whether currency and tax treatment are specified.
    • Verify that the price quotation date and region match your deployment location.
    • Confirm whether input, cache-hit, cache-write, and output are billed separately, and whether any line items are missing.
    • Determine whether offline discounts are available and whether your business can accept asynchronous results.
    • Check whether a subscription plan exists and at what usage level the break-even point falls.
    • Check for time-of-day coefficients or free allowances, and confirm that your measurement methodology accounts for them.
    • Understand the lifecycle of the chosen model version and your migration plan.
    • Confirm you have the ability to track usage by feature module and set budget alerts.

    Combine these checks with your real usage numbers and you can produce a reliable monthly budget. Two platforms with identical unit prices can differ enormously in practice—the gap comes from these rules. The cost of auditing them line by line is far lower than the cost of optimizing a bill after the fact.