# Multi-Model LLM API Cost Comparison: Four Billing Models Broken Down and How to Choose

    Price lists are not cost estimates. This article explains why a low unit price can still mean a high bill, breaks down the four billing mechanisms behind most LLM APIs — time-of-day pricing, tiered context caching, mutually exclusive discounts, and flat pricing with context limits — and shows how they stack into a highly nonlinear bill. It includes a break-even formula for cache economics, a runnable Python cost estimator, a four-step selection workflow, and guidance on turning cost into a monitored runtime metric. Readers will learn which traffic data to collect before comparing models, how to choose the right path for interactive versus offline workloads, and which pitfalls to avoid when budgeting.}

    1. First, kill one myth: a low unit price does not mean a low cost

    When many teams pick a model, the first thing they do is open a price sheet, sort the rows by “input price + output price”, and take the cheapest one. As an engineering decision, that step is almost always wrong, for a simple reason: a price sheet quotes unit rates, while a bill settles on total volume multiplied by structure.

    Take a common case: a request with 6,000 input tokens and 300 output tokens, an input-to-output ratio of 20:1. A model whose unit price is 20% lower, but whose cache-hit discount is weaker, which does not support batch requests, or which adds a peak-hour surcharge, can easily end up more expensive overall. The reverse holds too: for a code-generation workload where the output token share is high, the output price dominates, and input-side savings are barely noticeable.

    So before asking “which one is cheaper”, you have to answer three questions:

    1. What is the real input-to-output ratio? This sets how the unit prices are weighted.
    2. How is request volume distributed over time? Concentrated in weekday business hours, or run as overnight batches?
    3. Are system prompts, long documents, and few-shot examples reusable? This determines whether caching activates at all.

    Those three answers, not the price sheet, are the real input to cost accounting.

    2. Structural breakdown of the four mainstream billing models

    Abstracting current market pricing gives roughly four mechanisms. They are usually not mutually exclusive; they stack onto the same bill.

    2.1 Time-of-day pricing: two prices, peak and off-peak

    Some platforms publish two rates per request time slot. The table below shows the structure (the numbers illustrate the structure only, not any vendor’s real pricing):

    Tier Input (cache miss) Input (cache hit) Output
    Peak 2.0 0.04 8.0
    Off-peak 1.0 0.02 4.0

    Units: $ per 1M tokens. Peak windows are typically fixed blocks during weekday business hours (for example, one in the morning and one in the afternoon). Everything else — nights, weekends, and all day on public holidays — falls in the off-peak tier, usually half the peak rate.

    Three details here are easy to trip over:

    • Tier assignment follows request completion time, not start time. A 60-second long-output job launched at 19:59 and finishing at 20:00 is billed as off-peak, and the other way round. Long jobs that straddle a boundary see price jitter, so your budget model needs headroom.
    • Holiday classification follows the official calendar, not the natural weekend. Weekends designated as working days (typical in regions that shift working days to consolidate holidays) are usually still billed at off-peak rates. Good news for cost, but scheduling scripts must not take it for granted.
    • Cache-hit price is tiered too. The gap between peak and off-peak hit prices compounds under heavy reuse, so do not estimate the hit price as simply half of the input price.

    2.2 Tiered cache pricing: hit, write, storage

    Context caching (prefill cache) is currently the most effective lever for cutting long-context cost, but its billing is not a simple “discount”. It splits into three segments:

    • Cache hit: the cached portion is billed at a rate well below normal input, commonly 10%~20% of the original.
    • Cache write: the first write into cache costs extra. Some platforms price a 5-minute TTL and a 1-hour TTL separately, with the latter typically about twice the former.
    • Cache storage: billed per 1M tokens per hour. Even when no request hits, storage accumulates as long as the cache entry stays inside its validity window.

    That gives an important decision rule: whether caching pays off is governed by hit rate, not by how large the discount looks on a single request. A simple break-even condition:

    1. Let:
    2. P_in = standard input unit price
    3. P_hit = cache hit unit price
    4. P_wr = cache write unit price (one-off per TTL window)
    5. P_st = cache storage unit price ($/1M tokens/hour)
    6. T = hours the cache stays alive
    7. N = number of hits per TTL window
    8. L = token volume of the cached content (in millions)
    9. Savings = N × L × (P_in - P_hit)
    10. Cost = L × P_wr + L × P_st × T
    11. Break-even: N × (P_in - P_hit) > P_wr + P_st × T

    Plug in the sample parameters (P_in=4.5, P_hit=0.15, P_wr=4.5, P_st=0.017, T=1) and you get roughly 1.03 hits to break even. But that conclusion depends heavily on the scenario: if the cached content expires every 5 minutes while requests arrive every 10 minutes, then N=0, and the write and storage charges become pure expense. Caching is not a free speedup; it is an inventory investment that needs turnover to work.

    2.3 Discount stacking and mutual exclusivity: the easiest thing to miss

    Some platforms’ billing rules explicitly restrict how discounts combine, and the classic case is batch discounts and context cache discounts cannot apply at the same time. Constraints like this live in the billing notes rather than the price table, which makes them easy to overlook.

    When making an architecture decision, handle it in this order:

    1. Estimate both paths first: path A = “batch discount + no cache”, path B = “standard price + cache hit discount”.
    2. Plug in your real hit rate rather than assuming both paths get the best price.
    3. Send non-real-time pipelines (offline summarization, log analysis, bulk tagging) down the batch channel and trade latency for cost; keep interactive paths on caching to cut repeated prefill overhead.
    4. If some request class meets both conditions (a large reusable prefix and no real-time requirement), benchmark both paths with real traffic. Do not estimate from the theoretical optimum.

    2.4 Flat pricing and per-request context limits

    The fourth structure is the plainest: a single input and output price, plus a few constraints — for example, a per-request context length cap (on the order of 1M tokens), half price on batch, and a discount on context caching. Worth noting too: “thinking mode” and “non-thinking mode” are often priced differently. Some models publish a tier price only for non-thinking mode, while thinking mode burns extra reasoning tokens that are typically billed as output.

    This kind of model is the easiest to account for, but its hidden constraint costs are the highest: a context cap means long documents must be chunked, and the extra requests plus repeated prefixes created by chunking can cancel out the unit-price advantage.

    3. A unified cost model

    Normalize the four mechanisms above and you get an implementable cost function:

    1. total_cost =
    2. in_uncached_tokens × P_in
    3. + in_cached_tokens × P_hit
    4. + cache_write_tokens × P_wr
    5. + cached_tokens × P_st × hours_alive
    6. + out_tokens × P_out
    7. ) × period_multiplier × discount_factor

    Here period_multiplier is the time-of-day factor (off-peak 1.0, peak 2.0) and discount_factor is the discount factor (standard 1.0, batch 0.5~0.6), and the exclusivity constraint must hold: the cache discount and the batch discount cannot be multiplied together.

    A minimal implementation you can run directly:

    1. from dataclasses import dataclass
    2. @dataclass
    3. class PriceCard:
    4. p_in: float # input, cache miss
    5. p_hit: float # input, cache hit
    6. p_wr: float = 0.0 # cache write
    7. p_store: float = 0.0 # cache storage ($/1M tokens/hour)
    8. p_out: float = 0.0 # output
    9. peak_mult: float = 2.0
    10. def estimate(card, *, in_total, in_cached, out, write=0,
    11. cache_hours=0.0, is_peak=False, batch=False):
    12. """in_total / in_cached / out / write in tokens"""
    13. M = 1_000_000
    14. uncached = max(in_total - in_cached, 0)
    15. cost = (uncached * card.p_in
    16. + in_cached * card.p_hit
    17. + write * card.p_wr
    18. + out * card.p_out) / M
    19. cost += in_cached / M * card.p_store * cache_hours
    20. if is_peak:
    21. cost *= card.peak_mult
    22. if batch:
    23. cost *= 0.5 # batch discount; does not stack with cache discount
    24. return cost

    Wire this function into your load-test script and replay it against real traffic distribution instead of estimating from theoretical averages. That is the dividing line for whether a cost estimate can be trusted.

    4. Measure first, select second: a four-step decision process

    1. Collect a real traffic profile. Use 1-2 days of production logs to compute the input/output token ratio, the request-time distribution, and the prefix repetition rate. Without these three data points, any selection is a guess.
    2. Build the candidate set and unify pricing. Map each platform’s billing mechanics onto the same PriceCard structure, making sure time-of-day, cache, batch, and context limits are all modeled.
    3. Split cost by scenario. For interactive paths, cost under the P95 latency constraint; for offline paths, cost on the batch channel; for high-frequency repeated-prefix scenarios, the cache break-even point on its own.
    4. Leave structural headroom. Output token growth often outpaces input growth (more capable models produce longer outputs), so reserve 30%~50% in the budget, and run a stress test on the worst case of peak-hour surcharge.

    5. Make cost an observable metric

    The point of cost governance is not computing correctly once, it is being able to keep monitoring after computing. Four things need to land in engineering:

    • Usage instrumentation. Log input, output, and cache-hit token counts and the billing tier on every call, linked to a business ID and tenant ID, so you get drill-down cost detail.
    • Budget alerts. Set daily and monthly budget thresholds per project or environment and alert when approaching them rather than reconciling after the fact. General cloud capabilities such as a log service, monitoring and alerting, and object storage can carry this pipeline: ship usage data into object storage for offline accounting, then configure threshold rules on top of it.
    • Routing and degradation. Route requests of different difficulty to different model tiers, and when a budget or rate limit trips, degrade automatically to a lighter tier so availability comes first.
    • Cache lifecycle management. Enable caching for stable prefixes such as system prompts and long documents, and watch the hit rate. A persistently falling hit rate means the cache strategy no longer matches the traffic pattern, and you should switch it off to stop the bleeding.

    6. Common pitfalls

    • Ranking by unit price instead of computing total cost, and ignoring the input/output ratio weighting.
    • Assuming caches always hit, without checking TTL against request interval.
    • Treating cache write and storage charges as zero cost.
    • Estimating with batch and cache discounts both applied, which understates the budget badly.
    • Ignoring time-of-day pricing and extrapolating the monthly bill linearly at a flat rate.
    • Overlooking extra output tokens from thinking mode, and the “non-thinking mode only” price constraint.
    • Not accounting for the chunking overhead that a context cap creates.

    7. Summary

    Comparing costs across multiple model APIs is not a matter of “comparing prices”; it is a matter of modeling. The four billing mechanisms — time-of-day, tiered caching, mutually exclusive discounts, and flat pricing with context limits — stack into a bill structure that is highly nonlinear. The reliable approach is: collect a traffic profile first, then replay your bill through a single cost function, and land budget alerts and degradation strategies in engineering. Treat cost as a runtime metric you observe continuously rather than a static number computed once, and selection will not be led astray by the price sheet.