LLM API Pricing: Why Identical Token Rates Lead to Bills That Differ by 50%

    Most teams compare LLM providers by lining up pricing pages, sorting by input and output price, and picking the cheapest option — then discover that the same workload produces invoices differing by nearly half. This article shows why per-token rates are only one multiplier in the invoice formula. It breaks LLM API billing into four calculable variables: what providers actually meter (input, output, cache reads, cache writes), which rules create the gap when unit prices match (batch discounts, prefix caching, subscription plans with overage, regional and off-peak pricing), how to estimate monthly cost with a formula and a runnable Python billing estimator, and how to match a workload to the right billing model. Includes break-even math for subscription plans versus pay-as-you-go, a post-launch checklist, and the common misconceptions that lead to cost surprises.

    1. The symptom: same unit price, different invoice

    Most teams approaching an LLM API do the same thing: open the pricing pages of a few major providers side by side, sort them by input price plus output price, and pick whichever looks cheapest. After a couple of rounds of comparison, a pattern usually shows up — for the same token volume, the pay-as-you-go rates across providers are identical, or close enough. So price comparison feels finished. Pick any one of them.

    Then, at month end, the same workload and a similar call volume produce invoices that differ by nearly half. The easiest conclusion to reach is that one provider quietly raised prices, or that another has a hidden discount. In reality, the cause is usually not the unit price but this: the per-token rate is only one multiplier in the invoice formula. What really decides the bill is which pricing tier each token falls into, what discount applies, and whether the spend can be offset. Those variables are driven mainly by how you call the API, not by the provider alone.

    This article is not a benchmark or a recommendation of any specific platform. It splits the question into four parts you can calculate and verify:

    1. What LLM APIs actually meter.
    2. Which billing rules create a gap when per-token rates are the same.
    3. How to estimate monthly cost with a formula and a script.
    4. How to choose an approach for different workloads.

    2. Prerequisites: what you are actually billed for

    2.1 The unit is tokens, not characters

    LLM APIs are not billed by duration, by request count, or by character count. They are metered in tokens — the base units produced when the model’s tokenizer splits text. An English word may be a single token, a common Chinese character is usually one token, and code, punctuation, rare characters, and emoji may each be split into several. The same workload can therefore produce noticeably different token counts depending on how the prompt is written.

    Which means cost optimization cannot stop at “how much content.” You also have to look at how content is split, reused, and cached.

    2.2 Four metering categories must be read separately

    Break the invoice down and almost all spend falls into four categories:

    Billing item What it covers Typical characteristics
    Input tokens Everything sent to the model: system prompt, conversation history, retrieved documents, the user’s question Often underestimated; dominates long-context workloads
    Output tokens The text the model generates Unit price is usually several times the input price
    Cache hits A repeated prefix reused directly Clearly cheaper than normal input
    Cache writes Writing reusable content into the cache for the first time Billed separately; a one-time cost

    Two conclusions follow:

    • Output is more expensive than input. For most models the output price is several times the input price, so “make the model say less” usually saves more than “make the input shorter.”
    • Caching is not free. It only gets cheap when the repeated prefix is hit. On low-frequency workloads, cache write costs can push the bill up instead of down.

    3. Where the gap comes from when unit prices match

    Assume that for a given provider and call mode, the unit prices of input, cache read, cache write, and output are p_in, p_cr, p_cw, and p_out, all expressed in currency per million tokens. The cost of that portion is:

    C = (Q_in × p_in + Q_cr × p_cr + Q_cw × p_cw + Q_out × p_out) / 1,000,000

    Here Q is the token count for each billing item. In this formula, the provider decides the price p; your workload decides the volume Q. On a real platform, though, a few more rules wrap around the formula:

    C_month = Σ [ C_raw × k_discount ] − F_credit

    • k_discount: batch discounts, off-peak discounts, and similar multipliers.
    • F_credit: new-account free credits, limited-time promotions, and similar offsets.

    This is the root of “same unit price, different bill.”

    3.1 Variable one: batch — trade waiting time for a lower price

    Many providers offer a separate batch endpoint: requests enter a queue, run asynchronously, and the caller retrieves outputs via a query call or a result file. In exchange, the price is usually discounted off the standard rate. A common industry range is roughly half to sixty percent of the standard price, but the exact discount should be taken from each provider’s official pricing page.

    Batch is a good fit for:

    • Backlog data processing: document translation, contract review, comment classification;
    • Offline summarization, knowledge base construction, text preprocessing ahead of embedding;
    • Running evaluation sets and prompt A/B comparisons;
    • Low-priority recurring report generation.

    It is not a fit for real-time chat, online completion, or interactive agents — anything latency-sensitive. Batch jobs typically queue for anywhere from a few minutes to several hours, which cannot meet second-level response requirements.

    Whether batch saves money depends on how much a task tolerates delay, not on how much volume you have. Even converting only 60% of calls to batch, with batch priced at half the standard rate, drops the overall bill by roughly 30%: that 60% of calls costs half as much, which is 60% × 50% = 30% of total spend removed. Looked at in isolation, the spend moved to batch falls by 50%.

    The precondition is that those tasks genuinely do not need a real-time response. Pushing work that should run online into batch can save far less than the experience you lose.

    3.2 Variable two: caching — hits and writes are two separate accounts

    Prompt caching on mainstream platforms tends to follow a few shared rules:

    1. Prefix matching. The cache is keyed on the beginning of the request. From the first differing token onward, nothing matches, even if the rest is identical.
    2. A lifetime exists. The common approach is to expire after a period without a hit, which requires writing the content again on next use.
    3. Hits and writes are priced separately. The hit portion is billed well below normal input price; the write portion is usually charged separately and may even exceed the normal input price.
    4. Usually handled automatically by the provider. Many platforms enable caching for repeated prefixes without the caller declaring anything, but the default behavior must be confirmed on the official pricing page.

    Whether caching pays off can be quantified. To avoid ambiguity, define N as the total number of calls, not just the hits. The first call writes the cache, so hits are N - 1. For a stable prefix of length L million tokens reused N times within the cache lifetime:

    • Cost without caching: N × L × p_in
    • Cost with caching: L × p_cw + (N - 1) × L × p_cr
    • Savings ratio: 1 − (p_cw + (N - 1) × p_cr) / (N × p_in)

    If hits are tracked separately as H, then H = N - 1, and the cache cost can also be written as:

    Cache cost = L × p_cw + H × L × p_cr

    Three things follow:

    • The larger N is, the closer caching gets to its ceiling. With enough calls, the ceiling is approximately 1 − p_cr / p_in. If the hit price is about ten percent of the input price, the theoretical ceiling is around 90%.
    • At N = 1, savings depend entirely on the ratio of write price to input price. With no subsequent hits, the cache write becomes pure overhead. If the write price is higher than the input price, using the cache costs more.
    • Once the interval between calls exceeds the cache lifetime, the cache expires. The next call has to write again, so the effective N is far below the theoretical value.

    Three kinds of load are most sensitive to caching:

    • Fixed system prompts, tool definitions, few-shot examples;
    • Document chunks reused repeatedly in RAG;
    • Conversation history that keeps accumulating across turns yet overlaps heavily.

    The biggest precondition for caching gains is that the prefix is absolutely stable: do not put timestamps, random IDs, or live data at the start of a prompt. Otherwise a one-character change can invalidate everything after it.

    One more thing to keep in mind: batch jobs generally cannot count on cache hits, because requests sharing a prefix do not necessarily arrive back to back within the cache lifetime. Cache and batch discounts are hard to collect at the same time, so avoid planning as if the two can stack without limit.

    3.3 Variable three: subscriptions plus overage — commit spend to get a unit discount

    Some providers offer subscription or credit-package plans, such as token plans and inference bundles: you buy a package containing a certain allowance first, then either buy additional usage at the standard rate or buy incremental packages at a discount. Some plans also include off-peak benefits, where usage during quiet overnight hours consumes less allowance or earns an extra discount.

    The economic nature of these plans is trading committed spend for a lower unit price. That raises two key questions:

    • Buy too much and the allowance expires unused — you paid for capacity you never used.
    • Buy too little and the overage is still billed at the standard rate, which makes the discount meaningless.

    You must also confirm the plan’s scope: whether it covers cache reads and writes, whether it restricts which models you can use, and whether exhausting the allowance cuts service off or lets you continue pay-as-you-go. Subscriptions therefore suit stable, predictable baseline workloads; burst traffic should be covered by pay-as-you-go.

    3.4 Variable four: region and time of day — prices are not uniform

    • Regional differences. The same model can be priced differently across regions, because pricing strategy, resource cost, and settlement rules are not aligned. Exact prices should be taken from each provider’s official pricing page. Connecting to a nearby endpoint and keeping inference in your primary region is itself an optimization.
    • Off-peak differences. Some providers discount low-traffic hours or compute plan allowance differently by time of day.
    • Version differences. Newly launched models often come with free credits for new users or limited-time promotions. These allowances are usually one-off: they affect the short-term bill only and are not a basis for long-term cost planning.

    4. Quantifying it: from formula to script

    4.1 Before integrating: profile your usage first

    Before picking a provider, answer four questions:

    1. What are your monthly call count, average input length, and average output length?
    2. Is the prompt prefix stable, and how many hits could it theoretically produce?
    3. What share of calls can be delayed?
    4. What does the peak-to-trough distribution look like, and can part of the traffic shift into the trough?

    These four answers determine whether the four variables above can actually do any work.

    4.2 Turn the bill into a number with a script

    Below is a minimal working billing estimator. It reads call usage, computes cost per billing item, and separately totals online, batch, and off-peak calls.

    1. from dataclasses import dataclass
    2. @dataclass
    3. class Price:
    4. input_per_mtok: float
    5. cache_read_per_mtok: float
    6. cache_write_per_mtok: float
    7. output_per_mtok: float
    8. # The example only demonstrates structure; copy real prices from each provider's official pricing page
    9. PRICES = {
    10. "flagship": Price(20, 2, 100, 20),
    11. "balanced": Price(13, 1.3, 68, 13),
    12. "light": Price(6.5, 1.3, 27, 6.5),
    13. }
    14. RULES = {
    15. "online": 1.0, # Online calls: standard price
    16. "batch": 0.5, # Batch: half of the standard price, for illustration
    17. "offpeak": 0.5, # Off-peak discount: 50% off, for illustration
    18. }
    19. FREE_QUOTA = 0.0 # One-time free credit, in your billing currency
    20. @dataclass
    21. class Usage:
    22. model: str
    23. n_input: int
    24. n_cache_read: int = 0
    25. n_cache_write: int = 0
    26. n_output: int = 0
    27. mode: str = "online" # online / batch / offpeak
    28. def estimate(usages):
    29. bills = {}
    30. for u in usages:
    31. p = PRICES[u.model]
    32. raw = (
    33. u.n_input * p.input_per_mtok
    34. + u.n_cache_read * p.cache_read_per_mtok
    35. + u.n_cache_write * p.cache_write_per_mtok
    36. + u.n_output * p.output_per_mtok
    37. ) / 1_000_000
    38. bills[u.mode] = bills.get(u.mode, 0.0) + raw * RULES[u.mode]
    39. total = sum(bills.values()) - FREE_QUOTA
    40. return {k: round(v, 2) for k, v in bills.items()}, round(total, 2)
    41. if __name__ == "__main__":
    42. assert all(p.output_per_mtok > 0 for p in PRICES.values())
    43. assert all(p.cache_write_per_mtok > 0 for p in PRICES.values())
    44. usages = [
    45. Usage("flagship", 400_000, n_cache_write=20_000, n_output=20_000, mode="online"),
    46. Usage("balanced", 800_000, n_output=500_000, mode="batch"),
    47. Usage("light", 300_000, n_output=150_000, mode="offpeak"),
    48. ]
    49. print(estimate(usages))

    For the script to be trustworthy, you need real usage data at integration time, not estimates made by guesswork. Instrument your business code or a shared gateway to record the model, input/output/cache token counts, call mode, and timestamp of every call. With that data you can compare “assume everything at standard price” against “what the actual billing rules charged,” and quantify exactly how much each optimization saved.

    4.3 Decision: subscription or pay-as-you-go?

    Let monthly total usage be Q million tokens, the plan allowance be V million tokens, the plan price be F in your billing currency, and the equivalent pay-as-you-go unit price be p per million tokens. The comparison splits into two cases.

    Case one: Q ≤ V, usage stays within the allowance.

    The subscription costs F and pay-as-you-go costs Q × p. The subscription is cheaper when:

    F < Q × p

    which is:

    Q > F / p

    So when Q ≤ V and Q > F / p, the subscription wins. Conversely, if Q ≤ V and Q ≤ F / p, pay-as-you-go is cheaper.

    Case two: Q > V, usage exceeds the allowance.

    The subscription costs:

    F + (Q − V) × p

    and pay-as-you-go costs:

    Q × p

    The subscription is cheaper when:

    F + (Q − V) × p < Q × p

    which simplifies to:

    F < V × p

    So when Q > V and F < V × p, the subscription baseline wins. This assumes overage is priced at the same equivalent unit price p; actual overage rules depend on each provider’s documentation.

    These piecewise formulas explain the typical “bill differs by half” case: near the plan allowance, the two cost curves cross, and the gap is widest on either side of the intersection. Which leads to three practices:

    1. Use the median of the last 3 to 6 months of usage for Q, not the peak. Cover peaks with pay-as-you-go rather than overbuying a plan for a few spikes.
    2. Do not take p as the input price. Compute the equivalent unit price from your actual output share and cache hit rate:

    p = (Q_in × p_in + Q_cr × p_cr + Q_out × p_out) / Q_total

    1. p is not fixed. Better prompts, higher cache hit rates, and shorter outputs all push p down, and a “good deal” may stop being one. Re-run the numbers periodically as usage changes.

    5. Selection: from comparing unit prices to matching the billing model

    Putting the variables together, the decision splits into two independent questions:

    • Question one: which model? This mainly affects unit price and output quality.
    • Question two: which billing mode? This mainly affects discounts and the effective metered volume.

    Pseudo-code:

    1. if task tolerates delay:
    2. mode = batch
    3. elif prefix is stable and call interval is within cache lifetime:
    4. mode = online + cache hits
    5. elif usage is stable and above the break-even point:
    6. mode = subscription plan + pay-as-you-go for overage
    7. elif an off-peak discount is available and the task can be shifted:
    8. mode = online + off-peak scheduling
    9. else:
    10. mode = online pay-as-you-go
    11. In every case:
    12. keep the prefix stable, hit the cache, cap output length

    Typical workloads map as follows:

    • Online customer support and chat: real-time calls dominate; use a subscription for the baseline and pay-as-you-go for peaks, while capping maximum output length.
    • Document batch processing: prefer batch, where the price gap versus online calls is usually the most significant.
    • Knowledge base Q&A: system prompts and document chunks stay stable, so maximize cache hits.
    • Code completion: the most latency-sensitive case; batch and off-peak scheduling barely apply, so focus on compressing input context and completion length.
    • Coding tools and agents: long outputs that accumulate over multiple turns make caching valuable, but a single task’s runtime is hard to predict, so pay-as-you-go should be the default.
    • Evaluations and regression testing: naturally suited to batch and off-peak execution.

    6. Ongoing optimization: five checks after launch

    1. Metering must be observable. Record the billing item and token count for every call, and break cost down by day and by business line, so you do not discover anomalies at month end.
    2. Stabilize the prefix. Put the system prompt and tool definitions at the very front, push dynamic content such as timestamps and session IDs to the back, and use one shared template across all calls.
    3. Constrain output. Set a reasonable maximum generation length and ask for structured, concise answers instead of dumping the entire chain of reasoning.
    4. Tier your routing. Send simple tasks to light models and reserve flagship models for complex ones, allocating budget by task value.
    5. Recalculate periodically. When plans expire, model prices change, or the usage mix shifts, re-run the estimation script and the break-even formula.

    7. Common misconceptions

    • “Unit prices are the same, so it doesn’t matter which provider I pick.” When unit prices match, the billing rules and your call pattern decide the bill — not the numbers on the pricing page.
    • “Caching is free, so using it is always a win.” Cache writes are billed separately, and beyond the cache lifetime there are no hits, so low-repetition workloads can cost more.
    • “Batch is always cheaper.” Only deferrable tasks get the discount. Sending real-time requests into batch only hurts the experience.
    • “Buying a plan always saves money.” Below the break-even point, the plan costs more; and if overage is billed at the standard rate, much of the discount disappears.
    • “Free credits let me project long-term bills.” Free credits and limited-time offers usually affect the first month or the short term, and should not be used to forecast long-run cost.
    • “A price cut will automatically lower my cost.” Lower unit prices are often offset by more calls and longer contexts — without metering, the real bill stays invisible.

    8. Bottom line

    Back to the opening question: why can bills differ by half when unit prices are the same? Because the bill is not determined by unit price alone:

    Bill = unit price × effective metered volume × discount − credits

    Providers can be nearly identical on per-token rates while differing in batch discounts, caching mechanics, plan rules, and off-peak policies — and whether those rules help you depends entirely on how you call the API.

    For developers, the first step is not opening a pricing page but working out your usage: how much of it can be delayed, how much of the prefix can be reused, how much volume you can commit to. Handle those three, and whichever provider you integrate, cost stays inside a predictable, optimizable range.