LLM API Cost Breakdown: Why the Same Token Price Can Change Your Bill by Half

    Two platforms can quote the same price per million input and output tokens and still produce monthly bills that differ by nearly half. This article explains why: a bill is the product of unit price, billing rules, and how usage is distributed. It decomposes a baseline bill into non-cacheable input, cacheable input, and output, then quantifies prefix caching, splitting work between real-time and batch channels, subscription plans with overage rules, off-peak pricing, and routing easy tasks to lighter models. A runnable Python estimator and its output are included, along with a pre-signing vendor checklist and the per-call fields and metrics needed to keep costs attributable over time.

    Same Token Price Doesn’t Mean the Same Bill

    Many teams compare model service platforms on a single number: the price per million input and output tokens. If two or three platforms quote the same figure, the natural conclusion is “it doesn’t matter which one we pick.” Then the monthly bill arrives, and the same usage costs nearly twice as much on one platform as on another — with no clear explanation of where the extra money went.

    The cause isn’t a hidden price increase. It’s that the unit price is only one parameter in the billing formula. A bill is the combined result of billing rules, task mix, and how calls are made. This article compares no specific vendors. Instead, it breaks down what determines an LLM API bill, then provides a unified estimation model, a reusable script, and a task-tiering approach to cutting cost — so you can move from “comparing unit prices” to “adding up the total.”

    Same unit price, different bill

    The conclusion up front: the monthly bill depends not on the price sheet, but on unit price × rules × usage distribution.

    Layer that drives the bill Typical content Optimizable through engineering?
    Billing dimensions Input, output, cache hits, and cache writes priced separately Yes: compress context, keep prefixes stable
    Billing channel Real-time calls vs. batch/asynchronous calls Yes: route offline work away
    Commercial model Pure pay-as-you-go, prepaid subscription or credit pack, volume tiers Yes: switch once usage stabilizes
    Time-of-day pricing Flat standard price, or off-peak/peak split Yes: shift deferrable jobs
    Usage mix Input/output ratio, share of cacheable content, task difficulty distribution Yes: tiered routing
    Hidden costs Network and data transfer, operations, evaluation, migration Partly: co-locate the architecture, automate the workflow

    The first four layers decide “how much the same usage costs,” and they are the main source of bill gaps in practice: per-token pay-as-you-go prices can be identical across platforms while batch discounts, subscription rules, and time-of-day policies differ completely. The fifth and sixth layers decide “whether the same work can be done with fewer, cheaper tokens,” and their impact is often larger.

    First, Understand What the Baseline Bill Is Made Of

    Abstract conclusions need numbers. The figures below are fictional and illustrative only — they demonstrate the calculation method and do not represent any vendor’s actual pricing. Amounts are in CNY 10K (ten thousand CNY) units, token prices in CNY per million tokens.

    • Unit price: flagship model, input 20 CNY/M tokens, output 100 CNY/M tokens.
    • Monthly usage: 2 million requests, averaging 2,000 input tokens and 300 output tokens each — 4 billion input tokens and 600 million output tokens.
    • Totals: 4,000 million input tokens, 600 million output tokens.
    • Cache price: 10% of the standard input price by default.
    • Batch channel: 50% of standard price, and conservatively assumed not to benefit from caching.
    • Subscription plan: monthly fee of 2.4 (24,000 CNY), covering 3.0 (30,000 CNY) of standard-price usage; overage billed at standard price, overage multiplier of 1.

    At standard prices only, the baseline bill is:

    1. 4000 × 20 + 600 × 100 = 140000 CNY = 14.0 (CNY 10K)

    The effective unit price is:

    1. 140000 CNY ÷ 4600 M tokens ≈ 30.4 CNY per million tokens

    That bill splits into four parts:

    Component Standard-price cost (CNY 10K) Share of standard-price bill Notes
    Non-cacheable input 3.2 22.9% Current-turn input and retrieved content; savings come from trimming and compression
    Cacheable input 4.8 34.3% System instructions, examples, fixed knowledge; counted at standard price here, billed at the cache price on a hit
    Output 6.0 42.9% Highest unit price; savings come from constraining generated length
    Total 14.0 100% All lines at standard price

    This table exposes two facts that are easy to miss:

    • Input accounts for roughly 87% of total tokens, but only about 57.2% of the bill at standard prices. Optimizing input length alone has a low ceiling.
    • Output accounts for only about 13% of total tokens, yet 42.9% of the standard-price bill. Under this illustrative data, removing one output token typically saves more than removing one input token.

    Also note that average usage per request is only a starting point. In real workloads, long-tail requests can account for over half of input tokens — which is exactly where context compression and tiered routing pay off most.

    Caching: What a Stable Prefix Buys You

    Most model services support prefix caching. When a request arrives, the service compares tokens from the start of the prompt, finds the longest prefix matching earlier requests, and reuses it. The cache lifetime is usually counted from the last hit.

    Billing generally follows two rules:

    • The portion that must be written on a miss is billed separately, usually at a price higher than normal input.
    • The portion served from cache is billed at the cache price, typically a small fraction of the standard input price.

    So a hit is not automatically worth having. Whether it pays off depends on how long the prefix is and how long it stays stable. Three factors drive the hit rate:

    1. Prefix stability. System instructions, tool descriptions, few-shot examples, and fixed knowledge documents belong at the front. Timestamps, request IDs, retrieval results, and current-turn input are dynamic and belong at the back.
    2. How the conversation progresses. Each turn of a multi-turn chat appends new content. If the gap between turns exceeds the cache lifetime, the next turn may have to write the cache again. When history grows long, summarize and compress it — but weigh the input savings against the cost of rebuilding the cache.
    3. Traffic density. Under high concurrency the prefix is hit continuously and the lifetime keeps refreshing. During quiet hours, longer gaps appear, and the first requests after a cold start may be billed as misses.

    Under this section’s illustrative data, suppose each request carries a 1,200-token cacheable prefix — 60% of input — with a 90% hit rate. Raising the hit rate by 10 percentage points, from 90% to 100%, means:

    1. 2M requests × 1200 tokens × 10% = 240 M tokens

    Those 240 million tokens move from “cache write on a miss” to “cache hit.” At a write price of 25 CNY and a hit price of 2 CNY:

    1. (25 − 2) × 240 = 5520 CNY = 0.55 (CNY 10K)

    That is roughly 3.9% of the 14.0 standard-price bill. Measured against the 20 CNY standard input price instead, it would be about 0.43 (4,320 CNY). This article consistently uses the write-price basis.

    One common mistake is putting retrieval results at the very front “so the model reads the material first.” That pushes content which changes every turn into the prefix, and the system instructions and examples that follow can no longer hit the cache — often a net loss.

    Batch: Two Lanes for Real-Time and Offline Work

    Whether a call suits the batch channel comes down to three questions:

    • Is a user waiting for the result in real time?
    • Can the result tolerate minutes or even hours of delay?
    • Can the task be organized in batches and retried on failure?
    Task type Recommended channel Reason
    Online customer support, code completion, real-time search Q&A Real-time The user is waiting
    Document summarization and archiving, content tagging, bulk translation, offline report generation Batch Delay is acceptable, and a discount is usually available
    Data cleaning, embedding/vectorization, other re-runnable jobs Batch or off-peak Retries are fine, and timeliness doesn’t matter

    The batch channel usually does not support streaming output, and completion time depends on the platform’s queue. Anything involving user interaction or a hard deadline should stay out of the batch lane.

    Continuing with the illustrative data, suppose 40% of calls are identified as deferrable and moved to batch:

    1. Batch calls = 2M × 40% = 800K calls
    2. Real-time calls = 2M × 60% = 1.2M calls

    The batch portion has 800K × 2,000 = 1,600 M input tokens and 800K × 300 = 240 M output tokens. At standard prices:

    1. 1600 × 20 + 240 × 100 = 56000 CNY = 5.60 (CNY 10K)

    At 50% off:

    1. 5.60 × 50% = 2.80 (CNY 10K)

    The real-time portion has 1.2M × 2,000 = 2,400 M input tokens and 1.2M × 300 = 360 M output tokens. With the caching assumptions: a cacheable prefix of 60% of input, or 1,440 M tokens; a 90% hit rate, so 1,296 M tokens hit and 144 M cacheable tokens miss; and 960 M tokens of non-cacheable input.

    The real-time cost is:

    1. 1296 × 2 + 144 × 25 + 960 × 20 + 360 × 100
    2. = 2592 + 3600 + 19200 + 36000
    3. = 61392 CNY
    4. = 6.14 (CNY 10K)

    Combined:

    1. 6.14 + 2.80 = 8.94 (CNY 10K)

    Against the 14.0 baseline, that is a reduction of roughly:

    1. (14.0 − 8.94) ÷ 14.0 ≈ 36.1%

    Under this illustrative data, “caching + task routing” alone cuts more than a third off the bill.

    Subscriptions and Off-Peak Pricing: When They Pay Off

    Let S be the standard-price cost, F the monthly fee, Q the standard-price amount the plan covers, and m the overage multiplier. Then:

    1. Subscription cost = F + max(0, S − Q) × m
    2. Pay-as-you-go cost = S

    If you ignore plan capacity and only look at trading a fixed monthly fee for a discount, a rough saving estimate is:

    1. Simplified saving rate ≈ (S − F) ÷ S

    But real subscriptions include a plan capacity Q and overage rules, so the full formula must carry max(0, S − Q) × m.

    With no overage markup (m = 1) and no other add-on fees, whether the subscription wins depends on the relationship between S and F, Q:

    • If S ≤ Q, the subscription costs F, so it is only cheaper when S > F.
    • If S > Q, the subscription costs S − Q + F, and the saving is roughly Q − F.
    • Higher usage does not necessarily mean higher savings; the relative saving rate may actually fall.

    For example, with F = 2.4, Q = 3, and m = 1, if S = 6.0 the saving is about 0.6 (6,000 CNY), or roughly 10%. You cannot assume that doubling usage doubles the saving.

    If the platform marks up overage, the winning range narrows further. With F = 2.4, Q = 3, m = 1.5:

    1. Subscription cost = 2.4 + max(0, S − 3) × 1.5

    When S < 4.2 (42,000 CNY) — plan utilization below 140% — the subscription may cost less than pay-as-you-go. The maximum relative saving is about 20%, typically appearing when S is near 3.0 (30,000 CNY). Beyond 4.2, the overage markup can cancel out the subscription discount.

    Time-of-day discounts can be estimated as:

    1. Savings = Σ q(t) × p(t) × (1 − d(t))

    where q(t) is the call volume in a time window, p(t) the standard price, and d(t) the discount factor. Savings depend only on “volume actually migrated × discount depth,” not on total volume. The decision to shift load off-peak usually rests on just two things: whether the task is deferrable, and whether stable offline job scheduling already exists.

    A subscription is therefore not inherently cheaper. Its essence is trading certainty for a discount. While a workload is still being validated — usage could double or halve — prefer pay-as-you-go. Once it reaches steady state with limited fluctuation, re-evaluate the subscription using three months of rolling data. Other commercial forms such as tiered pricing and credit packs should likewise be converted into F and Q and fed into the same model for comparison.

    Model Tiering and Version Governance

    There is a second way to bring the same bill down: send simple tasks to a lighter model and reserve the flagship for complex ones. In the illustrative data, the light model costs 6.5 CNY input and 27 CNY output, roughly one third of the flagship. Assume 70% of calls can be shifted down, and to avoid stacking discounts this section’s tiering example does not also count caching gains:

    1. Light-model portion:
    2. (4000 × 70% × 6.5 + 600 × 70% × 27) ÷ 10000
    3. = (18200 + 11340) ÷ 10000
    4. = 2.95 (CNY 10K)
    5. Flagship portion:
    6. (4000 × 30% × 20 + 600 × 30% × 100) ÷ 10000
    7. = (24000 + 18000) ÷ 10000
    8. = 4.20 (CNY 10K)
    9. Total:
    10. 2.95 + 4.20 = 7.15 (CNY 10K)

    Against the 14.0 baseline, that is a reduction of roughly 48.9%. A real system could layer caching, batching, or off-peak scheduling on top and go lower still, but for verifiability this article computes the tiering plan separately in the script below.

    Recommendations for putting this into practice:

    • Choose models by task, not by price. Start with a fixed set of business evaluation samples, and measure each model’s pass rate and cost per call.
    • Compute cost per completed task: cost per task = cost per call ÷ pass rate. Only a model with a lower figure on this metric, and a quality gap within an acceptable range, deserves more traffic.
    • Keep a fallback path. When output fails format validation or a call fails, retry automatically with a stronger model.

    Models keep iterating, and it is normal in this industry for old versions to close to new users and eventually be retired. Bill risk comes not only from unit prices but from being forced to migrate. Prepare in advance:

    • Reference model names and versions through configuration so business code never hardcodes them.
    • On every version change, compare quality and cost on the fixed evaluation set, and roll out to a small traffic share as a canary.
    • Convert the people-hours and elapsed time for evaluation, prompt tuning, and pipeline changes into a cost, and include them in the “is the new version actually cheaper” decision.
    • Some platforms offer a free token allowance for new models. It lowers the barrier to trying them out, but it is not a long-term cost advantage; in estimates, treat the free quota as a one-time benefit and leave it out of the steady-state bill.

    A Runnable Estimation Script and Its Real Output

    The script below is illustrative. Amounts are in CNY 10K, unit prices in CNY per million tokens, and usage is the monthly total in millions of tokens. The core conversion is:

    1. tokens (in millions) × CNY per million tokens ÷ 10000 = CNY 10K

    Save it as estimate.py and run python estimate.py. The actual output follows.

    1. # Monthly LLM API bill estimator (illustrative)
    2. # Amounts: CNY 10K (ten thousand CNY)
    3. # Unit prices: CNY per million tokens
    4. # Usage: monthly totals, in millions of tokens
    5. # Conversion: tokens (millions) x CNY per million tokens / 10000 = CNY 10K
    6. from dataclasses import dataclass, replace
    7. @dataclass
    8. class Profile:
    9. requests: int = 200 # 10K requests per month
    10. input_per_req: int = 2000 # average input tokens per request
    11. output_per_req: int = 300 # average output tokens per request
    12. cacheable_ratio: float = 0.6
    13. hit_rate: float = 0.9
    14. batch_ratio: float = 0.4
    15. lite_ratio: float = 0.7
    16. @dataclass
    17. class Price:
    18. p_in: float = 20.0
    19. p_out: float = 100.0
    20. p_hit: float = 2.0
    21. p_write: float = 25.0
    22. lite_in: float = 6.5
    23. lite_out: float = 27.0
    24. batch_discount: float = 0.5
    25. plan_fee: float = 0.0
    26. plan_quota: float = 0.0
    27. over_mult: float = 1.0
    28. def cost_realtime(p: Profile, price: Price, use_cache: bool = True) -> float:
    29. # 10K requests x tokens / 100 = million tokens
    30. in_tokens = p.requests * p.input_per_req / 100
    31. out_tokens = p.requests * p.output_per_req / 100
    32. if not use_cache:
    33. return (in_tokens * price.p_in + out_tokens * price.p_out) / 10000
    34. cached = in_tokens * p.cacheable_ratio
    35. hit = cached * p.hit_rate
    36. miss = cached * (1.0 - p.hit_rate)
    37. return (
    38. hit * price.p_hit
    39. + miss * price.p_write
    40. + (in_tokens - cached) * price.p_in
    41. + out_tokens * price.p_out
    42. ) / 10000
    43. def scheme_baseline(p: Profile, price: Price) -> float:
    44. return cost_realtime(replace(p, batch_ratio=0.0), price, use_cache=False)
    45. def scheme_batch_cache(p: Profile, price: Price) -> float:
    46. if p.batch_ratio <= 0:
    47. return scheme_baseline(p, price)
    48. real = replace(p, requests=p.requests * (1.0 - p.batch_ratio), batch_ratio=0.0)
    49. batch = replace(p, requests=p.requests * p.batch_ratio, batch_ratio=0.0)
    50. realtime_cost = cost_realtime(real, price, use_cache=True)
    51. batch_cost = cost_realtime(batch, price, use_cache=False) * price.batch_discount
    52. return realtime_cost + batch_cost
    53. def scheme_subscription(p: Profile, price: Price) -> float:
    54. # Conservative assumption: subscription and batch do not stack;
    55. # the post-cache realtime cost is used as the subscription basis.
    56. standard = cost_realtime(replace(p, batch_ratio=0.0), price, use_cache=True)
    57. return price.plan_fee + max(0.0, standard - price.plan_quota) * price.over_mult
    58. def scheme_tiered(p: Profile, price: Price) -> float:
    59. # Computed separately to avoid stacking discounts with caching.
    60. in_tokens = p.requests * p.input_per_req / 100
    61. out_tokens = p.requests * p.output_per_req / 100
    62. lite = p.lite_ratio
    63. heavy = 1.0 - lite
    64. lite_cost = (
    65. in_tokens * lite * price.lite_in
    66. + out_tokens * lite * price.lite_out
    67. ) / 10000
    68. heavy_cost = (
    69. in_tokens * heavy * price.p_in
    70. + out_tokens * heavy * price.p_out
    71. ) / 10000
    72. return lite_cost + heavy_cost
    73. def compare(p: Profile, price: Price) -> None:
    74. baseline = scheme_baseline(p, price)
    75. plans = {
    76. "Pay-as-you-go": baseline,
    77. "Batch + cache": scheme_batch_cache(p, price),
    78. "Subscription+overage": scheme_subscription(p, price),
    79. "Model tiering": scheme_tiered(p, price),
    80. }
    81. total_tokens = p.requests * (p.input_per_req + p.output_per_req) / 100
    82. print(f"{'Plan':<20}{'Bill (CNY 10K)':>14}{'Effective (CNY/M tok)':>26}{'vs base':>10}")
    83. for name, cost in plans.items():
    84. eff = cost * 10000 / total_tokens
    85. rel = cost / baseline - 1.0
    86. print(f"{name:<20}{cost:>14.2f}{eff:>26.2f}{rel:>9.2%}")
    87. if __name__ == "__main__":
    88. profile = Profile()
    89. price = Price(plan_fee=2.4, plan_quota=3.0, over_mult=1.0)
    90. compare(profile, price)

    Actual output:

    1. Plan Bill (CNY 10K) Effective (CNY/M tok) vs base
    2. Pay-as-you-go 14.00 30.43 0.00%
    3. Batch + cache 8.94 19.43 -36.15%
    4. Subscription+overage 9.63 20.94 -31.20%
    5. Model tiering 7.15 15.55 -48.90%

    Under this illustrative data, the biggest win does not come from the lowest-priced option, but from the combination of “caching + task routing + model tiering.” A subscription only suits workloads with stable usage and utilization at or near the plan capacity.

    Selection Checklist: What to Ask Before Signing

    The same unit price does not mean the same rules. When evaluating a platform, confirm each item:

    1. Are all billing items listed? Are input, output, cache hits, and cache writes priced separately? How long is the cache lifetime, and can it be adjusted?
    2. Batch API: What discounts apply? What is the maximum time to return results? Does it support caching, and are input/output sizes capped?
    3. Subscription rules: How is the quota measured? Are input and output converted separately? Can unused quota roll over to the next month? How is overage priced, and is there a cap?
    4. Time-of-day policy: Are peak and off-peak distinguished? Are the discount rate and applicable windows stated explicitly?
    5. Models and versions: Are models of different parameter sizes offered? What is the release cadence, and how are old versions retired?
    6. Usage accounting: How are billable tokens counted? How are over-long contexts handled?
    7. Data and compliance: Is data used for model training? Where is it stored, and for how long is it retained?
    8. Change process: How much notice is given before prices or rules change?

    Two more practical points:

    • Any platform’s pricing page can be out of date or incomplete. A missing line item does not mean that item is free — confirm directly with the platform or consult the formal contract.
    • The same model name may point to different version snapshots on different platforms, so behavior may differ. Before switching, run a small-traffic A/B test and confirm quality does not drop.

    Continuous Cost Reduction: Make the Bill Observable

    A single optimization pass is only the starting point. Turning cost reduction into a lasting mechanism requires bills that can be attributed and alerted on.

    Log the following fields after each call:

    1. biz_line business line / department, for cost allocation
    2. feature feature point, to locate high-consumption scenarios
    3. channel realtime / batch
    4. model model name
    5. input_tokens input tokens
    6. cached_tokens tokens served from cache
    7. output_tokens output tokens
    8. latency_ms latency

    Track these metrics:

    • Effective unit price: monthly bill divided by total tokens, as a common basis for comparing across months and plans.
    • Cache hit rate: when it falls below expectations, check whether dynamic content is polluting the prefix.
    • Output tokens per call: a sudden jump usually means output constraints were loosened.
    • Share of batch traffic: reflects how much offline work has been routed away.
    • Subscription utilization: persistently low values call for re-evaluating pay-as-you-go versus subscription.
    • Cost per completed task: read together with quality metrics so that cost cutting does not damage results.

    Set up two lines of defense as well:

    • Define budget thresholds per business line and feature point, with monitoring alerts triggered when they are crossed.
    • Cap output length and retry count per call, so abnormal traffic cannot send the bill out of control.

    Conclusion

    “Same unit price, bill differs by half” is not a contradiction, because the unit price is only one input to the bill. What actually determines spending is unit price × rules × usage distribution:

    • Compress input cost through caching and prefix discipline;
    • Use the batch channel by routing offline work away;
    • Lock in a discount with a subscription once usage is stable;
    • Cut unnecessary expensive calls through model tiering.

    Understand your workload’s usage first, then choose a billing model. Price comparison only sets the starting point; the combination of billing rules and calling patterns is what determines the final bill.