LLM API Pricing Comparison: Unit Prices Match, So Why Do Bills Differ by Half?

    For mainstream large language model (LLM) APIs, per-token list prices have largely converged, yet the monthly bill for identical usage can still differ by nearly half depending on the platform and how you call it. This article explains why: the per-million-token price is only the starting point, while a handful of rules built around it — caching, asynchronous batch processing, subscriptions and resource packages, off-peak scheduling, and regional pricing — determine the real cost. It breaks down token-based billing into its components, quantifies how each rule affects a monthly bill through a reproducible worked example, and lays out a task-based selection and optimization methodology along with a cost-governance checklist for keeping token spend observable over time. All prices used here are hypothetical demonstration values.

    1. Introduction: Unit Prices Match, So Why Do Bills Differ by Half?

    Over the past two years, the pay-as-you-go unit prices of general-purpose LLM APIs have steadily fallen, and list prices for mainstream models across platforms have clearly converged. For workloads with stable usage, price comparison often reaches “no conclusion”: list prices are nearly identical, so the decision collapses into integration convenience and compliance requirements.

    Yet at month-end reconciliation, many teams run into a different situation: with the same call volume, switching platforms or changing the calling method can produce bills that differ substantially — by as little as ten percent, or by nearly half.

    This is not a contradiction. The per-million-token price is only the starting point of billing. What actually determines the bill is a set of rules designed around that price:

    • Caching: repeated prefixes are billed at a lower price;
    • Asynchronous batch processing: non-real-time tasks get a discount;
    • Subscriptions or resource packages: committed usage in exchange for a lower effective unit price, usually paired with off-peak scheduling;
    • Region: quoted prices are not the same across regions.

    These rules usually appear on separate pages within the pricing documentation. If you look only at the main price table during selection, or treat “the same unit price” as “the same cost” during evaluation, cost deviation becomes almost unavoidable.

    This article addresses one general-purpose LLM API integration problem:

    1. Starting from the billing model, it breaks down the components of a bill;
    2. It quantifies how the rules above affect the monthly bill;
    3. It provides selection and optimization methods matched to task types.

    All prices below are hypothetical values for demonstration. For a specific platform, its actual unit prices, caching policy, discount rates, and effective time windows are governed by the latest information on that platform’s official pricing page.

    2. Breaking Down the Bill: The Cost Structure of Token Billing

    2.1 Three-Tier Pricing and the “Pseudo Unit Price”

    The billing unit for mainstream LLM APIs is dollars per million tokens, but a single request is usually split into three parts for billing:

    • Input (non-cache-hit portion): billed at the input price;
    • Cache hit: when the model reuses a prefix it previously computed and cached, this portion is billed only at the hit price, which is typically far below the input price;
    • Output: billed at the output price, and usually the most expensive of the three. In common pricing, the output unit price is roughly 4–6× the input price.

    Let the input tokens of a single request be N_in, of which the cache-hit portion is N_hit (N_hit = 0 means no hit), and the output tokens be N_out. The cost of a single request is:

    1. cost_per_call = [(N_in - N_hit) × P_in + N_hit × P_hit + N_out × P_out] / 1,000,000
    2. N_in input tokens
    3. N_hit cache-hit tokens
    4. P_in input price ($ per 1M tokens)
    5. P_hit cache-hit price ($ per 1M tokens)
    6. P_out output price ($ per 1M tokens)

    This shows that the “input price” on the main price table is only a ceiling: when the prefix is stable and the hit rate is high, the actual average input cost may be only a fraction of the input price. In scenarios with a high output share, cost is instead dominated by the output price.

    2.2 Cache Writes: An Easily Overlooked Hidden Cost

    Most platforms’ caching mechanisms automatically detect a repeated common prefix in requests (such as the system prompt, tool descriptions, and fixed few-shot examples) and reuse the computed result. The hit portion is billed only at the hit price.

    But some platforms bill cache writes separately: tokens written to the cache for the first time are settled at a certain price, and subsequent hits within the validity window are billed at the hit price. When writes are billed separately, whether caching pays off depends on the comparison between “write cost plus the hit price” and “billing everything at the input price.” Let the cache-write unit price be P_write, the hit price be P_hit, the input price be P_in, the prefix length be L, and the number of accesses within the validity window be k. Then:

    1. hit_savings = L × (P_in − P_hit)
    2. write_cost = L × P_write
    3. Across k accesses there are at least (k−1) hits, so caching only starts producing a net gain when (k−1) × (P_in − P_hit) > P_write.

    Therefore, “long sessions, high frequency, stable prefix” are the ideal conditions for caching. Scenarios where every request carries changing content, and a hit can only happen again after a long gap, not only fail to benefit but may even raise costs because of write charges.

    Note that platforms differ greatly on whether caching is enabled automatically, whether writes are billed, how long the default validity window is, and whether prices differ by validity window. Verify each item before integration; do not carry over experience from another platform.

    2.3 A Unified Cost Model

    Writing the variables above as a parameterized function lets you quickly estimate the monthly bill across different task shapes. In the sample code, the unit prices, batch discount, and off-peak discount are all hypothetical values used to demonstrate the direction of each factor’s impact:

    1. from dataclasses import dataclass
    2. MILLION = 1_000_000
    3. @dataclass(frozen=True)
    4. class PriceBook:
    5. """Unit: $ per million tokens"""
    6. input_price: float = 20.0 # input (cache miss)
    7. cache_hit_price: float = 2.0 # cache hit
    8. cache_write_price: float = 0.0 # cache write; 0 when writes are not billed separately
    9. output_price: float = 100.0 # output
    10. batch_factor: float = 1.0 # async batch factor; 1.0 means no discount
    11. offpeak_factor: float = 1.0 # off-peak factor; 1.0 means no discount
    12. def effective_input_price(self, prefix_len: int, reuse_times: int) -> float:
    13. """Return the average price per input token; reuse_times is how many times the prefix is accessed within its validity window.
    14. When cache writes are billed separately, each access amortizes one write cost.
    15. """
    16. price = self.input_price
    17. if self.cache_write_price > 0:
    18. # When some first accesses are billed at the input price and only later accesses write to cache, fold the write term into the hit price
    19. price += self.cache_write_price / reuse_times
    20. if reuse_times > 1:
    21. price -= (price - self.cache_hit_price) * (reuse_times - 1) / reuse_times
    22. return price
    23. def cost(self, input_tokens: float, output_tokens: float, *,
    24. batch_ratio: float = 0.0, offpeak_ratio: float = 0.0,
    25. prefix_len: int = 0, reuse_times: int = 1) -> float:
    26. """Estimate cost ($) for the given usage and task mix.
    27. batch_ratio / offpeak_ratio: share of batch / off-peak requests, counted over all requests.
    28. Batch requests don't consume the realtime quota and are scheduled first; the off-peak-eligible share of the rest is then allocated by offpeak_ratio.
    29. Relationship: realtime + offpeak = 1 - batch_ratio.
    30. """
    31. if not 0 <= batch_ratio <= 1 or not 0 <= offpeak_ratio <= 1 - batch_ratio:
    32. raise ValueError("batch_ratio + offpeak_ratio must not exceed 1")
    33. eff_in = self.effective_input_price(prefix_len, reuse_times)
    34. per_call = eff_in * input_tokens + self.output_price * output_tokens
    35. realtime_ratio = 1 - batch_ratio - offpeak_ratio
    36. return per_call / MILLION * (realtime_ratio
    37. + batch_ratio * self.batch_factor
    38. + offpeak_ratio * self.offpeak_factor)

    Using “input $20 / hit $2 / output $100” as an example:

    • With no separate write billing and a prefix that lives 5 times within its validity window, the average input price is about $3.2 per million tokens, only 16% of the input price;
    • With separate write billing, where the write price equals the input price, the average price is about $5.2;
    • When a large share of prefixes live only once — i.e., a 0% hit rate — the average price is still $20.

    The same system prompt can bring cost savings only depending on how many times it is actually reused within its validity window.

    3. The Four Rules That Really Separate Bills

    Extending the calculation model from section 2.3 slightly, do a one-factor analysis: hold monthly usage constant and change only one parameter at a time to observe how the monthly bill changes. This makes it clear where the bill differences actually come from.

    Rule How it works Scenarios that benefit most Risks and prerequisites
    Pay-as-you-go Input, cache hit, and output billed in segments All scenarios Comparing only list prices ignores rule differences
    Caching Repeated prefixes billed at the hit price High-frequency calls with stable prefixes When writes are billed separately, short-lived content can actually increase cost
    Asynchronous batch Non-real-time tasks discounted off the standard price Offline tasks such as batch archiving and nightly data processing Not suitable for user-facing real-time requests
    Subscription or resource package Committed usage for a lower unit price, often paired with off-peak windows Steady always-on workloads with predictable usage Requires accurate usage forecasting; watch overage and validity
    Region Prices differ across regions Nearby access Cross-border access involves compliance and extra network costs
    Free quota Measured in tokens, covers limited usage Validation and trial phases The price after the quota is exhausted is the long-term cost

    Rule 1: Caching Is Fundamentally an “Application Design Problem”

    Caching saves on repeatedly computed input, so there are three things the application side can control:

    • Make the prefix stable: the system prompt, tool and function definitions, and fixed examples should sit at the very front of the request, with content changed as little as possible. As conversation turns increase, only append rather than rewrite history; push dynamic information (timestamps, request IDs, user info) as far back as possible, or remove it from the prefix.
    • Converge prefix variation: when a single request mixes in a lot of one-off content (such as retrieval snippets that change per request), split conversational requests and retrieval-augmented requests into different call shapes, so the stable part always stays in the same prefix.
    • Measure real gains: the hit rate should be counted by token count, not by request count, and compared before and after optimization using the same metric.

    On platforms where caching takes effect automatically and writes are not billed separately, this kind of optimization involves no architecture changes, yet it is often the most certain and fastest-acting item.

    Rule 2: Batch Processing Depends on Whether the Task Can “Wait”

    Asynchronous batch processing targets tasks that don’t need an immediate return, such as offline data labeling, content preprocessing, log summarization, and scheduled report generation. Its cost: results are not guaranteed to return at a particular time, and tasks usually need to be uploaded to a batch queue first, then polled or received via callback.

    So the criterion is straightforward: is the user waiting on the screen for this result?

    A practical approach is to split two call paths — “interactive” and “batch” — from the very start of system design, and maintain different timeout and retry policies for each. If you only discover at month-end that a large volume of “could-have-been-async” requests all went through the real-time path, reworking them usually means changing caller logic and interaction flows.

    Rule 3: Subscriptions and Off-Peak Trade Predictability for Unit Price

    Resource packages and subscription plans work on similar logic: the more predictable the usage, the more willing the platform is to offer a lower effective unit price. Their advantages usually show in two places:

    • Price certainty: monthly cost can be locked in advance, easing financial budgeting and capacity planning;
    • Supporting scheduling rules: a common form is a low-price window over a fixed period, such as overnight into early morning. Regular requests are billed at the standard price, while the portion falling within the low-price window is automatically discounted.

    Subscriptions also have clear costs:

    • Unused committed quota is usually non-refundable (subject to specific rules);
    • Overage typically reverts to pay-as-you-go pricing;
    • Quota often has a validity period, after which its value drops to zero.

    So subscriptions can beat pay-as-you-go only when “long-term monthly usage steadily covers the quota.” See section 4 for how to judge this.

    Rule 4: Region, an Easily Overlooked Hidden Price Gap

    For the same model, unit prices at access points in different regions may differ. Common patterns:

    • Prices are lowest in the primary region;
    • Neighboring or secondary regions carry a price premium;
    • Distant or cross-region access also brings latency and egress bandwidth costs.

    Also note that even when prices across providers are identical in your primary region, it doesn’t mean you can freely access across regions. Beyond network latency and stability, you must consider compliance requirements related to cross-border data. Under equal compliance conditions, deploying the workload and the model service in the same region is usually also the lowest total-cost choice.

    Also: Free Quotas and Model Lifecycle

    Many platforms offer free token quotas or limited-time discounts when a new model launches or during campaigns. These noticeably affect trial-phase bills but don’t change the long-term unit price. When doing capacity planning, use the price after the quota is exhausted as the baseline.

    In addition, model versions have lifecycles. Old versions may stop being served at a certain point, and the vendor usually recommends migrating to a new version, which often comes with a unit-price change. So selection should also factor in “replaceability”: keeping a lightweight model-adapter layer in the architecture, with model identifiers and default parameter values centralized in configuration, can reduce rework costs when versions change.

    4. A Reproducible Worked Example: The Same Usage, How Much Do Bills Differ?

    Below is a complete example. All prices and ratios are hypothetical values, used to show the magnitude of impact when the rules stack up.

    Example setup

    • Monthly call volume: 300M input tokens, 20M output tokens;
    • Model prices (hypothetical): input $20, cache hit $2, output $100 (per million tokens);
    • Task structure: user-facing real-time chat accounts for 60%, regular requests that can be shifted off-peak to the night window account for 20%, and offline tasks that can be batch-submitted account for 20%;
    • Comparison plans:
      • Plan A: pure pay-as-you-go;
      • Plan B: pay-as-you-go, but make the prefix a stable prefix so 60% of input tokens hit the cache;
      • Plan C: pay-as-you-go, and route the off-peak-eligible and batchable requests to their respective windows;
      • Plan D: on top of B, adopt “subscription quota + pay-as-you-go overage”;
      • Plan E: on top of D, migrate all calls to an overseas-region access point (assume prices there are 1.1× the primary region).

    Result

    1. if __name__ == "__main__":
    2. M = MILLION
    3. input_tokens, output_tokens = 300 * M, 20 * M
    4. on_demand = PriceBook() # Plan A
    5. cached = PriceBook() # Plan B/C
    6. subscribed = PriceBook(input_price=16, cache_hit_price=1.6,
    7. output_price=80,
    8. offpeak_factor=0.6) # Plan D/E
    9. overseas = PriceBook(input_price=22, cache_hit_price=2.2,
    10. output_price=110) # Plan E
    11. plans = {
    12. "A On-demand": dict(input_tokens=input_tokens, output_tokens=output_tokens),
    13. "B Stable prefix + cache": dict(input_tokens=input_tokens, output_tokens=output_tokens,
    14. prefix_len=4000, reuse_times=3),
    15. "C Batch + off-peak": dict(input_tokens=input_tokens, output_tokens=output_tokens,
    16. prefix_len=4000, reuse_times=3,
    17. batch_ratio=0.2, offpeak_ratio=0.2),
    18. "D Subscription + overage": dict(input_tokens=input_tokens, output_tokens=output_tokens,
    19. prefix_len=4000, reuse_times=3,
    20. batch_ratio=0.2, offpeak_ratio=0.2),
    21. "E D + overseas region": dict(input_tokens=input_tokens, output_tokens=output_tokens,
    22. prefix_len=4000, reuse_times=3,
    23. batch_ratio=0.2, offpeak_ratio=0.2),
    24. }
    25. books = [on_demand, cached, cached, subscribed, overseas]
    26. for (name, kwargs), book in zip(plans.items(), books):
    27. print(f"{name:<32}{book.cost(**kwargs):>10,.2f} USD")

    Output:

    1. A On-demand 5,800.00 USD
    2. B Stable prefix + cache 4,014.85 USD
    3. C Batch + off-peak 3,616.36 USD
    4. D Subscription + overage 2,898.07 USD
    5. E D + overseas region 3,187.88 USD

    Interpreting the results

    1. Caching: merely making the prefix stable lets 60% of input tokens enjoy the hit price, cutting the monthly bill by about 30.8%, with no platform change involved.
    2. Scheduling: on top of the caching optimization, moving 20% of requests to batch and shifting 20% off-peak to the night window brings another ~9.9% drop.
    3. Combined: caching and scheduling together push the monthly bill down to about 62% of the pay-as-you-go baseline.
    4. Subscription: with the B and C optimizations already in effect, switching to subscription quota at an 80% unit price drops the monthly bill further to about $2,900 — about half the pay-as-you-go baseline. But this result depends heavily on the usage assumption: if monthly usage were only about half of the plan, the committed quota would go unused, and the actual outcome could be worse than pay-as-you-go.
    5. Region: switching all calls to an overseas access point (assuming a 10% price premium) raises the monthly bill by about 10%.

    This shows that a “bill difference of half” is the combined result of several rules working together, not the difference of a single price.

    Note that these ratios come from hypothetical parameters, and the real magnitude depends on the model’s unit-price structure, the cache hit rate, and scheduling capability. But the direction of the conclusion is stable: caching and scheduling are application-side changes with low cost and fast results; subscriptions and region are contract- and architecture-side decisions that need quantified usage forecasts and compliance assessments to support.

    5. Selection Method: Split by Task First, Then Choose a Platform

    A practical conclusion follows directly from the billing model:

    • When the input share is high, the cache hit rate is low, the output share is high, and you can neither go async nor go off-peak, the pay-as-you-go unit price is almost the actual cost. This is when price comparison matters; and since platform prices have converged, use integration convenience, compliance requirements, and network quality as the main basis.
    • Conversely, for any task with a stable prefix, batchability, off-peak flexibility, or predictable usage, what really determines cost is the rules, not the list price. In this case, prioritize platforms that offer caching, batch, subscription, or off-peak capabilities, and use the model from section 2 to quantify the estimate.

    Accordingly, split workloads by task type:

    Task type Typical characteristics Recommended integration Key actions
    Interactive chat Users wait in real time, multi-turn appends, stable prefix Pay-as-you-go + cache optimization Stabilize the system prefix, push dynamic content back, count hit rate by actual tokens
    Delay-tolerant requests Nightly batch, scheduled jobs, results needn’t return immediately Asynchronous batch (if the platform supports it) Design submission and retrieval flows ahead of time, avoid mixing with the real-time path
    Steady always-on workloads Predictable monthly volume, daytime peak, nighttime trough Subscription quota + pay-as-you-go overage + off-peak Forecast usage from historical data, leave enough buffer, set quota alerts
    Low-frequency trial or exploration Unstable usage, model still under evaluation Pay-as-you-go + free quota Don’t treat trial prices as the long-term unit price
    Workloads already deployed in the cloud Same region as cloud data and services Choose a model service in the same region nearby Compare total cost (including egress traffic and cross-region latency)

    For workloads already deployed in a cloud environment, one more thing is often overlooked: accessing the model service in the same region as the workload not only avoids the added cost of cross-region calls but also reduces public egress traffic. Model responses need to be sent back to the application server, and the traffic cost of long-text output is not trivial. This kind of fee doesn’t show up on the token bill, but it enters the overall cloud bill.

    Note that no matter which cloud provider’s model service you choose, the specific form of rules like caching, batch, and off-peak may differ. It’s advisable to follow the latest information on each platform’s official pricing page, and during selection to run a small-scale real test with the same real request sample (covering a typical distribution of input and output lengths), using measured data to calibrate the estimation model.

    6. Cost Governance: Bring the Bill into an Observable System

    Billing rules change, model versions update, and workload usage fluctuates. To control cost over the long term, treat token consumption as a routine monitoring metric.

    6.1 Metrics

    • Record input, cache-hit, and output tokens by day, by model, and by business line, retaining call counts and failure counts;
    • Record the cache-hit portion separately, otherwise you can’t judge the effect of prefix optimization;
    • Distinguish among the three calling modes — realtime, off-peak, and batch — and track each one’s share.

    6.2 Budgets and Alerts

    • Set a monthly budget and alert at key thresholds such as 80% and 100%;
    • Set remaining-quota alerts for subscriptions to avoid a bill spike when overage reverts to pay-as-you-go;
    • Cap the maximum input length per request to prevent abnormal long contexts or looped calls from inflating consumption.

    6.3 Periodic Review

    • After each price or rule change (especially changes to how cache writes are billed), re-run the estimation model;
    • Periodically check model lifecycle announcements and plan version migrations in advance.

    Below is a simplified usage-statistics and cost-accounting example showing how to turn call records into comparable cost numbers:

    1. from collections import defaultdict
    2. def summarize(records, price: PriceBook, usage_plan=None):
    3. """records: [(model, mode, input_tokens, cache_hit_tokens, output_tokens), ...]
    4. mode: "realtime" | "offpeak" | "batch"
    5. usage_plan: {model: subscription rate per million tokens}; None means the model has no subscription
    6. """
    7. stat = defaultdict(lambda: dict(calls=0, input=0, hit=0, output=0, cost=0.0))
    8. for model, mode, n_in, n_hit, n_out in records:
    9. p = dict(price)
    10. if mode == "batch":
    11. p["batch_factor"] = price.batch_factor
    12. elif mode == "offpeak":
    13. p["offpeak_factor"] = price.offpeak_factor
    14. pb = PriceBook(**p)
    15. s = stat[model]
    16. s["calls"] += 1
    17. s["input"] += n_in
    18. s["hit"] += n_hit
    19. s["output"] += n_out
    20. s["cost"] += pb.cost(n_in - n_hit, n_out) + pb.cache_hit_price * n_hit / MILLION
    21. report = {}
    22. for model, s in stat.items():
    23. hit_rate = s["hit"] / s["input"] if s["input"] else 0.0
    24. plan_price = (usage_plan or {}).get(model)
    25. if plan_price is not None:
    26. # Subscription is amortized over effective tokens (input minus hits); follow the platform's actual billing rules
    27. effective_tokens = s["input"] - s["hit"]
    28. s["cost"] = max(s["cost"], plan_price * effective_tokens / MILLION)
    29. report[model] = {
    30. "calls": s["calls"],
    31. "cache_hit_rate(by token)": f"{hit_rate:.1%}",
    32. "effective_token_cost($/1M)": round(s["cost"] / max(s["input"], 1) * MILLION, 2),
    33. "cost($)": round(s["cost"], 2),
    34. }
    35. return report

    The core idea here is to use a unified “effective token cost” metric to bring spending across different task structures back onto the same scale, making it easier to compare across months and platforms.

    7. Practice Checklist

    1. Look at task structure first, then price: input share, cache hit rate, output share, whether it can go async, whether it can go off-peak, and whether usage is predictable — these six factors determine actual cost.
    2. Treat caching as an engineering problem: stabilize the prefix, push dynamic content back, converge retrieval snippets, count hit rate by token, and on platforms where writes are billed separately, calculate whether there’s a net gain over the content’s lifetime.
    3. Go async whenever possible: design the batch path separately, decouple submission and retrieval logic, and don’t mix it with real-time requests.
    4. Off-peak requires “tasks can queue” as a prerequisite: without queued background tasks, a low-price window can’t be used either.
    5. Let data speak for subscriptions: forecast coverage from historical usage, and watch validity periods, overage prices, and refund rules.
    6. Evaluate region holistically: compare unit price, latency, compliance, and egress traffic cost, and prefer same-region deployment of workload and model.
    7. Establish a budget and alerting mechanism: token statistics, remaining quota, and abnormal long-input limits all need to be part of daily monitoring.
    8. Watch out for free quotas and temporary discounts: base long-term cost on the unit price after the quota is exhausted.

    Conclusion

    Price competition in LLM APIs has already flattened the “unit price” dimension, but bills haven’t become any simpler as a result. Under the same usage, caching rules, batch discounts, subscriptions and off-peak, and regional differences jointly determine the final spend.

    For developers, the direction worth investing in isn’t repeated price comparison, but:

    • Designing the application to be cache-friendly, async-friendly, and scheduling-friendly;
    • Using a unified billing model to quantify each rule’s impact during selection and capacity planning;
    • Continuously validating assumptions with observable usage and cost data.

    Do these, and even when platforms’ pay-as-you-go unit prices are completely identical, you can still land a noticeably lower monthly bill for the same business needs.