LLM API Cost Comparison: Why Identical Unit Prices Can Still Produce Half-Different Bills

    Identical sticker prices across model and cloud providers do not mean identical bills. This article breaks the LLM API billing model into a computable formula and shows how four rules—batch channels, subscription and off-peak scheduling, prompt caching, and regional premiums—drive the real difference. You will learn to estimate your own bill, compare platforms by their rules rather than their price columns, and reduce cost through engineering choices such as routing async traffic, optimizing cache prefixes, and scheduling workloads off-peak.

    LLM API Cost Comparison: Why Identical Unit Prices Can Still Produce Half-Different Bills

    1. A Counterintuitive Observation

    Imagine you are on a team embedding large-model inference into a product. As your user base triples, you start comparing prices. You open the developer platforms of a few cloud vendors and model vendors, and you notice the flagship models have exactly the same sticker price—the same CNY 20 per million input tokens, CNY 2 for cache hits, and CNY 100 per million output tokens. Mid-tier models are equally aligned at 13 / 1.3 / 68. Looking at the price page alone, the conclusion seems to be “any vendor will do.”

    Yet when the same call volume runs on different platforms, the monthly bill can differ by a factor of two. This is not a data error or marketing spin. It is simply that the per-unit price is only one multiplier in the billing formula. What actually separates the bills are four rules—cache hits, batch channels, subscriptions/resource packages, and regional premiums—plus the way you schedule your traffic.

    The goal of this article is not to compare one specific model. It is to abstract LLM API billing into a computable formula, so you can estimate “what my bill will look like” on any platform yourself, and then use engineering methods to bring it down.

    2. Breaking the Bill into Five Variables

    A rough monthly bill can be written as:

    1. Monthly cost = input volume × input price × (1 - cache hit rate) × cache penalty factor
    2. + input volume × cache hit rate × cache-hit price
    3. + output volume × output price
    4. - batch discount
    5. - subscription/resource-package credit
    6. - free tier

    Of these, three variables are genuinely affected by engineering effort: cache hit rate, async call ratio, and call-time distribution. The remaining two—unit price and discount rules—are set by the platform, so your job there is simply to pick the right platform.

    A typical input/output asymmetry looks like this: input at CNY 20 and output at CNY 100, with output five times the price of input. This means compressing output length yields far more savings than compressing input length. Trimming 30% from an input passage often saves less than making the model talk 30% less.

    3. The Four Rules That Actually Move the Bill

    3.1 Batch Channels: Half Price for Offline Work

    Most platforms offer a discount of roughly 50–60% of standard price on the asynchronous batch API, at the cost of no guaranteed real-time return. If your workload includes data cleaning, batch summarization, offline labeling, or evaluation regressions, that traffic should go through batch processing rather than being crammed into the synchronous API. A rough rule of thumb: whatever share of your traffic can be made asynchronous is roughly the share by which your bill can drop.

    3.2 Subscriptions and Off-Peak Scheduling

    Some platforms offer token resource packages or subscription quotas: you pay for a quota per cycle, which covers most calls, and anything beyond it is billed pay-as-you-go. These usually come with off-peak rules—deducting from quota or not billing at all during low-traffic windows. This approach only pays off when “real-time call volume is stable and you can shift part of the traffic to off-peak.” The test is simple:

    • After subscribing, is the pay-as-you-go cost of the overage plus the subscription fee lower than pure pay-as-you-go?
    • If you can automatically schedule batch jobs, offline warm-ups, and non-critical paths to after 18:00, the savings can scale from a dozen-odd percentage points to nearly half.

    3.3 Caching: Pay Once for the Same Prefix

    Prefix caching is the most easily overlooked lever in the cost structure. Once enabled, repeated system prompts, long documents, and tool definitions are charged only at a very low cache-hit price (typically on the order of 10–20% of the standard rate). But note two things: first, the default time-to-live is often only a few minutes, so short-lived caches are nearly useless in low-frequency scenarios; second, cache writes are usually billed separately, tiered by different lifetimes, and frequently switching prefixes means paying both write costs and miss costs.

    Design principle: concatenate the invariant parts (system prompt, few-shot examples, tool schema, fixed knowledge snippets) into a stable prefix placed up front, and put the variable parts at the back; avoid repeatedly changing prefix content within the same session, as that keeps triggering cache writes.

    3.4 Regional Premiums and Free Quotas

    The same platform can charge different prices in different regions. Take regions like Singapore as an example: compared with mainland regions, there is often a premium of around 10%, applied to both input and output. Cross-border calls also require accounting for latency and compliance costs, and this premium is not worth trying to avoid.

    On the other end are free quotas during new-model launch windows: some platforms grant a certain volume of tokens for new models. These are useful for load testing and migration validation, but should not be counted into a long-term cost model.

    4. A Generalized Price Comparison Example

    For illustration, here is a representative price comparison of a flagship model, a mid-tier model, and a code model across four hosting channels (unit: CNY per million tokens, input / cache hit / output):

    Model tier Model vendor platform Cloud vendor A (mainland region) Cloud vendor B (mainland region) Cloud vendor C
    Flagship model 20 / 2 / 100 20 / 2 / 100 20 / not listed / 100 20 / 2 / 100
    Balanced model 13 / 1.3 / 68 13 / 1.3 / 68 13 / 1.3 / 68 13 / 1.3 / 68
    Code model 6.5 / 1.3 / 27 6.5 / 1.3 / 27 6.5 / not listed / 27 6.5 / 1.3 / 27
    Rule that separates them Batch at 60% of standard Not stated on page Free quota for new models Subscription quota with off-peak rules

    The way to read this table is: use the price columns to eliminate, and the rules column to decide. When the price columns are all identical, the first four rows carry no information for you; only the last row is useful.

    5. Running the Numbers: The Real Gap Across Three Combinations

    Take a workload of 10 million input tokens and 0.5 million output tokens per day, with a 30% cache hit rate. The daily bill magnitude comes out roughly as follows (figures illustrate the scale only; refer to each platform’s price page for actual numbers):

    Combination 1: Pure pay-as-you-go, cache fully enabled

    • Input: 7M × 20 + 3M × 2 = 146
    • Output: 0.5M × 100 = 50
    • Total ≈ CNY 196/day

    Combination 2: Pure pay-as-you-go, no cache

    • Input: 10M × 20 = 200
    • Output: 0.5M × 100 = 50
    • Total ≈ CNY 250/day

    Combination 3: 70% through batch (60% discount) + cache fully enabled

    • Synchronous part (30% × 250 ≈ 75) stays at pay-as-you-go
    • Asynchronous part (70% × 250 = 175) × 60% ≈ 105
    • Total ≈ CNY 175/day, about 30% lower than Combination 1

    Combination 4: Subscription covers all real-time calls, non-critical traffic scheduled to off-peak
    Given a pay-as-you-go overage, the savings depend on the subscription’s coverage ratio and the overage ratio. A rule of thumb: when real-time share is high and traffic can be shifted off-peak, overall savings can scale from a dozen-odd percentage points to nearly half.

    Write this arithmetic into your load-testing scripts; it is more reliable than any subjective impression:

    1. def estimate_monthly_cost(input_tokens, output_tokens,
    2. in_price=20, cache_price=2, out_price=100,
    3. cache_hit_rate=0.0, async_ratio=0.0,
    4. batch_discount=1.0, peak_ratio=1.0):
    5. """Estimate monthly inference cost using per-million-token prices"""
    6. uncached_in = input_tokens * (1 - cache_hit_rate)
    7. cached_in = input_tokens * cache_hit_rate
    8. cost_sync = uncached_in * in_price + cached_in * cache_price + output_tokens * out_price
    9. cost_async = cost_sync * batch_discount # batch channel billed at a discount
    10. cost = cost_sync * (1 - async_ratio) + cost_async * async_ratio
    11. return cost * peak_ratio # off-peak scheduling factor; <=1 means savings

    6. A Five-Step Decision Process

    1. Pick the model tier first, then the platform. The price columns are often identical, so your cost ceiling is mainly set by the model tier. If a balanced model can solve the problem, don’t reach for a flagship model just because it is “stronger”—that is an order-of-magnitude difference in cost.
    2. Never call synchronously what can be async. Move batch jobs, offline warm-ups, evaluation regressions, and historical-data processing all to the batch channel.
    3. Enable caching and reorder the prefix. Use a fixed prefix plus a variable suffix, and optimize hit rate and cache-write frequency together.
    4. For real-time traffic, compare subscription vs. pay-as-you-go. Switch only when subscription quota plus pay-as-you-go overage totals less than pure pay-as-you-go.
    5. Connect to the cloud environment you already run on. If your workload already runs on a given cloud (for example, Baidu AI Cloud), prioritize reusing the existing account system, internal network links, and monitoring stack—this is usually both faster and cheaper than cross-cloud calls.

    7. Details That Are Easy to Overlook

    • Cache TTL is very short: often only a few minutes by default. In low-frequency scenarios, a long document placed at the start of the request will almost never hit; make the frequently reused fixed content large and long instead.
    • Cache writes are billed separately: switching the system prompt “invalidates” the prefix, and repeated experimentation quietly piles up cost in unseen places.
    • Old models get retired: earlier model families usually publish a cutoff for new sign-ups and are fully retired months later, with official guidance to migrate to the next generation. Before launch, confirm “the target model will still be available during the planned period”—otherwise migration cost will eat all your savings.
    • Cross-border regional premium is about 10%: applied to both input and output, and cross-border also adds compliance and latency costs.
    • Don’t count free quotas in the long-term model: fine for load testing, not for planning.
    • Stay skeptical of third-party price pages: pages may lag or be vaguely worded, so validate the bill once with a small real request before integration—that is more reliable than reading the table yourself.

    8. Conclusion

    When sticker prices are completely identical, the question “which vendor is cheaper” is itself a false premise. What truly determines the monthly bill is three things: async call ratio, cache hit rate, and off-peak scheduling ability—and all three are engineering problems, not procurement problems.

    In practice, establish three routine mechanisms: monitor cost trends across call volume, cache hit rate, and async ratio; set budget thresholds and alerts for each business line; and complete a migration validation before a model is retired. Do these well, and you won’t need to keep moving between four platforms—because the money you save comes from your own scheduling strategy, not someone else’s price sheet.