LLM API Cost Comparison: Why Identical Token Prices Produce Bills That Differ by Half

    Even when several LLM API platforms quote identical per-million-token prices for the same model, their monthly bills can differ by nearly half. The reason is that an LLM API bill is not a single multiplication of price by usage, but the combined effect of usage structure and billing rules. This article breaks down what an LLM API bill is made of, explains five levers that can materially reshape it—batch discounts, subscription credit with off-peak scheduling, prompt engineering and cache hit rate, regional pricing, and model version lifecycle—and provides a rule-based estimation model together with a selection and implementation checklist. Readers will learn how to profile their usage, quantify the real effect of each lever, and turn "the same unit price" into a controllable bill.

    1. The Problem: Same Unit Price, Bills Differ by Half

    In enterprise LLM integration projects, selection reviews often hit this scenario: several platforms quote exactly the same per-million-token price for the same model. Checking the input, cache-hit, and output prices line by line, they match to the last digit. By the “unit price × usage” estimate, the monthly bills should come out roughly the same. Yet after a single billing cycle, some teams’ bills are far off from the initial estimate.

    The root cause is that LLM API billing is not a single multiplication but a stack of rules. The per-unit price is only one input. The actual bill is determined jointly by “usage structure” and “billing rules,” which include at least five factors: batch discounts, subscription credit offset, cache hits and writes, regional pricing, and model version and lifecycle. The first three often directly reshape the magnitude of the bill. The same model at the same unit price can produce a bill difference close to a factor of two, depending on how it is used. The illustrative calculations below confirm this.

    2. How the Bill Is Calculated: First, Understand the Components

    Pricing unit: three-part pricing

    Mainstream LLM APIs typically use three-part pricing, quoted per million tokens:

    • Input price: input tokens that do not hit the cache;
    • Cache-hit price: once a repeated prefix hits the cache, tokens are billed at a price far below the input price;
    • Output price: tokens generated by the model, usually several times the input unit price.

    Some platforms also bill cache writes separately. The billing boundary also includes: a single call is typically billed as “input + output,” not by character count; multimodal content is first converted to tokens; and the longer the context, the higher the input-side cost share.

    Usage profile: six statistics

    To estimate the bill, first translate the business into six statistics. Total monthly cost can be expressed as:

    1. Total monthly cost = Σ(subscription fee per platform + overage cost after credit offset × overage discount factor + cache write cost)
    2. Overage cost = Σ(input tokens × input price + cache-hit tokens × hit price + output tokens × output price)
    3. Overage discount factor = share of standard-price calls × 1 + share of batch calls × batch discount + share of off-peak calls × off-peak discount
    • Input/output length distribution: determines the cost structure. The multiple relationship between input and output prices directly determines where optimization should focus;
    • Prompt prefix repetition rate: sets the ceiling on cache benefits;
    • Cache reuse count k: how many times the same prefix is reused on average within the TTL;
    • Batch-migratable share: the share of requests that tolerate longer completion latency;
    • Off-peak-migratable share: the share of calls the business allows scheduling into low-traffic windows;
    • Total monthly token volume: determines whether the subscription credit can be fully used.

    Three easily overlooked billing rules

    • Cache writes are billed separately. Automatic cache reuse means repeated prefixes need no extra engineering, but some content must first be “written” to the cache, and the benefit only materializes after hits within the TTL.
    • Batch applies only to offline jobs. The batch channel trades longer completion latency for a price discount; it is meaningless for real-time online requests.
    • Subscription credit has an offset order and a validity period. Credit usually offsets first and billing only then switches to pay-as-you-go; most credit is valid only within the subscription cycle, and any unused portion becomes a sunk cost.

    3. Decomposing the Variables: Five Levers That Rewrite the Bill

    Lever 1: Batch — trade latency for discount

    Some platforms’ pricing pages list both a standard and a batch tier; for example, batch for the same model may be billed at 60% of the standard price (illustrative; refer to each platform’s official pricing page for the actual rules). This lever applies only to tasks insensitive to response time: log analysis and summarization, batch translation and document extraction, offline index building, evaluation/regression, content-moderation pre-screening, and so on.

    Forcing batch onto online chat, real-time risk control, and similar tasks brings timeouts and degraded experience—trading availability for cost is not worth it. The more robust engineering approach is to split into two channels: real-time requests go through the standard channel, while offline jobs enter a task queue and are dispatched to the batch channel by a scheduler.

    Lever 2: Subscription credit and off-peak scheduling — “scheduling calls into the valley”

    Some platforms offer token subscriptions or resource packs that offset models within their coverage against the plan’s credit, with overage switching to pay-as-you-go; some packs also grant off-peak discounts for low-traffic windows. The core of such plans is not “a lower unit price” but concentrating deferrable calls into off-peak windows and using the credit as fully as possible.

    Three lessons from practice:

    • First profile the call distribution across time slots to identify the migratable share;
    • Use scheduled jobs and message queues to place non-real-time tasks into off-peak windows;
    • Record consumed credit at the gateway layer, and pause off-peak retries when the credit is nearly exhausted, so that not everything converts to standard-price overage once the credit runs out.

    Note that when there is no task to schedule during off-peak or low-traffic windows, this lever’s benefit is zero.

    Lever 3: Prompt engineering and cache hit rate — the reuse benefit of repeated prefixes

    Taking the common illustrative price of “output price = 5× input price”: the cost of consuming one output token equals that of consuming five input tokens. Therefore, shortening output length (constraining answer length, avoiding redundant explanation) is often more direct than compressing input.

    Cache hits, in turn, depend on whether the prefix is stable. Keeping the system prompt, tool definitions, and few-shot examples byte-for-byte consistent, and avoiding concatenation of timestamps, random IDs, or dynamic retrieval results into a fixed prefix, can significantly raise the reuse rate of repeated prefixes. The specific benefit can be estimated with:

    1. Cache net savings rate = (1 - hit rate) × input share × (1 - hit price/input price) - write share × (write price/input price - 1)

    When “write price = input price, hit rate = 50%, input share = 50%,” the cache net saving is about 12.5%. If the write price exceeds the input price, a low enough hit rate can even produce a net loss. Therefore, cache strategy should be evaluated on the measured hit rate, not assumed to save money simply because it is turned on.

    Lever 4: Region and network path — certain regions are priced higher

    Some platforms price different geographic regions differently, and certain regions carry higher per-token rates than others. Higher-cost regions also add cross-border network latency and data-residency/compliance requirements. For latency-sensitive online workloads, cross-region calls may trigger more retries and indirectly raise token consumption.

    Two architectural recommendations: first, include region in the cost model rather than comparing only model unit prices; second, prefer access points in the same region as the business users, treating “nearest access” as part of cost optimization rather than a mere technical preference.

    Lever 5: Model version and lifecycle — version strategy is a long-term variable

    Large models iterate frequently, and new-generation versions usually offer better unit prices at the same capability tier. The hidden costs of locking to an old version long-term include: missing out on later price optimizations, a compressed migration window before the version is retired, and cache invalidation and prompt-adaptation costs during migration.

    Three recommended strategies:

    • Abstract the model at the business integration layer, making the model identifier configurable to avoid hardcoding;
    • Establish an offline evaluation and canary-comparison mechanism for old and new versions, deciding switches on real business metrics rather than leaderboard scores;
    • Watch official announcements of service-shutdown dates and reserve at least one iteration cycle of migration buffer.

    4. Quantification: Encode the Rules into Your Estimation Model

    1. Illustrative estimation script

    1. def estimate(usage: dict, price: dict) -> dict:
    2. """
    3. Estimate monthly cost based on the rules (illustrative code).
    4. usage: number of requests, average input/output length, cache hit rate,
    5. batch share, off-peak share, etc.
    6. price: input price, cache-hit price, cache-write price, output price,
    7. batch discount, off-peak discount
    8. """
    9. reqs = usage["reqs"]
    10. in_tok = reqs * usage["in_len"] / 1e6 # million tokens
    11. out_tok = reqs * usage["out_len"] / 1e6
    12. hit_rate = usage["cache_hit_rate"]
    13. # Hit tokens are billed at the hit price; the miss portion at the input
    14. # price (including the first write; substitute the corresponding unit
    15. # price if the write is billed separately).
    16. cache_hit_tok = in_tok * hit_rate
    17. miss_tok = in_tok - cache_hit_tok
    18. cost = (miss_tok * price["in"] + cache_hit_tok * price["cache_hit"]
    19. + out_tok * price["out"])
    20. batch_ratio = usage["batch_ratio"]
    21. offpeak_ratio = usage["offpeak_ratio"]
    22. # Estimated stacking of the batch and off-peak discounts; defer to the
    23. # platform's actual rules.
    24. factor = (1 - batch_ratio) * (1 - offpeak_ratio) * 1.0 \
    25. + batch_ratio * price["batch_discount"] \
    26. + offpeak_ratio * price["offpeak_discount"]
    27. return {"monthly_cost": cost * factor, "tokens": in_tok + out_tok}

    2. Typical scenario estimation

    Take a business that is “mostly real-time chat with some offline summarization”: 100 million requests per month, averaging 2,000 input tokens and 300 output tokens, for a monthly total of about 230 billion tokens.

    Plan Input Cache hit Output Additional rules Monthly cost (illustrative) vs. Plan A
    A Standard pay-as-you-go + cache 200B 40% 300B None ≈¥1.24M —
    B Switch to new-generation version 200B 40% 300B None ≈¥708K -43%
    C 30% offline via batch 140B 28% 210B 70% standard + 30% batch (at 60%) ≈¥1.152M -7%
    D 60% of calls scheduled off-peak 80B 16% 120B 40% standard + 60% off-peak (at 50%) ≈¥809K -35%
    E Subscription + overage 200B 40% 300B 50B-token credit offset, overage at 80% ≈¥1.128M -9%

    Estimation assumptions: subscription fee ¥10,000, cache hit rate 40%, batch and off-peak both at 60%, overage at 80%; unit is ¥ per million tokens.

    Two conclusions:

    • Lever benefits show diminishing returns. Used alone, plans C and D yield limited savings, while stacking them saves about 42%; plan B saves about 43% by version replacement alone. Multiple levers must stack in the same direction to push the bill down by roughly half.
    • Input share determines the optimization focus. If output length doubles to 600 tokens, the input share drops from 87% to 75%, and plan D’s relative saving would shrink from 35% to about 30%—the value of cache-type optimizations declines as output grows, at which point compressing output should take priority.

    5. Selection and Implementation Checklist

    1. Five-question decision table

    Decision question If “yes” If “no”
    Are there many offline jobs that can be batched? Prefer platforms with batch support, and run the offline pipeline through the batch channel separately Limited batch benefit; focus on caching and output compression
    Can call volume be concentrated in a fixed off-peak window? Consider a subscription/resource-pack plan and schedule off-peak tasks into the low-traffic window Subscription credit tends to sit idle; prefer standard pay-as-you-go + cache optimization
    Is the prompt prefix highly repetitive? Prioritize cache-hit optimization, but verify the net benefit by measurement First clean up the prompt structure, then talk about caching
    Does the business depend heavily on a single model’s capability? Watch version lifecycle and migration cost, and keep the integration layer abstracted Keep multiple versions selectable and route tasks to different price tiers
    Does it involve cross-border data transfer or cross-region latency? Prefer same-region access, and factor it into the cost and compliance evaluation Other regions can be compared, but latency and compliance costs must be included

    2. Four engineering must-dos

    • Metering and allocation: instrument at the gateway or SDK layer to break down token consumption and cost by business line, tenant, and endpoint, so that “when the bill arrives, you can pinpoint it to the business”;
    • Budget and alerts: set daily/monthly budget thresholds and anomaly-spike alerts to catch hidden consumption such as retry loops and over-long context concatenation;
    • Output-side optimization: cap maximum output length, streamline the system prompt, and use more compact formats for structured output, directly compressing the most expensive tokens;
    • Cache hit monitoring: put cache hit rate and write share on a monitoring dashboard, periodically recomputing the net benefit to avoid “turning on the cache actually costs more.”

    3. Common misconceptions

    • Looking only at unit price, not rules: on platforms with identical unit prices, rule differences are the real source of bill differences;
    • Treating the cache as saving money by default: with a low hit rate or a high write price, the cache can increase cost;
    • Buying subscription credit at peak: unused credit is a sunk cost, so estimate it based on the actually migratable volume;
    • Ignoring the version migration window: switching only near shutdown amplifies both the migration cost and the risk;
    • Ignoring region and compliance: the price premium and network latency of certain regions are often left out of the cost model.

    6. Conclusion

    Optimizing LLM API cost is essentially translating “billing rules” into “architectural decisions”: batch corresponds to channel splitting, subscription and off-peak correspond to scheduling strategy, cache hits correspond to prompt engineering, and region and version correspond to architectural abstraction and lifecycle management. Only after clearly calculating the usage profile and the billing rules can you decide which rule to apply—turning “the same unit price” into “a controllable bill.”