Why Do Identical LLM API Prices Produce Bills That Differ by 50%? A Deep Dive into Billing Rules

    Published token prices tell only part of the story. Batch discounts, subscription quotas, off-peak pricing, cache write fees, and TTL policies can make the actual cost of identical LLM workloads vary significantly across cloud platforms. This article explains the main billing differences, quantifies four pricing strategies, and shows how to match billing models with real-time, batch, scheduling, and caching requirements.

    1. The Paradox: Identical Unit Prices, Very Different Bills

    When evaluating large language model APIs, most developers start by comparing per-token prices. Consider two mainstream models, referred to below as Model A and Model B. On the public pricing pages of several cloud providers, their prices per million tokens for input, cache hits, and output may appear to be identical. For example:

    • Model A: $20 per million input tokens, $2 per million cached tokens, and $100 per million output tokens
    • Model B: $13 per million input tokens, $1.30 per million cached tokens, and $68 per million output tokens

    If pricing is the only factor considered, choosing between providers appears to make little difference.

    In practice, however, the same monthly workload—including both real-time requests and offline batch jobs—can produce bills that differ by nearly 50% across platforms. This difference is not necessarily caused by price increases or hidden fees. It usually comes from subtle variations in billing policies, especially:

    • Discounts for batch processing
    • The scope of subscription plans
    • Off-peak pricing
    • Cache write and expiration rules

    These policies are often documented on secondary or tertiary pricing pages rather than on the main price table, making them easy to overlook.

    2. The Three Billing Rules That Determine Total Cost

    2.1 Batch Processing Discounts

    Many cloud providers offer batch APIs for workloads that do not require immediate results. Because these requests can tolerate delays, batch processing is typically priced below standard pay-as-you-go rates.

    For example, one provider may charge 60% of the standard price for batch requests to Model A and Model B. Under this pricing model, Model A would cost:

    • Input: $12 per million tokens
    • Output: $60 per million tokens

    Not every platform clearly publishes its batch discount. Some pricing pages do not mention batch billing at all.

    For workloads such as data cleaning, document summarization, and large-scale content review, choosing a platform with a documented batch discount can reduce the relevant processing cost by 40% without changing the model or the workload.

    2.2 Subscription Plans and Pay-As-You-Go Overage

    Another common pricing structure combines a monthly subscription with usage-based overage charges. Users pay a fixed monthly fee and receive a specified token allowance. Usage beyond that allowance is billed at the standard rate.

    These plans often apply only to selected models. Some also offer off-peak discounts. For example, usage beyond the included quota might be charged at 60% of the standard rate between 6:00 p.m. and 8:00 a.m. the following day.

    Assume a subscription covers Model A, costs X per month, and includes Y million input tokens. If excess usage during off-peak hours is billed at 60% of the standard price, users with variable workloads may reduce their overall cost by approximately 24% to 48%, provided that some jobs can be scheduled during those hours.

    2.3 Automatic Caching and TTL Policies

    LLM APIs often cache repeated prefixes. When a prefix is served from cache, the request may be charged at a lower cache-hit rate. However, whether cache writes are billed separately—and how long the cached content remains valid—can have a major effect on the final cost.

    For example, one model family may charge separately for cache writes under a five-minute or one-hour TTL, while another platform may not prominently display this rule. If an application repeatedly sends the same system prompt or a long context prefix, its cache-hit rate may be high, but creating the cache can still generate additional charges.

    Ignoring cache write fees can cause actual spending to exceed estimates, particularly in multi-turn conversations and batch-generation workflows where prefixes are highly repetitive.

    3. A Quantitative Example: Four Possible Outcomes

    Assume that a business uses Model A over one month and consumes:

    • 50 million input tokens
    • 10 million output tokens
    • Half of the workload in real time and half in offline jobs
    • Offline jobs that can tolerate delays
    • 30% of real-time requests that can be moved to off-peak hours beginning at 6:00 p.m.

    The following four strategies produce different results.

    Strategy 1: Standard Pay-As-You-Go with No Subscription or Batch Processing

    1. Cost = (50 million × $20 + 10 million × $100) / 1 million
    2. = $1,000 + $1,000
    3. = $2,000

    Strategy 2: Batch Processing for Offline Jobs

    The offline half uses batch pricing at 60% of the standard rate. Real-time requests remain on standard pay-as-you-go pricing.

    1. Offline portion:
    2. 25 million input × $20 × 60% = $300
    3. 5 million output × $100 × 60% = $300
    4. Real-time portion:
    5. 25 million input × $20 = $500
    6. 5 million output × $100 = $500
    7. Total cost = $600 + $1,000 = $1,600

    This reduces the cost by 20% compared with Strategy 1.

    Strategy 3: Subscription Plan Without an Off-Peak Discount

    Assume the subscription costs $300 per month and includes:

    • 20 million input tokens
    • 4 million output tokens

    The remaining usage is billed at standard rates.

    1. Subscription fee = $300
    2. Excess usage:
    3. 30 million input × $20 = $600
    4. 6 million output × $100 = $600
    5. Total cost = $300 + $600 + $600
    6. = $1,500

    This reduces the cost by 25%.

    Strategy 4: Subscription Plan with an Off-Peak Discount

    Now assume 30% of the real-time workload is moved to off-peak hours, and excess usage during those hours is billed at 60% of the standard rate.

    The shifted portion consists of:

    • 7.5 million input tokens
    • 1.5 million output tokens

    The excess usage is therefore divided as follows:

    • Non-off-peak: 22.5 million input tokens and 4.5 million output tokens
    • Off-peak: 7.5 million input tokens and 1.5 million output tokens
    1. Total cost = $300
    2. + [22.5 million × $20
    3. + 4.5 million × $100
    4. + 7.5 million × $20 × 60%
    5. + 1.5 million × $100 × 60%] / 1 million
    6. = $300 + ($450 + $450 + $90 + $90)
    7. = $1,380

    This produces a 31% saving compared with Strategy 1.

    If the share of delay-tolerant work increases to 50% and all of it can be run during off-peak hours, the total cost may fall to approximately $1,200, representing a saving of about 40%.

    4. Choosing a Platform Based on Workload Characteristics

    4.1 Mostly Real-Time Inference with No Scheduling Flexibility

    Choose a provider with the lowest effective pay-as-you-go price and no additional conditions. If all providers have the same token prices and the workload must run in real time, the direct API cost will be the same. Service reliability and latency then become the more important selection criteria.

    4.2 A Large Volume of Offline Batch Work

    Prioritize providers that clearly offer batch discounts, such as pricing at 60% of the standard rate. Batch discounts are typically independent of subscriptions and do not require a long-term usage commitment, making them suitable for workloads with uncertain timing but substantial total volume.

    4.3 Predictable Usage with the Ability to Schedule Off-Peak Jobs

    A combination of a subscription plan and off-peak overage pricing may be more economical. The benefit is greatest when the subscription quota is close to actual monthly consumption.

    However, verify the details carefully:

    • The subscription may cover only specific models.
    • Overage usage may not receive any discount outside off-peak hours.
    • The included input and output quotas may be calculated separately.
    • The subscription fee must be included when calculating the break-even point.

    4.4 Workloads with High Cache Utilization

    Check whether the provider charges separately for cache writes and how long cached content remains valid.

    For applications with many repeated prefixes, a provider with free cache writes or a longer TTL may be preferable. At the application level, developers can also improve cache efficiency by:

    • Keeping system prompts consistent
    • Reusing long, stable prefixes
    • Combining short requests where appropriate

    These practices can increase the cache-hit rate, although the resulting savings still depend on the provider’s billing rules.

    5. Model Lifecycle Policies Can Also Affect Cost

    In addition to the three billing categories above, model lifecycle management can indirectly change spending.

    Some older models may be scheduled for deprecation. New users may no longer be able to access them, while existing deployments may eventually be migrated to newer models. A replacement model may offer lower prices, but it may also introduce a different billing structure, such as separate cache write charges.

    Developers should monitor model release notes and deprecation announcements, then plan migrations in advance. This reduces the risk of being forced onto a more expensive replacement when an older model is retired.

    6. Conclusion: Do Not Compare API Prices in Isolation

    The listed token price is only the visible part of LLM API economics. The real cost difference often comes from how a platform handles:

    • Batch processing
    • Subscription quotas
    • Off-peak usage
    • Cache writes
    • Cache TTLs
    • Model lifecycle changes

    Classifying workloads by execution mode—real time or offline, peak or off-peak—can help teams select a more suitable pricing model. Combining workload scheduling with effective cache usage may reduce monthly spending by 40% to 50% without sacrificing service quality.

    A regular billing audit is also valuable. Compare equivalent rules across providers and use a spreadsheet or cost model to simulate different request distributions.

    The cheapest platform is not necessarily the one with the lowest listed token price. It is the one whose billing rules best match the way your application actually runs.