LLM API Cost Breakdown: Why the Same Unit Price Can Produce a Monthly Bill That's Twice as High

    Unit price is only one variable in the LLM API billing formula. This guide breaks down every billing dimension that affects your monthly bill—including input and output tokens, context caching, batch discounts, regions, and subscription plans—and provides a Python-based cost estimation model with worked examples showing how an 80% cache hit rate can cut a monthly bill from 1260 to about 623 in a hypothetical scenario.

    Unit Price Is Just One Variable in the Billing Formula

    The basic formula for pay-as-you-go LLM API billing is: monthly bill = Σ (token usage of each type × corresponding unit price). When comparing platforms, many people only look at input and output unit prices, while overlooking caching, batching, regional, and plan rules. Even with identical unit prices, differences in usage structure or billing rules can produce dramatically different totals.

    Evaluating cost therefore takes three steps: first, confirm how many tokens each call consumes; second, confirm the price applied to each token type; third, confirm which rules change the actual billed amount.

    Common Billing Dimensions

    Pay-as-you-go billing typically involves the following items, each with logic that must be checked separately.

    • Input tokens: everything submitted in the request, including system instructions, conversation history, retrieved documents, and the user’s question.
    • Output tokens: the content the model generates. The industry convention is to price output higher than input, so long responses have a major cost impact.
    • Cache-hit tokens: when a repeated prefix is reused, it is billed at a significantly lower price than normal input.
    • Cache writes: some services charge separately the first time a prefix is written to the cache. The cache also has a time-to-live (TTL); after expiry, the prefix must be written again.
    • Batch and asynchronous calls: some services offer discounts for latency-insensitive offline tasks.
    • Region: the same service may be priced differently across regions. Cross-region calls also add latency and affect data residency compliance.
    • Free quotas and subscription plans: free quotas for new models, plus the coverage and usage limits of monthly plans, all change the effective cost.

    Caching: The Most Overlooked Cost Lever

    Most caching mechanisms work by prefix matching. The beginning of a request must exactly match a previous request to hit the cache. Your prompt structure should therefore follow the principle of “stable content first, variable content last.”

    1. messages = [
    2. # Stable prefix: system instructions, tool definitions, fixed knowledge — repeatedly cacheable
    3. {"role": "system", "content": STATIC_SYSTEM_PROMPT},
    4. # Reference material that stays constant within the same session
    5. {"role": "user", "content": f"Reference:\n{static_reference}"},
    6. # Parts that change on every request go last
    7. {"role": "user", "content": user_question},
    8. ]

    Two design points to watch:

    1. Avoid embedding dynamic information in the prefix. Timestamps, request IDs, and random values at the start make every prefix unique, so the cache never hits.
    2. Mind the relationship between call intervals and TTL. If the interval between two calls exceeds the cache’s lifetime, the cache expires. High-frequency, continuous sessions benefit fully from caching; low-frequency tasks may see very low hit rates even with caching enabled.

    When estimating caching benefits, use measured hit rates rather than assumed values.

    Batch and Asynchronous Calls

    Batch calls suit offline evaluation, data cleaning, and bulk summarization—tasks that don’t require real-time responses. Because these tasks can be queued, providers can offer discounts.

    When using batch discounts, confirm which models and billing items the discount applies to. Some services limit it to certain models; others apply it only to input or output tokens. In your calculations, multiply the discount only by eligible usage—don’t discount the entire bill.

    Regions and Compliance

    Region selection requires weighing three factors:

    • Price: unit prices may differ by region, and overseas regions are often more expensive.
    • Latency: the greater the distance between your application servers and the model service’s region, the longer each call takes.
    • Data residency: some industries and jurisdictions require data to be stored or processed within their borders, and this requirement usually takes priority over price.

    If you use multiple regions, also account for cross-region network and data transfer costs instead of comparing unit prices alone.

    Break-Even Analysis for Plans and Free Quotas

    Whether a subscription plan pays off depends on whether your actual usage covers its fixed cost. Calculate it as follows:

    • Effective unit price of a plan = monthly plan fee ÷ monthly tokens covered by the plan
    • A plan beats pay-as-you-go only when actual usage stays steadily above the break-even point: break-even usage = monthly plan fee ÷ average pay-as-you-go unit price

    Also verify three things: which models the plan covers, whether there are off-peak or concurrency limits, and at what price overage is billed. These rules are often ignored but directly affect the outcome.

    Model Iteration and Retirement

    Model versions get updated, and older versions may be closed to new users and later retired. During migration, unit prices, cache behavior, output style, and context length can all change—simply swapping the model name is not enough.

    Recommended practices:

    1. Build a model abstraction layer. Business code should read the model name and parameters from configuration, so switching models means changing only the config.
    2. Maintain a fixed evaluation set. Before and after migration, compare quality, latency, and cost on the same sample set to confirm results still meet requirements.
    3. Periodically re-verify billing rules. Pricing pages and rule descriptions get updated; record verification dates and re-check at key milestones.

    The same approach applies to Baidu AI Cloud’s large model services; refer to its official pricing page for specific prices and rules.

    Cost Estimation Method and Example

    The following function combines the dimensions above into a monthly cost estimation model. Prices are in yuan per million tokens.

    1. def monthly_cost(
    2. calls_per_day: int,
    3. input_tokens: int,
    4. output_tokens: int,
    5. cache_hit_ratio: float,
    6. cache_write_ratio: float,
    7. price_input: float,
    8. price_cache_hit: float,
    9. price_cache_write: float,
    10. price_output: float,
    11. batch_discount: float = 1.0,
    12. days: int = 30,
    13. ) -> float:
    14. """Estimate monthly cost (yuan). Prices in yuan per million tokens."""
    15. if cache_hit_ratio + cache_write_ratio > 1:
    16. raise ValueError("Cache hit ratio plus cache write ratio cannot exceed 1")
    17. calls = calls_per_day * days
    18. total_in = calls * input_tokens
    19. total_out = calls * output_tokens
    20. hit_tokens = total_in * cache_hit_ratio
    21. write_tokens = total_in * cache_write_ratio
    22. normal_tokens = total_in - hit_tokens - write_tokens
    23. cost = (
    24. normal_tokens * price_input
    25. + hit_tokens * price_cache_hit
    26. + write_tokens * price_cache_write
    27. + total_out * price_output
    28. ) / 1_000_000
    29. return cost * batch_discount

    The following hypothetical numbers illustrate the impact of caching on a bill. They are for demonstration only and do not represent any platform’s actual pricing.

    • Scenario setup: 1,000 calls per day, 3,000 input tokens and 300 output tokens per call, over 30 days.
    • Assumed prices: input 10, cache hit 1, cache write 12.5, output 40 (all per million tokens).

    Scenario A: No cache hits. Total input is 90M tokens, costing 900. Total output is 9M tokens, costing 360. Monthly total: 1,260.

    Scenario B: 80% hit rate, 5% write ratio. Cache hits cover 72M tokens, costing 72. Writes cover 4.5M tokens, costing 56.25. Normal input is 13.5M tokens, costing 135. Input totals 263.25; adding 360 of output gives a monthly total of about 623.

    Between these two scenarios, the unit prices are identical, yet the bill differs by roughly half—driven entirely by how the usage is structured. The same logic applies to every dimension in this guide: before comparing platforms, model your own token usage, apply each billing rule correctly, and measure caching behavior in production rather than assuming it.