How LLM API Pricing Actually Works: Cost Estimation and Model Selection Guide

    Large language model API costs are rarely what the listed prices suggest. This article breaks down four common pricing mechanisms—time-based tiered pricing, context caching, batch processing discounts, and priority channels—explains the pitfalls of each, and provides a reusable cost estimation script and a practical selection checklist to help you model real-world token spend before committing to a provider.

    Why Comparing Unit Prices Gets Your Budget Wrong

    Many teams select an LLM provider by comparing output token prices alone, but the actual bill depends on multiple variables: the input-to-output ratio, cache hit rate, traffic distribution across time slots, whether batch processing is used, and context length limits. For the same model, prices can differ by several times—or even an order of magnitude—across billing dimensions. Ignoring any single factor can push your cost estimate more than 50% off the real invoice.

    Based on typical pricing structures published by four domestic Chinese providers, this article abstracts four representative billing mechanisms and provides a general-purpose cost estimation method.

    Four Typical Pricing Mechanisms Explained

    1. Time-Based Tiered Pricing: Availability Off-Peak in Exchange for Discounts

    A flagship model from one provider uses a dual-tier “peak / off-peak” price structure:

    Tier Input (cache hit) Input (cache miss) Output
    Off-peak 0.15 4.5 13.5
    Peak 0.30 9.0 27.0

    Peak tiers cover weekday morning and afternoon business hours; off-peak prices are roughly half. Key rule details:

    • Billing tier is determined by the request completion time, not the submission time. A long request submitted at 11:58 and completed at 12:02 is billed at the off-peak rate—a favorable rule for batch long-text jobs.
    • The hidden cost of the off-peak tier is possible throttling or queuing. For latency-sensitive online services, the cheaper tier may not be viable at all.

    When it fits: Offline jobs (log summarization, data labeling assistance, overnight batch runs) can be scheduled entirely into off-peak windows, cutting costs roughly in half. Online customer-facing services are effectively locked into peak rates.

    2. Tiered Cache Pricing: Hit Rate Determines Your Real Unit Price

    Several providers now offer “context caching,” where cache-hit input tokens cost as little as one-tenth of the cache-miss price—but the patterns differ:

    • Automatic caching with storage fees: One provider charges separately for cache storage duration (on the order of roughly 0.02 RMB per million tokens per hour). The longer and larger the cache, the higher the storage cost, so you must weigh the hit rate gains against growing storage fees.
    • Explicit write pricing: Another provider charges for cache writes tiered by TTL—a 1-hour TTL write costs twice as much as a 5-minute TTL. The TTL should match your application’s session rhythm; otherwise you pay to maintain caches that never get used.

    The core formula for estimating cache savings:

    1. Effective input price = hit_rate × hit_price + (1 - hit_rate) × miss_price

    Take a model with a hit price of 0.02 and a miss price of 1. At an 80% hit rate, the effective input price is only 0.216—about a 78% reduction versus no caching. Multi-turn conversations and RAG with fixed prefixes (system prompt + retrieved knowledge base content) are the two scenarios with the highest hit rates.

    3. Batch Processing Discounts and Stacking Restrictions

    Half-price batch channels are widespread, but there are two common traps:

    • No stacking: One provider’s official rules state explicitly that the Batch half-price discount and the context caching discount cannot apply simultaneously. If your batch jobs have a high cache hit rate (e.g., repeatedly processing the same set of documents), caching may beat the Batch discount—compute both options separately before choosing a channel.
    • Non-real-time: Batch channels typically complete within a few hours to 24 hours, with no throughput guarantee. They only suit latency-insensitive tasks.

    The comparison formula is simple:

    1. Batch cost = list_price × 0.5 × input_tokens (no cache)
    2. Cached cost = input_tokens × (hit_rate × hit_price + (1 - hit_rate) × list_price)

    When the hit rate exceeds roughly 50%, the caching route usually beats the Batch half-price option (verify with your actual prices).

    4. Priority / High-Speed Channel Premiums

    One provider offers a standard version and a high-speed version priced at twice the standard rate. These channels get looser rate limits during peak hours and more stable latency. The sensible approach is not to “move everything to the fast lane” but to tier by business criticality: use the high-speed version for core request paths, and the standard version for non-critical features (draft generation, internal tools). Overall cost stays within roughly 1.2–1.5× the standard price.

    A General-Purpose Cost Estimation Script

    The following script estimates monthly cost given a business profile:

    1. def monthly_cost(input_tokens, output_tokens,
    2. hit_rate=0.0, in_hit=0.0, in_miss=1.0,
    3. out_price=4.0, peak_ratio=1.0, discount=1.0):
    4. in_price = hit_rate * in_hit + (1 - hit_rate) * in_miss
    5. cost = input_tokens * in_price + output_tokens * out_price
    6. return cost * peak_ratio * discount
    7. # Example: 200M input tokens + 40M output tokens per month,
    8. # 70% cache hit rate, 60% of traffic during peak hours
    9. cost = monthly_cost(
    10. input_tokens=200, output_tokens=40, # Unit: million tokens
    11. hit_rate=0.7, in_hit=0.04, in_miss=2.0,
    12. out_price=8.0, peak_ratio=0.6 + 0.4 * 0.5, # Off-peak billed at half price
    13. )
    14. print(f"Estimated monthly cost: {cost:.1f}")

    Model Selection Checklist

    Combining the mechanisms above, check the following in order:

    1. Profile your workload: Quantify four parameters first—input/output ratio, context repetition rate, traffic distribution across time slots, and latency requirements.
    2. Calculate effective prices, not list prices: Factor in cache hit rate, time-based discounts, and batch discounts to get an “effective blended unit price” before comparing providers.
    3. Check stacking rules: Look for mutually exclusive discount clauses, cache TTLs, and per-request token limits (some providers cap a single request at around one million tokens; very long documents need to be split).
    4. Budget for migration costs: Message structures and tool-calling protocols are largely converging across providers, but streaming behavior and error codes differ. Keep an adapter layer in your code to lower future switching costs.
    5. Validate with limited traffic: Run a 1–2 week gray release with real traffic, compare the actual invoice against your estimate, and recalibrate assumptions such as the hit rate.

    Summary

    LLM API cost optimization is fundamentally a matching problem between billing mechanisms and workload profiles. Combining time-based scheduling, cache hit rate optimization, channel tiering, and batch processing can typically compress the bill to less than half of what naive provider selection would cost. Model first, estimate second, then validate with a gray release—this is the reliable path to avoiding billing surprises.