LLM API Pricing Explained: How to Evaluate Real Usage Costs

    This article explains how large language model APIs are actually priced, focusing on token categories, cache hits, subscription plans, pay-as-you-go billing, quota pools, and reset rules. It shows why unit price alone does not determine total cost and provides practical guidance on estimating costs, designing cache-friendly prompts, compressing context, and monitoring quotas.

    Introduction: Real Costs Behind Price Numbers

    When a large language model’s listed price rises sharply, many developers immediately assume the service has become unaffordable. However, focusing only on the per-unit price often leads to the wrong conclusion. Pricing for large-model services is a multidimensional system: unit prices, subscription plans, quota pools, and caching policies all determine the actual amount you pay. Understanding this system is far more important than remembering a single price number.

    This article removes product-specific details and analyzes the cost structure of large-model services from a general technical perspective. It provides a reusable methodology for evaluating real usage costs.

    1. Three Basic Pricing Dimensions

    Most mainstream large-model APIs use itemized, token-based pricing, usually split into three categories:

    • Input tokens: All content sent to the model, including system prompts, conversation history, user input, and attached code context.
    • Output tokens: Content generated by the model. This is usually the most expensive category because generation is computed token by token and requires the most compute.
    • Cache-hit tokens: For repeated input prefixes that can be reused, some services offer discounted pricing, often a fraction of the standard input price.

    A typical scale relationship might be: assume the standard input price is 20 CNY per million tokens, the output price is 100 CNY per million tokens, and the cache-hit price is only 1 CNY per million tokens. The output-to-input price difference is about 5x, and the cache-to-standard-input difference is about 20x.

    This relationship reveals a key fact: the bill is never determined by unit price alone; it is determined by the structure of token flow.

    2. Higher Unit Price Does Not Mean Higher Cost

    Returning to the initial question: if the unit price multiplies several times, does total cost also multiply? The answer depends on three variables.

    1. Cache hit rate

    If your application has many reusable prefixes—such as fixed system prompts, unchanged code-context, or repeated system instructions—cache-hit pricing can sharply reduce that portion of the cost. A well-designed caching strategy can make actual input costs much lower than the standard input price.

    2. Buffering effect of subscription plans

    For users who access services through subscription plans rather than pure pay-as-you-go, the billing structure looks completely different. Subscription plans usually provide a fixed monthly quota pool. Within that pool, what you experience is the “rate of quota consumption,” not the “unit price.” Unit-price increases may be absorbed by the plan’s bundled pricing, so short-term differences may be minimal.

    3. Output share

    Because output tokens are priced much higher than input tokens, an output-heavy workload—such as long-form generation or complex code completion—has a much greater impact on cost than an input-heavy workload. If your task has long input but short output, the impact of unit-price increases is diluted.

    3. Subscription Plans vs. Pay-as-You-Go

    This is a common source of confusion. The cost logic of the two models is fundamentally different:

    Dimension Pay-as-you-go (API mode) Subscription plan mode
    Billing unit Per million tokens Fixed monthly quota
    Unit-price sensitivity High; unit price directly affects the bill Low; absorbed by bundled plan pricing
    Quota limit No fixed cap (subject to account limits) Fixed pool; exhaustion requires waiting or purchasing more
    Cost predictability Fluctuates with usage Relatively predictable

    Quota pools and reset mechanisms

    A core concept in subscription plans is the “quota pool.” Different features—such as chat APIs and code-completion APIs—may belong to different quota pools and be measured separately. This has two practical implications:

    • You cannot use remaining quota in one pool to cover a shortfall in another pool.
    • Quotas may be reset periodically—for example, weekly or daily.

    This means that even if the subscription advertises a large monthly total, heavy users may still run out of quota mid-period if the quota resets weekly with a low weekly cap. When evaluating a subscription plan, you must check the reset cycle and per-cycle limit, not just the monthly total.

    4. Cost Quantification: Calculate Your Bill with Code

    Do not rely on intuition; rely on calculation. Below is a general cost estimation function (illustrative):

    1. def estimate_cost(
    2. input_tokens: int,
    3. output_tokens: int,
    4. cache_hit_tokens: int,
    5. price_input: float = 20.0, # CNY per million tokens, standard input
    6. price_output: float = 100.0, # CNY per million tokens, output
    7. price_cache: float = 1.0, # CNY per million tokens, cache hit
    8. ) -> dict:
    9. """Estimate the cost of a single call using itemized token pricing (CNY)."""
    10. # Note: cache-hit tokens are usually deducted from standard input tokens
    11. billable_input = max(0, input_tokens - cache_hit_tokens)
    12. cost_input = billable_input / 1_000_000 * price_input
    13. cost_output = output_tokens / 1_000_000 * price_output
    14. cost_cache = cache_hit_tokens / 1_000_000 * price_cache
    15. total = cost_input + cost_output + cost_cache
    16. return {
    17. "input_cost": round(cost_input, 6),
    18. "output_cost": round(cost_output, 6),
    19. "cache_cost": round(cost_cache, 6),
    20. "total_cost": round(total, 6),
    21. }

    Example: input 8,000 tokens, of which 6,000 are cache hits; output 2,000 tokens.

    1. result = estimate_cost(
    2. input_tokens=8000,
    3. output_tokens=2000,
    4. cache_hit_tokens=6000,
    5. )
    6. # Output:
    7. # billable_input = 2000
    8. # input_cost = 2000 / 1e6 * 20 = 0.04 CNY
    9. # output_cost = 2000 / 1e6 * 100 = 0.20 CNY
    10. # cache_cost = 6000 / 1e6 * 1 = 0.006 CNY
    11. # total_cost ≈ 0.246 CNY

    Without cache hits, the same 8,000 input tokens would cost 0.16 CNY, making the total bill 0.36 CNY. Caching reduces the cost of this call by nearly 32%. This is why cache hit rate is often more important to monitor than unit price when evaluating costs.

    5. Implementing Caching Strategies

    To benefit from cache discounts, make reusable prefixes as stable and front-loaded as possible. The core principle: put invariant content first and variable content later.

    1. def build_prompt(system_static, retrieved_docs, user_query, timestamp):
    2. """
    3. Prompt segmentation order:
    4. 1. system_static: fixed system instructions, long-lived, prioritize cache hits
    5. 2. retrieved_docs: retrieval-augmented context, may change
    6. 3. user_query: real-time user input, different every time
    7. 4. timestamp: dynamic information, placed at the end
    8. """
    9. return "\n".join([
    10. system_static, # Front: stable prefix to maximize cache reuse
    11. retrieved_docs,
    12. f"User question: {user_query}",
    13. f"Current time: {timestamp}", # End: dynamic content should not break prefix caching
    14. ])

    If you insert dynamic content into the prefix—for example, placing a timestamp at the very beginning—the entire prefix cache is invalidated, and subsequent tokens are billed at the standard price. This is a hidden reason why many teams’ bills exceed expectations.

    6. Context Compression and Quota Management

    In addition to caching, two engineering methods help control costs.

    Context compression: Conversation history accumulates tokens linearly. Using a sliding window to keep only the most recent N turns and summarizing earlier conversations can significantly reduce input tokens. In practice, define a clear history-trimming strategy rather than stacking history indefinitely.

    Quota pool monitoring: Under subscription plans, monitor the remaining quota and reset cycle for each feature pool separately. A simple state tracker can help avoid mid-period exhaustion:

    1. class QuotaTracker:
    2. def __init__(self, pool_limit: int, reset_interval_days: int = 7):
    3. self.pool_limit = pool_limit
    4. self.reset_interval = reset_interval_days
    5. self.used = 0
    6. def consume(self, amount: int) -> bool:
    7. if self.used + amount > self.pool_limit:
    8. return False # Quota insufficient; wait for reset or switch to a fallback strategy
    9. self.used += amount
    10. return True
    11. def remaining(self) -> int:
    12. return self.pool_limit - self.used

    7. Summary of the Cost Evaluation Methodology

    Based on the analysis above, evaluating the real cost of a large-model service should follow this framework:

    1. Look at structure, not unit price: First measure the ratio of input/output/cache, rather than memorizing a unit price.
    2. Maximize cache hits: Design stable prefixes and make full use of cache discounts. This is often the most cost-effective way to reduce costs.
    3. Distinguish billing models: For subscriptions, focus on quota pools, reset cycles, and pool isolation; for pay-as-you-go, focus on usage curves and account limits.
    4. Compress and trim context in parallel: Summarize and window-trim conversation history to avoid linear growth in input tokens.
    5. Build a quantitative dashboard: Use code to formalize cost estimation and quota monitoring, turning “perceived” cost into observable data.

    Conclusion

    A doubled unit price alone is not a decision-making basis. What really determines cost is your call structure, caching strategy, and choice of billing model. On Baidu AI Cloud’s large-model service platform, the same general model of token-based itemized pricing and cache reuse applies. Mastering this methodology helps you keep costs under control no matter which service you use.

    Instead of worrying about price fluctuations from one model generation to the next, focus on the engineering areas you can control: cache design, context management, and quota planning. That is the real lever for improving efficiency while reducing cost.