LLM API Cost Breakdown: Unit Price, Token Usage, and Cache Reuse

    This article explains why comparing per-token unit prices alone can lead to incorrect LLM cost estimates and how to calculate cost per task using input, output, cache-hit, and reasoning-related token usage. It covers how to convert subscription quotas into task capacity, a sample Python calculation, engineering factors that drive costs up, practical optimization methods, and a checklist for model or service selection.

    1. Why Unit Price Alone Can Lead to Wrong Cost Estimates

    Many teams compare only the price per million tokens when selecting a model. This comparison has two problems. First, input and output prices are usually different, and output is often significantly more expensive, so the two must be calculated separately. Second, different models can consume very different numbers of tokens to complete the same task. A model with a lower unit price but verbose output may have a higher cost per task than a model with a higher unit price but concise answers.

    The comparison baseline should therefore be cost per task—the actual amount spent to complete a business task—rather than unit price alone.

    2. The Four Components of the Billing Model

    1. Input tokens
    Input includes system prompts, conversation history, retrieved document chunks, and tool-call results. In multi-turn conversations, history accumulates with each turn, and the number of input tokens grows approximately linearly with conversation length.

    2. Output tokens
    Output is the content generated by the model. Output unit prices are generally higher than input unit prices, so controlling response length and requiring structured output can have a clear impact on cost reduction.

    3. Cache hits and misses
    Some services offer cached billing for repeated request prefixes, with the cached portion priced significantly lower than normal input. For caching to take effect, the request prefix must be stable—for example, the system prompt and fixed examples should be placed first, and dynamic content should be placed at the end. If a timestamp or random field is inserted at the front of every request, the cache will rarely hit.

    4. Reasoning-related tokens
    When a longer reasoning process is enabled, intermediate generated content may be counted as output tokens. Whether it is billed and how it is billed should be based on the official documentation of the service used. This part must be included in estimates; otherwise, actual costs will be underestimated.

    3. Converting Subscription Quotas into Cost per Task

    Subscription plans usually express quotas in terms of request counts, time, or converted points. To determine whether a plan is cost-effective, you need to convert the quota into the number of tasks that can be completed. The conversion steps are:

    1. Extract a representative set of real tasks, preferably no fewer than 100, covering simple, ordinary, and complex scenarios.
    2. Record the input tokens, cache-hit tokens, and output tokens for each task.
    3. Calculate the cost per task using the formula.
    4. Divide the plan quota by the cost per task to get the number of tasks that can be completed.

    The cost per task formula is:

    Cost per task = uncached input tokens × input unit price + cache-hit tokens × cache unit price + output tokens × output unit price

    The following code demonstrates the calculation. The unit prices are example values used only to illustrate the method and do not represent the pricing of any actual service.

    1. # Example unit prices, in CNY per million tokens, for demonstration only
    2. PRICE = {"input": 10.0, "cache_hit": 1.0, "output": 40.0}
    3. def task_cost(input_tokens: int, cached_tokens: int, output_tokens: int,
    4. price: dict = PRICE) -> float:
    5. """Calculate the cost of a single task in CNY."""
    6. if cached_tokens > input_tokens:
    7. raise ValueError("Cache-hit tokens cannot exceed input tokens")
    8. uncached = input_tokens - cached_tokens
    9. total = (uncached * price["input"]
    10. + cached_tokens * price["cache_hit"]
    11. + output_tokens * price["output"])
    12. return total / 1_000_000
    13. if __name__ == "__main__":
    14. # 20,000 input tokens, of which 15,000 hit the cache; 2,000 output tokens
    15. cost = task_cost(20000, 15000, 2000)
    16. print(f"Cost per task: {cost:.4f} CNY")
    17. print(f"Cost per 1,000 tasks: {cost * 1000:.2f} CNY")
    18. print(f"Tasks possible with a 1,000 CNY quota: {int(1000 / cost)}")

    The output is a cost per task of 0.1450 CNY and a cost of 145.00 CNY per 1,000 tasks. In this example, cache hits account for 75% of input tokens, but because their unit price is low, their impact on total cost is much smaller than that of output. Although output tokens are only 2,000, they contribute about 55% of the cost. This indicates that output control should be the first optimization target.

    4. Engineering Factors That Affect Task Cost

    Context bloat
    In multi-turn conversations, input tokens keep growing if history is not trimmed or summarized. Each turn costs more than the previous one, and the total cost of a long session may far exceed expectations.

    Prompt redundancy
    Repeated rule descriptions, overly long examples, and uncleaned debugging information all increase input. Prompts should be simplified as needed, keeping only the content required for the model to complete the task.

    Failed retries
    Retries caused by request timeouts or output formats that do not meet requirements can multiply the cost of the same task. Retry counts should be capped, and retry reasons should be logged so they can be fixed specifically.

    Tool-call loops
    When the model repeatedly calls multiple tools, each call result is fed back into the context. A maximum number of steps and termination conditions should be set for loops.

    Tiered model use
    Simple tasks such as classification, extraction, and format conversion usually do not require the strongest model. Routing tasks to models of different specifications based on difficulty can significantly reduce total cost while maintaining quality.

    5. Practical Ways to Reduce Costs

    1. Stabilize the request prefix: Put the system prompt and fixed examples at the beginning of the request and dynamic content at the end to improve cache hit rates.
    2. Compress conversation history: After a certain number of turns, replace the original conversation record with a summary.
    3. Limit output length: Control response length through clear output requirements and length limits, and prefer structured formats such as JSON fields or fixed templates.
    4. Reuse results: For identical or highly similar requests, cache results at the application layer to avoid repeated calls.
    5. Batch processing: For tasks that do not need real-time responses, combine them into batches to reduce the number of calls and connection overhead.
    6. Establish cost monitoring: Record token usage and cost at the request layer, break down statistics by feature module, user, or tenant, and review them regularly.

    6. Selection Evaluation Checklist

    When selecting a service or planning a budget, confirm the following questions one by one:

    • Is the comparison based on cost per task rather than unit price?
    • Does the task sample cover the difficulty distribution of real business scenarios?
    • Are input, output, and cache hits priced and counted separately?
    • Is the quota unit consistent with the actual billing basis?
    • Are retries, failed requests, and tool calls included in usage?
    • Will peak-period rate limiting affect business delivery?
    • Are there tasks that can be moved to lower-cost models?

    7. Conclusion

    The cost of LLM services is determined by input, output, caching, and hidden usage together. To judge whether a solution is cost-effective, the key is to estimate cost per task using real task samples and then combine quotas and business volume to make projections. On this basis, teams can continuously optimize through stable prefixes, compressed history, output control, and tiered routing to keep costs within a predictable range.