Model API Pricing Decoded: Unit Price, Cache, and Real Cost per Task

    Understanding the true cost of LLM API calls requires more than comparing per-million-token prices. This article explains why unit price alone is misleading, breaks down the billing structure (input, output, and cached tokens), and provides a practical formula for estimating task-level costs. It also introduces the concept of effective cost per successful task, outlines a step-by-step evaluation method, and shares engineering strategies to reduce expenses without compromising quality.

    Why Unit Price Alone Is Misleading

    Model pricing is typically quoted as a cost per million tokens. While this figure tells you the price of each token, it does not reflect the total cost of completing a task. One model may consume fewer tokens to finish the same job, or require fewer retries to arrive at a correct answer. To assess cost meaningfully, you need to consider three variables together: unit price, token consumption per task, and task success rate.

    This distinction becomes especially important when a newer model launches with a higher price. If the new model completes tasks with fewer reasoning steps and fewer retries, a higher unit price does not necessarily mean higher overall cost. Conversely, a model that produces verbose output and requires multiple rounds of correction can drive up total spend quickly, even if its listed price is low.

    Breaking Down the Billing Structure

    Most LLM APIs charge based on several components.

    1. Input tokens. This includes system prompts, conversation history, retrieved document chunks, and tool results. The longer the context, the more input tokens are consumed.

    2. Output tokens. Every token generated by the model is billed. Output prices are typically higher than input prices because generation is autoregressive—each token requires a full computation pass, resulting in higher compute cost and latency.

    3. Cache-hit tokens. When the prefix of a request matches a recent request, the server can reuse previously computed intermediate results and charge a lower rate for that portion. This discount is especially effective for repetitive workloads, such as fixed system prompts, multiple rounds of Q&A over a long document, or iterative analysis of the same codebase.

    The billing formula can be expressed as:

    Total cost = (non-cached input tokens × input price) + (cached tokens × cache price) + (output tokens × output price)

    Here, non-cached input tokens equal total input tokens minus cache-hit tokens.

    Estimating Task Cost with Code

    The function below implements the formula above. The prices used are illustrative only and do not reflect any real product’s pricing—replace them with official values when calculating actual costs.

    1. def estimate_task_cost(
    2. input_tokens: int,
    3. output_tokens: int,
    4. cached_tokens: int,
    5. price_in: float, # currency unit / million input tokens
    6. price_out: float, # currency unit / million output tokens
    7. price_cache: float, # currency unit / million cached tokens
    8. ) -> float:
    9. """Estimate the cost of a single API call."""
    10. if cached_tokens > input_tokens:
    11. raise ValueError("Cached tokens cannot exceed total input tokens")
    12. uncached = input_tokens - cached_tokens
    13. cost = (
    14. uncached * price_in
    15. + cached_tokens * price_cache
    16. + output_tokens * price_out
    17. )
    18. return cost / 1_000_000
    19. # Example scenario: 50,000 input tokens, 40,000 cache hits, 3,000 output tokens
    20. cost = estimate_task_cost(
    21. input_tokens=50_000,
    22. output_tokens=3_000,
    23. cached_tokens=40_000,
    24. price_in=10,
    25. price_out=40,
    26. price_cache=1,
    27. )
    28. print(f"Cost per call: {cost:.2f}") # Output: 0.26

    In this example, output tokens account for nearly half of the total cost. Reducing output token count would noticeably lower the overall expense. This is why controlling output length is one of the most common and effective cost-reduction strategies.

    The Trade-off Between Unit Price and Usage

    If the unit price changes by a factor of k and the token consumption per task changes by a factor of m, the cost per task changes by a factor of k × m. Total cost only decreases when k × m is less than 1.

    This means that if the unit price doubles, token consumption per task must drop by more than half for the total cost to decline. If only output tokens are reduced while a large portion of input tokens misses the cache, the result can be very different. When evaluating cost, you should track input, cache hits, and output separately rather than relying on a single aggregate figure.

    Estimating Task Capacity from Plan Quotas

    Subscription plans typically provide allowances in the form of quota—expressed as a monetary amount, a number of requests, or a token allocation. Regardless of the format, you can convert it to an estimated number of completable tasks:

    Approximate tasks = plan quota ÷ average cost per task

    This conversion rests on two assumptions. First, the average cost per task should come from real workload data, not a single test run. Second, you should check whether the quota is constrained by model, time period, or concurrency limits. If the plan imposes rate limits during peak hours, the actual number of completable tasks may fall below the theoretical estimate.

    A Practical Evaluation Method

    To assess the real cost of a model or solution, follow these steps.

    1. Define a task set. Select a representative sample of tasks from your business, covering simple, moderate, and complex difficulty levels. Use at least dozens of samples to reduce variance.
    2. Run under fixed conditions. Use the same prompts, context, and tool configurations across all runs. Record input, output, and cache-hit tokens for each call.
    3. Determine success criteria. Define pass conditions for each task—for example, code passes unit tests, structured output meets the schema, or answers are consistent with reference material.
    4. Calculate effective cost. Effective cost is the average spend per successful task:

      Cost per successful task = total cost of all tasks ÷ number of successful tasks

    5. Compare and verify. Compare the effective cost across different options while also recording latency and failure modes. If results are close, investigate stability further to avoid drawing conclusions from a single fluctuation.

    Effective cost is a better reflection of real expenditure than list price. A solution with a higher unit price but a higher success rate and fewer retries may end up costing less than a cheaper alternative.

    Engineering-Level Cost Reduction

    Without changing the model itself, you can reduce costs in several ways.

    • Trim the context. Keep only the history and document chunks relevant to the current question; remove duplicated content.
    • Design a stable prefix. Place invariant content—such as system prompts and fixed examples—at the beginning of the request to improve cache hit rates.
    • Constrain output length. Use explicit format requirements, structured output, and length limits to reduce verbose generation.
    • Route by difficulty. Send simple tasks to lower-cost models and escalate complex ones to more capable models, with clear criteria for escalation.
    • Batch processing. For non-real-time tasks, combine multiple calls to reduce repeated system prompt and context overhead.

    Caveats When Evaluating

    First, third-party benchmark methodologies vary. Different organizations use different task sets, prompts, and scoring criteria. When comparing results across sources, verify the test conditions.

    Second, prices and plan policies change. Any cost conclusion should be dated and checked against the latest official announcements.

    Third, cache hit rates are highly dependent on request structure. If the prefix changes slightly between requests, caching will rarely take effect, and estimates based on theoretical discounts will overstate savings.

    Fourth, cost is only one dimension of evaluation. Latency, stability, data compliance, and operational complexity also matter when making a final decision.

    Summary

    The cost of using LLM APIs is determined by unit price, token consumption, and success rate combined. When comparing options, use the effective cost per successful task as the primary metric, track input, cache hits, and output separately, and apply engineering techniques to continuously optimize context length and output size. Only then can your conclusions reliably support budget decisions and technology selection.