LLM API Cost Accounting: Why Unit Price Alone Does Not Determine Task Cost

    Published per-million-token prices are only the starting point for estimating LLM API costs. This article explains how input and cache-hit tokens, output length, retries, and multi-turn calls affect actual consumption, then shows how to define measurable tasks, calculate call and task costs, and track cost per successful task with formulas and example code.

    1. Unit Price Does Not Directly Represent Cost

    LLM API billing is usually based on a price per million tokens. The unit price a provider publishes is the starting point for calculation, but it cannot tell you how much a particular task will cost. Task cost is determined by two factors: the unit price and the number of tokens actually consumed to complete the task.

    At the same unit price, different teams can end up with very different bills. Prompt length, the level of detail in model output, the number of retries after failures, and cache hit rates all change actual consumption. For this reason, API cost evaluation should shift from “price per call” to “cost per task.”

    2. Three Billing Factors and Their Impact

    2.1 Input Tokens and Cache Hits

    Input tokens include system instructions, tool definitions, conversation history, and business data to be processed. Much of this content remains unchanged across requests, such as fixed system instructions and knowledge snippets. Most providers offer cached pricing for repeated prefixes, and the unit price for cached hits is significantly lower than that for ordinary input.

    Cache hits depend on an exact prefix match. If dynamic content such as timestamps or user IDs is placed at the beginning of the prompt, the prefix changes on every request and the cache can hardly be hit. Stable content should be placed first and dynamic content at the end. This is one of the most direct ways to reduce cost.

    2.2 Output Tokens

    Output tokens usually have a higher unit price than input tokens, and their number is determined by the length of the content generated by the model. Verbose explanations, repeated paragraphs, and unnecessary formatting markers all directly increase cost. When output unit prices are high, a change in output volume often has a more noticeable impact on total cost than an input change of the same magnitude.

    2.3 Retries and Multi-Turn Calls

    Many tasks require multiple calls—for example, first calling a tool to retrieve data and then generating an answer based on the result. Each turn is charged again for input and output tokens. Retries triggered by format validation failures or results that do not meet acceptance criteria are also billed per call. Therefore, when measuring usage, record the usage of every call rather than looking only at the final result.

    3. How to Calculate Cost per Task

    3.1 Define the Task First

    Cost comparisons must be based on the same task. A team needs to fix three things: the input sample set, the acceptance criteria, and the maximum allowed number of calls. For example, “extract five fields from a ticket; success means all fields are complete and correctly formatted” is a quantifiable task definition.

    3.2 Calculation Formulas

    The cost of a single call can be expressed as:

    Single-call cost = (uncached input tokens × input unit price + cached-hit tokens × cached unit price + output tokens × output unit price) / 1,000,000

    The cost per task is the sum of the costs of all calls for that task. A more useful metric is cost per successful task:

    Cost per successful task = total spend across all tasks / number of tasks that pass acceptance

    This metric accounts for the cost of failures and retries. A solution with a lower unit price but a lower success rate may have a higher cost per successful task.

    3.3 Example Code

    The following code aggregates costs from task logs. The pricing fields below are for demonstration only and do not represent the pricing of any real service.

    1. from dataclasses import dataclass
    2. from typing import Iterable, Optional
    3. @dataclass
    4. class Pricing:
    5. input_per_m: float # price per million uncached input tokens
    6. cached_per_m: float # price per million cached input tokens
    7. output_per_m: float # price per million output tokens
    8. def call_cost(usage: dict, p: Pricing) -> float:
    9. """Calculate the cost of a single call."""
    10. uncached = usage["uncached_input_tokens"]
    11. cached = usage["cached_input_tokens"]
    12. output = usage["output_tokens"]
    13. return (
    14. uncached * p.input_per_m
    15. + cached * p.cached_per_m
    16. + output * p.output_per_m
    17. ) / 1_000_000
    18. def task_cost(calls: Iterable[dict], p: Pricing) -> float:
    19. """Calculate the cost of one task across its calls."""
    20. return sum(call_cost(usage, p) for usage in calls)
    21. def cost_per_success(task_logs: Iterable[dict], p: Pricing) -> Optional[float]:
    22. """
    23. Calculate cost per successful task.
    24. Each task log contains its calls and whether it passed acceptance.
    25. """
    26. total_cost = 0.0
    27. successful_tasks = 0
    28. for task in task_logs:
    29. total_cost += task_cost(task["calls"], p)
    30. if task["passed"]:
    31. successful_tasks += 1
    32. if successful_tasks == 0:
    33. return None
    34. return total_cost / successful_tasks

    This code follows the formulas above and keeps failed or retried calls in the total cost, while successful_tasks counts only tasks that pass acceptance.