画板 1

    Beyond Unit Price: How to Calculate the True Cost of an LLM Task

    4 hr ago0 views

    LLM price cuts are easy to misread if you only watch the price per million tokens. This article breaks task cost into unit price, token usage, cache hits, and success rate, shows a small estimator you can run on real request logs, and outlines how to compare plans without letting retries or unstable prefixes erase the savings.

    1. Why price competition is back

    Pricing for large language model services has shifted several times over the past two years. Periodically, a provider releases a more capable model at a lower price, and the industry starts talking about a renewed price war. Those discussions usually fixate on a single number: the price per million tokens.

    Unit price is the most visible metric, but it is not what a developer ultimately pays. The cost of one business request depends on three things: the unit price, how many tokens the task consumes, and whether repeated content hits the cache. Looking only at unit price is like looking only at the price of fuel and ignoring fuel consumption per hundred kilometers. The conclusion is easy to distort.

    2. A formula for total cost

    The cost of a single task can be described as:

    Single-task cost = Σ (token count of each type × its unit price)

    Token types include ordinary input, cache-hit input, and output. Task token usage is driven by three factors: how efficiently the model understands the task, whether the output is concise, and whether multiple calls are required.

    In real workloads, unit price, usage, and cache hit rate change at the same time, so no single metric should be evaluated in isolation.

    2.1 A lower unit price

    A drop in unit price is the easiest change to quantify. Suppose the old plan is priced at P and the new plan is 20% cheaper, or 0.8P. On price alone, each token costs one-fifth less.

    2.2 Lower token usage for the same task

    The second variable is often ignored. On the same task, a newer model may reach a conclusion in fewer reasoning steps and produce more compact output, which reduces token consumption. If usage falls from T to 0.75T, the cost of one task becomes:

    0.8P × 0.75T = 0.6PT

    Total cost falls by 40%. The drop in typical task cost can be larger than the drop in unit price, but only if usage actually decreases.

    2.3 Cheaper cache reads

    The third variable is caching. In many applications, system prompts, fixed knowledge snippets, and tool definitions are repeated on every request. If the service supports prefix caching, the repeated portion can be read at a lower price.

    Let the cache hit rate be h, and let the cache-read price be 40% of the ordinary input price (a 60% discount). The blended input price is:

    (1 − h) × P + h × 0.4P

    When h = 0.7, the blended price is 0.3P + 0.28P = 0.58P, which is 42% below the original input price. The higher the cache hit rate, the stronger the discount.

    3. Estimating single-task cost in code

    The example below estimates the cost of one task. Keep the price unit consistent with your own billing unit. The functions do not encode any specific vendor API or pricing rule.

    1. def estimate_task_cost(
    2.     input_tokens: int,
    3.     output_tokens: int,
    4.     cache_hit_ratio: float,
    5.     input_price: float,       # ordinary input price per million tokens
    6.     output_price: float,      # output price per million tokens
    7.     cache_discount: float,    # cache-read price as a fraction of ordinary input price, e.g. 0.4
    8. ) -> float:
    9.     cached_tokens = input_tokens * cache_hit_ratio
    10.     fresh_tokens = input_tokens - cached_tokens
    12.     input_cost = (
    13.         fresh_tokens * input_price
    14.         + cached_tokens * input_price * cache_discount
    15.     )
    16.     output_cost = output_tokens * output_price
    18.     return (input_cost + output_cost) / 1_000_000
    21. def compare_plans(old: dict, new: dict, samples: list[dict]) -> dict:
    22.     """Compare average single-task cost of two plans on the same real samples."""
    23.     def avg_cost(plan: dict) -> float:
    24.         costs = [
    25.             estimate_task_cost(
    26.                 s["input_tokens"],
    27.                 s["output_tokens"],
    28.                 s["cache_hit_ratio"],
    29.                 plan["input_price"],
    30.                 plan["output_price"],
    31.                 plan["cache_discount"],
    32.             )
    33.             for s in samples
    34.         ]
    35.         return sum(costs) / len(costs)
    37.     old_cost, new_cost = avg_cost(old), avg_cost(new)
    38.     return {
    39.         "old_avg": old_cost,
    40.         "new_avg": new_cost,
    41.         "saving_ratio": 1 - new_cost / old_cost,
    42.     }

    Two points matter in practice. First, samples should come from real business logs, not from idealized inputs you construct by hand. Second, record input and output tokens separately, because their prices usually differ and output is often more expensive.

    4. Practical notes for choosing and adopting a model

    4.1 Build your own benchmark sample

    Published price-cut figures are usually based on a test set chosen by the vendor. Workloads differ widely, so draw a representative set of requests from production logs. Cover simple Q&A, long-document processing, multi-turn conversation, and tool calls. For each type, record average token usage and completion rate.

    4.2 Evaluate quality and cost together

    A lower cost matters only when quality still meets the bar. If a new model is cheaper per call but fails more often, human review or retries can raise the real cost instead of lowering it. Define an acceptable cost per successful task as:

    Effective cost = single-task cost ÷ task success rate

    That metric puts quality and spend on the same scale.

    4.3 Structure prompts to raise cache hit rate

    Caching depends on a stable prefix. In practice:

    • Put fixed instructions, role settings, and tool definitions at the start of the request.
    • Put the user input that changes on every call at the end.
    • Do not insert timestamps, random IDs, or other dynamic content into the fixed prefix.
    • Monitor the cache hit rate and investigate abnormal swings promptly.

    These changes do not require a different model. They only change how requests are organized, and the cost of doing so is usually low.

    4.4 Control output length

    Output tokens are generally priced higher than input tokens. If the application only needs a structured result, state the format in the prompt and set a reasonable maximum output length. For batch work, a smaller model can screen easy cases first, and only hard samples need to go to the larger model.

    5. Risks to watch

    A low price opened by competitive cuts may not last. Vendor price changes, model version updates, and adjustments to rate-limit policy can all change the cost structure. Keep the cost model parameterized and rerun the benchmark sample after every price change instead of relying on a one-time estimate.

    A lower price also does not mean lower latency. Inference speed, concurrency quotas, and stability still need separate tests. Where user data is involved, also assess where data is processed, how long it is retained, and which compliance requirements apply. A cheaper unit price does not satisfy those requirements on its own.

    6. Closing

    A model price cut is a signal that industry efficiency is improving, but it is only one variable in the cost structure. To judge whether a cut is actually valuable, look at unit price, tokens per task, cache hit rate, and task success rate together. Put those four metrics in one estimation framework, and a team can use real workload data to decide when to migrate, how to adjust prompts, and which parts of the pipeline are still worth optimizing.

    09040904

    Top articles