LLM API Pricing: Token Economics and How to Calculate the Real Cost of a Model

    Sticker prices per million tokens rarely match the bill you actually pay. This article breaks large language model pricing down into its three independent billing dimensions—input tokens, output tokens, and cache-hit tokens—explains why the decoding stage makes output inherently more expensive than input, and shows how context caching and subscription multipliers reshape the math. It closes with a vendor-neutral cost estimation function and a usage-tracking pattern, so you can price a single request, or a full agent task, from measured usage instead of guesswork.

    1. The Price Tag Is Just the Surface — the Billing Structure Is What Matters

    When people say “this model costs X per million tokens,” it is tempting to treat price as a single number. In practice, cost is driven by three independent billing dimensions:

    • Input tokens (prompt): everything the model reads — the user’s prompt, the supplied context, system instructions.
    • Output tokens (completion): what the model generates back.
    • Cached tokens (cache hits): context that gets reused and does not have to be recomputed.

    Each dimension is normally priced per million tokens, and the spread between them can be several times over, sometimes tens of times. So when you see a rate card like “input 20, output 100,” the question that actually matters is: in my workload, what share of tokens falls into each of these three buckets? Until you decompose your usage structure, any number you read off a pricing page is close to meaningless.

    2. Why Output Tokens Cost More Than Input Tokens

    This is not an arbitrary pricing decision on the vendor’s side; it follows from how the model computes. Understanding that mechanism is the prerequisite for evaluating cost rationally.

    Serving a request happens in two phases:

    1. Prefill: the input tokens are processed. Because all input tokens are known up front, they can be computed in a highly parallel pass. GPU utilization is high, so the per-token cost is low.
    2. Decode: the output tokens are generated. This is an autoregressive process — each new token depends on the sequence produced so far and must be computed one step at a time. With no parallelism available, the compute cost per output token is substantially higher than for input.

    Expressed as pseudocode, that serial dependency looks like this:

    1. generated_sequence = []
    2. for step in range(max_output_length):
    3. next_token = model_forward(input_sequence + generated_sequence) # each step depends on the previous one
    4. generated_sequence.append(next_token)
    5. if end_of_sequence_reached:
    6. break

    Since every decode step requires another full forward pass, output is inherently more expensive than input. Vendors pricing output above input are reflecting a real difference in compute consumption, not padding the number.

    3. Cache Hit Billing: The Most Overlooked Cost Lever

    In real applications, a large share of requests share a stable, repeated prefix, for example:

    • a fixed system prompt;
    • long codebase context or knowledge base documents;
    • conversation history that keeps accumulating across turns but stays unchanged for prior turns.

    Recomputing prefill for that content on every request wastes compute and inflates cost. The industry answer is context caching: the already-computed intermediate state (the key-value cache) is stored, and later requests that hit the cache reuse it directly, computing only the new portion.

    Caching typically involves two charges:

    • Cache write: establishing the cache the first time carries extra overhead, and may cost slightly more than ordinary input.
    • Cache read (cache hit): the hit rate is far below ordinary input — sometimes a fraction of the list price, occasionally less.

    The core takeaway: for long-context workloads with a highly repetitive prefix, effective use of cache hits can compress input cost to a small fraction of what it would otherwise be. This is also why a headline like “cache hits at 1” — which looks absurd next to an input price of 20 — is perfectly rational for a particular workload shape.

    4. Subscription Plans vs. Pay-As-You-Go: Look at the Multiplier, Not the Price

    Alongside per-token pay-as-you-go billing, many platforms offer subscription plans: a fixed monthly fee in exchange for a consumable allowance. When you evaluate one of these, the sticker price is not the useful metric. The metric that matters is the amplification multiplier:

    1. multiplier = total token value consumable from the plan / monthly plan fee

    A higher multiplier means the same subscription fee unlocks more real usage, and therefore better value. As an illustration (the figures below are hypothetical, in the platform’s own billing currency):

    • a plan priced at 199 that yields roughly 1200–2800 in usage value works out to a multiplier of about 6–14x;
    • a plan priced at 199 that yields roughly 2000–5000 in usage value reaches a multiplier of about 10–25x.

    For developers, the value density of a subscription plan is what makes it comparable to pay-as-you-go. A plan with a high multiplier is genuinely competitive; when the multiplier is low, pay-as-you-go may be the cheaper option. The right way to evaluate is to convert your typical workload into token usage and run it through both billing models.

    5. How to Estimate the Real Cost of a Single Task

    Rather than judging “expensive or not” by feel, build a reusable estimation model. Here is a vendor-neutral cost estimation function:

    1. def estimate_cost(input_tokens, output_tokens, cached_tokens,
    2. price_in, price_out, price_cache):
    3. """
    4. Generic token cost estimation.
    5. Args:
    6. input_tokens : total input tokens (including the cached portion)
    7. output_tokens: number of output tokens
    8. cached_tokens: how many of the input tokens hit the cache
    9. price_in/out/cache: unit prices, all expressed per million tokens
    10. in the same billing currency
    11. Returns:
    12. total cost in the billing currency
    13. """
    14. fresh_input = max(input_tokens - cached_tokens, 0) # new input that missed the cache
    15. cost_input = fresh_input / 1_000_000 * price_in
    16. cost_output = output_tokens / 1_000_000 * price_out
    17. cost_cache = cached_tokens / 1_000_000 * price_cache
    18. return cost_input + cost_output + cost_cache
    19. # Example: a long-context task
    20. total = estimate_cost(
    21. input_tokens=20_000,
    22. output_tokens=2_000,
    23. cached_tokens=15_000, # 15k of stable prefix served from cache
    24. price_in=20, price_out=100, price_cache=1,
    25. )
    26. print(f"Estimated cost per task: {total:.4f}")

    One point deserves emphasis: agentic workloads often consume far more than ordinary chat. Tool calls, retrieval, code execution, and long reasoning chains mean a single task can burn tens of times the tokens of a simple question-and-answer exchange. So when estimating cost, do not measure single prompts — measure average consumption per completed end-to-end task.

    A more business-realistic approach is to instrument real usage:

    1. class CostTracker:
    2. def __init__(self, price_in, price_out, price_cache):
    3. self.price_in = price_in
    4. self.price_out = price_out
    5. self.price_cache = price_cache
    6. self.total_cost = 0.0
    7. def record(self, usage):
    8. """usage is the usage dictionary returned by a single API call"""
    9. cost = estimate_cost(
    10. usage["input_tokens"],
    11. usage["output_tokens"],
    12. usage.get("cached_tokens", 0),
    13. self.price_in, self.price_out, self.price_cache,
    14. )
    15. self.total_cost += cost
    16. return cost

    Continuously recording real production usage and aggregating it by task type gives you a reliable cost profile instead of a guess.

    6. Practical Guidance for Selection and Cost Optimization

    Pulling the above together, a few principles that hold up in practice:

    • Select by workload shape. For short conversations with low repetition, input and output unit prices dominate. For long context with a repetitive prefix, always evaluate the cache hit price.
    • Use context caching deliberately. Front-load stable system prompts, knowledge base content, and history so the cache hit rate is as high as possible — it is the most direct way to cut cost.
    • Evaluate plans by multiplier. Convert a subscription plan into an amplification multiplier and compare it against pay-as-you-go on the same basis, rather than looking only at the monthly fee.
    • Instrument your usage. Wire up usage telemetry, monitoring, and alerts, and visualize cost per task, per user, and per scenario so abnormal consumption surfaces early.
    • Control output length. Because output carries the higher unit price, prompt design that suppresses redundant output meaningfully reduces the total bill.
    • Re-evaluate models periodically. A new model generation changes both unit prices and capability, which shifts the whole cost structure. Selection should be adjusted as your business volume and usage change.

    In short, whether a model is “expensive” has little to do with the absolute number on the pricing page. It depends on separating input, output, and cache into three dimensions, feeding in your own real usage structure, and comparing billing models through the multiplier lens. With that token economics framework and estimation method in hand, developers and businesses can move from “this feels expensive” to “this is calculated” — and make genuinely rational technology and budget decisions.