LLM API Pricing Explained: How Token Costs Work and How to Optimize Them

    This guide breaks down why a larger model can end up costing more per call, how token-based billing actually works, and how the three pricing layers of input, output, and cache hits shape your bill. It walks through a practical cost formula and sample estimation script, then covers five optimization strategies — cache reuse, leaner context, output length control, tiered model routing, and usage monitoring — along with a framework for evaluating models beyond headline prices.

    1. A Common Puzzle: Why Do Bigger Models Cost More per Call?

    Over the past two years, as large models have scaled from hundreds of billions of parameters toward the trillion mark, many developers and teams have looked at their API bills and asked the same question: the new generation is clearly more capable, so why is the price per token higher instead of lower — sometimes several times higher than the previous generation? At the same time, the market keeps producing the opposite case too: models with more parameters that are also cheaper.

    This apparent contradiction shows that “parameter count” and “price per call” are not in a simple linear relationship. Parameter count determines a model’s upper bound on expressive capability and its compute overhead during inference, but the final price also layers in inference architecture, commercialization strategy, market competition, and the downward curve of unit costs. To reason about it clearly, you need to drop the intuition that “more parameters always means more expensive” and break the problem down along three dimensions: billing mechanics, cost structure, and business strategy.

    2. Token Billing: The Basis of All Cost Measurement

    Mainstream LLM APIs charge per token rather than per request or per character. Understanding tokens is the first step to reading your bill correctly.

    • What a token is: a token is the smallest semantic unit the model processes. One token typically corresponds to a single Chinese character, part of an English word, or a punctuation mark. For Chinese text the rough ratio is “1 character ≈ 1–2 tokens”; for English it is close to “1 word ≈ 1 token”. The exact split is determined by the model’s own tokenizer.
    • Why per token rather than per request: request lengths vary enormously, and token-based billing keeps the rule simple — you pay for the compute you consume — which maps much more closely to actual resource usage.
    • Multimodal extensions: for inputs such as images and audio, the content is normally converted into an equivalent token count before billing.

    Once you understand this layer, the three-tier pricing structure below makes sense.

    3. Three Layers of Pricing: Input, Output, and Cache Hits

    Current API pricing is usually split into three independent dimensions, and this split is the key to understanding the “price multiplied by several times” question:

    Billing dimension What it covers Cost characteristic
    Input Everything you send to the model Relatively low unit price
    Output Everything the model generates and returns Usually the highest unit price
    Cache hit Repeated, reusable prefix content Significantly below the normal input price

    The order-of-magnitude differences between the three are often the main driver of bill volatility:

    1. Why output is the most expensive: generative output is an autoregressive, token-by-token decoding process. Every new token has to be recomputed against the entire preceding context and cannot be processed in parallel the way input can, so the compute cost of a single forward pass is higher. In many scenarios, output pricing is several times input pricing.
    2. Input grows with context: as system prompts, conversation history, and retrieval-augmented (RAG) reference material get longer and longer, input tokens accumulate quickly. Long context is not “free”.
    3. The value of cache hits: when multiple requests share an identical prefix — a fixed system prompt, or the same long document — the provider can reuse the intermediate computation for that part, and those “cache-hit” tokens are billed at a fraction of the normal input price.

    4. Rethinking “More Parameters, Higher Price”

    With the three-tier framework in place, the puzzle from the beginning becomes clear:

    • Better capability ≠ higher unit price; it means a different workload mix. Larger models tend to be used for harder tasks, and developers tend to feed in longer context and ask for more detailed output, which pushes up total token consumption and therefore the bill — but that does not mean the unit price was raised arbitrarily.
    • Unit cost and total bill are two different things. Even if a provider keeps driving down unit inference cost through technical optimization, differences in pricing strategy, R&D payback period, and target customers can still result in a higher or lower list price. The two are not necessarily linked.
    • The existence of cache-hit pricing is itself a signal of price reduction. When a model offers a cache-hit discount, it means the provider has optimized its inference architecture for that case and is passing the savings from repeated computation on to users with high-frequency, long-prefix workloads.

    So a number like “5x” is hard to judge in isolation. Without the individual input, output, and cache unit prices and your actual token distribution, you cannot say whether it is expensive — you have to go back and account for it against your own real call patterns.

    5. How to Calculate Call Costs Properly

    Instead of arguing about list price, build a quantifiable cost model. The core formula:

    1. Cost per call = input tokens × input price
    2. + output tokens × output price
    3. + cache hit tokens × cache price
    4. Monthly total = Σ(all call costs) + fixed fees

    A practical estimation script (illustrative — token counts should be measured with the official tokenizer):

    1. def estimate_call_cost(input_tokens, output_tokens, cache_hit_tokens,
    2. price_input, price_output, price_cache):
    3. """Estimate the cost of a single call using three-tier pricing.
    4. Prices are expressed per 1,000 tokens."""
    5. cost = (
    6. input_tokens / 1000 * price_input
    7. + output_tokens / 1000 * price_output
    8. + cache_hit_tokens / 1000 * price_cache
    9. )
    10. return cost
    11. # Example: a scenario where the system prompt and document hit the cache
    12. cost = estimate_call_cost(
    13. input_tokens=2000, output_tokens=800, cache_hit_tokens=1500,
    14. price_input=20, price_output=100, price_cache=1
    15. )
    16. print(f"Estimated cost per call: {cost:.4f}")

    Three things matter most when you do the math: output share (long-form generation apps produce many output tokens and are inherently more expensive), context length (long context keeps pushing input cost up), and cache hit rate (whether you are actually reusing the prefix).

    6. Five Practical Cost Optimization Strategies

    1. Reuse the cache. Put stable system prompts and fixed knowledge base documents at the front of the request so the repeated part hits the cache. For high-frequency, long-prefix workloads, this is the most direct way to cut cost.
    2. Trim the input context. Use retrieval to bring back only the most relevant chunks instead of stuffing an entire knowledge base into the context, and summarize conversation history rather than sending it back in full.
    3. Control output length. Set sensible maximum output limits for generation tasks, and use compact formats for structured output (JSON rather than verbose natural language) to avoid wasting tokens.
    4. Route calls by tier. Send simple tasks to smaller, cheaper models and escalate to larger models only for complex reasoning. Match capability to cost instead of defaulting everything to the strongest model.
    5. Monitor usage. Use monitoring, alerting, and logging services to build dashboards for token consumption, cache hit rate, and per-application cost, and set budget threshold alerts so anomalous calls surface quickly.

    7. A Framework for Evaluating Models

    When comparing models from different providers with different parameter counts, don’t judge by list price alone. Build a broader set of evaluation dimensions:

    • Unit cost: compare the three unit prices — input, output, and cache hit — rather than a single blended number.
    • Capability fit: evaluate on your own real business samples, looking at accuracy and stability on the target task rather than general leaderboard rankings.
    • Cache and long-context support: if your workload is high-frequency calls over long documents with a fixed prompt, cache reuse support will strongly influence total cost.
    • Ecosystem and engineering readiness: SDK maturity, concurrency and rate-limiting behavior, observability, and data compliance capability all affect the real cost and risk of shipping.

    8. Conclusion

    “More parameters but more expensive” is, at its core, a structural cost problem hidden behind parameter count. What actually determines your bill is the combination of the three pricing layers — input, output, and cache hit — together with your own token distribution and call patterns. Rather than being led by unit price numbers, focus on the levers you control: cost accounting, cache reuse, context trimming, and tiered routing. Once your team builds the habit of quantitative accounting and continuous monitoring, you can find the optimum that genuinely fits you — between pushing capability to the limit and holding the budget line.