LLM Token Pricing Explained and Cost Optimization in Practice

    Why LLM token prices vary several-fold between generations, how input, output, and cache hit pricing work, and how to choose between pay-as-you-go APIs and subscription plans and cut LLM costs. It includes a cost estimation script, optimization strategies, and a decision checklist.

    Why Token Pricing Can Produce Several-Fold Price Differences

    LLM inference billing is essentially a way of metering compute resources. Mainstream cloud providers generally bill by token rather than by request. A token is the smallest unit of text a model processes. One Chinese character usually maps to 1–2 tokens, while an English word may be split into several subword tokens.

    When prices rise to “double or even several times” those of the previous generation, it usually comes down to three factors:

    • Model capability and scale upgrades: Models with more parameters and stronger reasoning cost more per forward pass, which pushes unit prices up.
    • Input and output priced separately: Output is generated autoregressively token by token, which consumes far more compute than a one-time encoding of the input.
    • The introduction of caching: When repeated prefixes such as system prompts or long documents have their KV cache reused, unit costs can drop significantly.

    These three points are the prerequisite for judging whether a price increase is reasonable. Comparing list prices alone tends to produce the wrong conclusion. What actually determines your cost is your call structure and the composition of your bill.

    Input, Output, and Cache Hit: The Logic Behind Three Prices

    Input tokens are the prompt submitted by the user. The model performs a full encoding pass over all of them and generates a KV cache. This work can be parallelized, so the unit cost is relatively low.

    Output tokens are generated autoregressively, one token at a time. Every new token has to be computed against the entire existing context, so the work cannot be fully parallelized. As a result, the per-token cost is usually several times that of input. The commonly seen pattern of “output priced at roughly 3–5x input” is exactly this compute asymmetry at work. Output costing several times more than input is technically a general phenomenon, not an exception at any particular provider.

    Cache hit means that when multiple requests share the same prefix (a fixed system prompt, few-shot examples, or a long document), the system reuses the already computed KV cache and skips the repeated encoding. Cached tokens are often priced at a fraction of the normal input price, and sometimes lower still. This is why, under differentiated pricing structures, the cache hit price sits well below the input price.

    The code below is a simple illustration of how to estimate the actual cost of a single call:

    1. # Estimate the actual cost of a single call
    2. # price_* is the price per 1,000 tokens
    3. def estimate_cost(input_tokens, output_tokens, cache_hit_tokens,
    4. price_input, price_output, price_cache):
    5. cost = (input_tokens * price_input
    6. + output_tokens * price_output
    7. + cache_hit_tokens * price_cache) / 1000.0
    8. return round(cost, 4)
    9. # Example: a cached input + a little new input + an output
    10. total = estimate_cost(
    11. input_tokens=500,
    12. output_tokens=1200,
    13. cache_hit_tokens=4500,
    14. price_input=20, price_output=100, price_cache=1
    15. )
    16. print(total)

    The formula makes two things clear: output tokens are the main source of cost, and the cache hit rate is the biggest cost lever. When your workload involves a lot of repeated prefixes, cache hits can pull the overall bill down by an order of magnitude.

    Taking “Price Doubling” in Stride

    To judge whether a price increase is justified, avoid looking only at the unit price. Consider these dimensions instead:

    1. Your actual cache hit rate: The list price may have multiplied several times over, but if a high share of your calls hit the cache, your real unit cost increase can be far smaller than the headline number.
    2. The marginal value delivered by the new capability: If the new generation of the model clearly reduces task failure rates and cuts down manual rework, the hidden costs saved may offset the higher unit price.
    3. Total cost of ownership (TCO): Account for API spend, integration effort, operations, and rework together, instead of staring only at the API bill.
    4. Breaking it down by workload: Split your workloads into high-frequency simple tasks and low-frequency complex tasks and evaluate them separately. You will usually conclude that a tiered approach beats a one-size-fits-all one.

    Subscription Plans vs. Pay-As-You-Go APIs

    Pay-as-you-go APIs are billed in close proportion to actual usage. They fit scenarios with volatile traffic, intermittent calls, or frequent switching among multiple models. The advantages are no upfront commitment and flexibility; the drawback is that the bill can get out of hand at high volumes.

    Subscription plans (for example, the monthly and annual plans offered by various AI coding tools) exchange a fixed fee for a certain quota of calls or credits. They suit stable, high-frequency, predictable coding and development work. The advantages are predictable cost and often a lower effective per-unit price; the drawback is wasted quota when usage falls short.

    The key to choosing is a break-even analysis:

    • Estimate your monthly token consumption (tally input, output, and cache hits separately);
    • Calculate one month of pay-as-you-go cost using list prices;
    • Compare that figure against the fixed fee of the subscription option.

    When monthly pay-as-you-go cost stays above the subscription fee over time, a subscription is the better deal. Otherwise, in volatile or low-frequency scenarios, pay-as-you-go comes first. Most teams land on a hybrid strategy: a subscription for day-to-day development to lock in costs, and pay-as-you-go for burst batch jobs or one-off tasks.

    Practical Cost Optimization Strategies

    • Make good use of prompt caching: Keep stable system prompts, tool definitions, and long documents in the request prefix and reuse them, maximizing the cache hit rate.
    • Route requests by model tier: Use lightweight models for simple classification, formatting, and extraction tasks, and reserve flagship models for complex reasoning. Avoid using a sledgehammer to crack a nut.
    • Trim context: Control the length of conversation history and summarize or compress it promptly to eliminate wasted tokens.
    • Batch requests and control streaming: Merge tasks that can be processed asynchronously, and cap the maximum output length to prevent over-generation.

    Selection Decision Checklist

    1. Tally your real token mix over the past month (the share of input, output, and cached tokens);
    2. Use the formula above to work out your monthly pay-as-you-go cost baseline;
    3. Compare the quota and effective per-unit price of available subscription plans to find the break-even point;
    4. Weigh that against your cache hit rate, your model capability needs, and how stable your traffic is;
    5. Once you are live, keep monitoring the bill, review it weekly, and adjust your strategy dynamically.

    Summary

    LLM billing looks complex, but it is essentially an engineering problem that can be broken down and calculated. Hold on to two points—output is expensive, and caching is the lever—and add a break-even analysis between pay-as-you-go and subscriptions, and you can turn “price increase anxiety” into cost optimization you can actually manage. Whether you use Baidu AI Cloud or another mainstream cloud platform’s general billing capabilities, the methodology is the same: quantify your usage structure first, choose your billing model next, and then keep pushing unit costs down through caching and model tiering.