# LLM Prompt Caching Pricing Explained: Input, Cache Read, and Output Tiers

1. Why LLM Calls Use Three Pricing Tiers

In traditional cloud computing, we are used to linear billing based on resource consumption: you pay for the CPU hours you use and the gigabytes of storage you consume, and the bill scales with what you actually use. Billing for large language models (LLMs) works differently. It is not purely “compute billing” — instead, a single request is split into several token categories, and each category carries a different unit price.

A typical LLM request, from the user’s input to the returned result, passes through three stages:

• Input processing: the model encodes, parses, and runs attention over the prompt the user provides. This is a one-time “cold start” computation.
• Intermediate result reuse: if multiple requests share exactly the same long prefix (for example, the same system prompt or the same knowledge-base document), caching that computed result lets subsequent requests “read it directly” instead of recomputing it.
• Output generation: the model produces the answer token by token. This stage involves the most computation, and it is the most expensive.

These three stages are what gave rise to the three-tier pricing model that the industry commonly applies:

Pricing tier	Meaning	Relative price
Input price	Uncached, freshly computed input tokens	Baseline price
Cache read price	Tokens served directly from cache, reusing already computed results	Well below input price
Output price	Tokens generated by the model	Usually the highest

Two points deserve emphasis. First, the cache read price is almost always significantly lower than the input price, because reading from cache skips the expensive attention computation and only incurs storage and retrieval costs. Second, the output price is usually the highest, because token-by-token autoregressive generation requires a continuous forward pass. This price gradient is the foundation for understanding cost optimization.

2. Quantifying the Three Prices

To make cost computable, we can write it as an explicit formula. Let:

• P_in = input price per million tokens
• P_cache = cache read price per million tokens
• P_out = output price per million tokens

The cost of a single request is then:

Cost per request = (uncached input tokens / 1,000,000) × P_in
                 + (cache-hit tokens       / 1,000,000) × P_cache
                 + (output tokens          / 1,000,000) × P_out

The point of this formula is that it turns caching from a vague optimization concept into a cost item you can plug numbers into. You no longer have to ask “how much does caching actually save?” — as long as you know the hit rate, you can calculate the bill precisely.

Here is a simple example to illustrate it. The unit prices below are illustrative values chosen for readability, not real quotes:

Illustrative prices (per million tokens):
  P_in    = 3.0
  P_cache = 0.3      # about 1/10 of the input price
  P_out   = 15.0
Assume one request:
  uncached input tokens = 500
  cache-hit tokens      = 9,500    (a long shared system prompt)
  output tokens         = 1,000

The corresponding Python calculation:

def request_cost(input_miss, cache_hit, output,
                 p_in=3.0, p_cache=0.3, p_out=15.0):
    """Estimate the cost of a single request under three-tier pricing
    (units match the unit prices)"""
    return (
        input_miss / 1_000_000 * p_in
        + cache_hit / 1_000_000 * p_cache
        + output     / 1_000_000 * p_out
    )
cost = request_cost(input_miss=500, cache_hit=9500, output=1000)
# = 500/1e6*3.0 + 9500/1e6*0.3 + 1000/1e6*15.0
# = 0.0015 + 0.00285 + 0.015 = 0.01935
print(f"Cost per request: {cost:.5f}")

Now compare that with a hypothetical no-cache scenario — recomputing all 9,500 cache-hit tokens at the input price:

# No-cache version: all 10,000 input tokens are billed at the input price
no_cache_cost = (10_000 / 1_000_000 * 3.0) + (1000 / 1_000_000 * 15.0)
# = 0.03 + 0.015 = 0.045
print(f"No-cache cost: {no_cache_cost:.5f}")

In this example, simply hitting 9,500 shared tokens reduces the input cost for that portion from 0.03 to 0.00285 — a saving of roughly 90%. That is the direct economic effect of the three-tier pricing model.

3. Cache Hit Rate: The Real Cost Lever

The calculation above leads to a key conclusion: the core variable in cost optimization is not “whether to use caching” but “how high the cache hit rate is.”

The hit rate can be defined as:

Cache hit rate = cache-hit tokens / total input tokens

Tokens that hit the cache are billed at the very low cache price, while tokens that miss are billed at the input price. So the longer and more stable you make a reusable prefix, the higher the hit rate, and the lower the bill. This also explains why caching delivers disproportionately large savings in scenarios like these:

• Long system prompts: a fixed persona setup, safety constraints, and output format requirements of several thousand tokens, resent on every call.
• Shared knowledge bases or long documents: the same product manual, codebase, or set of legal text repeatedly retrieved as context.
• Multi-turn conversations: conversation history accumulates within a session, so the context from earlier turns can be reused in later ones.
• Few-shot examples: a fixed set of examples used as a prefix and carried on every request.

4. Engineering Practices That Determine the Hit Rate

Since the hit rate is the deciding factor, the engineering question is how to “feed” it. The core principle is: make reusable content as early, as stable, and as long as possible.

1. Put stable content at the very front of the prompt

Caches are usually matched starting from the prefix of the prompt. The structure should therefore look like this:

[Stable prefix: system prompt + few-shot examples + shared knowledge]   ← as cacheable as possible
[Volatile part: the user's current question, timestamps, random IDs]     ← different on every call

If you put per-request content such as your own user ID or the current time at the very front of the prompt, you invalidate the entire prefix and drive the cache hit rate straight to zero.

2. Keep the prefix byte-for-byte identical

Cache matching often requires the prefix to be exactly the same. A single extra space, a newline, or a change in field order can break the prefix match. For templated prompts, make sure the serialization output is stable.

3. Watch the cache lifetime

Caches are not permanent. Most implementations set a time to live (TTL) for cached content. If it is not accessed again within that window, the entry expires and the next request has to be recomputed at the input price. That means high-volume, short-interval repeated requests capture the most caching benefit, while low-frequency requests separated by long gaps may repeatedly pay the cold-start cost.

5. Common Mistakes in Practice

A few traps are worth watching for when you adopt this pricing model:

• Mistake 1: assuming everything you send is billed at the input price. In reality, the portion that hits the cache is billed at a separate cache read price. The two must not be conflated.
• Mistake 2: ignoring the weight of the output price. Output tokens are often the most expensive. If you chase long context while generating excessively long answers, the output side can eat up your savings. Optimize input while also keeping output from becoming verbose.
• Mistake 3: sacrificing flexibility for the sake of caching. If the business depends heavily on a highly personalized, dynamic prefix, forcing everything into a unified prefix may actually hurt quality. Caching is a cost measure; it should not take priority over output quality.

6. Summary: Make Pricing a Design Input

LLM prompt caching uses a three-tier pricing model — uncached input, cache read, and output — which at its core is charging by the compute cost of each token class. Understanding this model matters not just for reconciling bills after the fact, but for predicting cost at the architecture design stage:

• Identify which tokens in a request are stable and reusable.
• Estimate cost with the formula uncached input × P_in + cache hit × P_cache + output × P_out.
• Raise the hit rate through stable prefixes, consistent serialization, and sensible cache lifetimes.
• Control output length at the same time, so you do not burn tokens at the most expensive tier.

Once you internalize this logic, caching stops being a black-box switch in the backend and becomes a cost lever you can calculate precisely and manage deliberately.