LLM Price Increases: How to Calculate Real Costs with Cache Hit Rates
When an LLM provider raises prices, the headline increase on a single billing tier rarely matches the actual impact on your bill. This article explains how to break down uncached input, cache-hit input, and output costs; build reusable weighted-cost formulas based on cache hit rate and input-output ratio; and compare how high-reuse long-context workloads differ from low-reuse short interactions. It also covers why high caching can only soften—not cancel—a price increase, how prefix caching works and its limits, engineering methods to improve cache reuse, how to trim output costs without sacrificing correctness, and why the final metric should be cost per successful task rather than cost per token.
1. Before Discussing a Price Increase, Align the Basis of Comparison
When a model provider adjusts pricing, the most common mistake is treating the increase on one billing tier as the increase in overall usage cost.
In practice, a single model call typically involves three categories of charges:
- Uncached input: the portion of the request input that the billing system does not recognize as reusable cache.
- Cache-hit input: the portion of input that meets cache reuse conditions and is billed at the cache-hit tier.
- Model output: the generated output billed under the output pricing rules.
Different workloads have very different proportions across these three categories. Even under the same price list, a long-context, multi-turn coding task and an occasional short question-answering request can produce completely different cost structures.
Therefore, deciding whether a price increase is acceptable requires answering at least three questions:
First, how much did each billing tier actually increase? Second, how much uncached input, cache-hit input, and output does the workload actually consume? Third, has the number of calls required to complete a qualified task changed?
Only by separating these three questions can you avoid conflating price, usage, and effectiveness.
2. Price Comparisons Must Be Precise—Avoid Vague Claims Like “Several Times Higher”
The following analysis uses a set of hypothetical prices solely to illustrate the calculation method. They do not represent the current pricing of any specific service. All prices are in “yuan per million tokens.”
| Billing Item | Before | After | Ratio (After / Before) |
|---|---|---|---|
| Uncached input | 6.5 | 20 | ~3.08x |
| Cache-hit input | 1.1 | 2 | ~1.82x |
| Output | 27 | 100 | ~3.70x |
It is important to distinguish “becomes X times the original” from “increases by X percent.” Output rising from 27 to 100 means it becomes about 3.70 times the original price, which corresponds to an increase of about 270%—not an increase of 370%.
Structurally, this hypothetical price list also reflects two changes:
First, the gap between uncached input and cache-hit input widened. Before the adjustment, the price ratio was about 5.91; after, it is 10.
Second, output became more expensive relative to cached input. Before, the ratio of output to cache-hit input price was about 24.55; after, it is 50.
This means cache reuse and output volume now have a more pronounced impact on the bill. However, it does not prove that high-cache workloads are “cheaper than before,” because the absolute prices of all three tiers increased.
Two concepts must be kept separate: being cheaper relative to low-cache workloads is not the same as being cheaper relative to historical prices.
3. Building a Reusable Cost Formula
Define:
- Total input as (I);
- Total output as (O);
- Cache hit ratio by input token volume as (h);
- Uncached input unit price as (P_u);
- Cache-hit input unit price as (P_c);
- Output unit price as (P_o).
Input and output quantities are measured in millions of tokens, and prices are in yuan per million tokens.
The total call cost is:
[
C=I[(1-h)P_u+hP_c]+OP_o
]
The weighted unit price on the input side is:
[
P_{\text{in}}=(1-h)P_u+hP_c
]
If the input-to-output ratio is (r=I/O), the weighted cost per million total consumed tokens is:
[
P{\text{avg}}=\frac{rP{\text{in}}+P_o}{r+1}
]
This formula is suitable for comparing the cost structures of different workloads, but it cannot replace task-value evaluation. Because input tokens and output tokens serve different purposes, “cheaper per million total tokens” does not mean “cheaper per task,” let alone higher task quality.
One important detail: the hit ratio in the formula must be calculated by input token volume, not simply by “the share of requests that hit the cache.” A request with only a short matching prefix may still be counted as one cache-hit request, but the input cost it saves remains limited.
4. Why Two Types of Workloads Produce Markedly Different Unit Costs
4.1 High-Reuse Long-Context Tasks
Assume an automated coding workload with the following characteristics:
- Input-to-output ratio of 20:1;
- Cache hit ratio of 90% by input token volume;
- Using the post-adjustment hypothetical prices.
The weighted input-side unit price is:
[
P_{\text{in}}=20\times0.1+2\times0.9=3.8
]
The weighted cost per million total tokens is:
[
P_{\text{avg}}=\frac{20\times3.8+100}{21}\approx8.38
]
In this workload, the input volume is large, but most of it is billed at the lower cache-hit tier, so the blended unit price is significantly lower than the listed prices for uncached input and output.
It should be emphasized that 90% is only a calculation assumption, not a level that long-context tasks naturally achieve.
4.2 Low-Reuse Sporadic Interactions
Now assume another type of workload:
- Input-to-output ratio of 5:1;
- Cache hit ratio of 20% by input token volume;
- Using the same post-adjustment prices.
The weighted input-side unit price is:
[
P_{\text{in}}=20\times0.8+2\times0.2=16.4
]
The corresponding weighted cost is:
[
P_{\text{avg}}=\frac{5\times16.4+100}{6}\approx30.33
]
Comparing the two, the second workload’s weighted cost per million total tokens is about 3.62 times that of the first.
However, this gap is caused jointly by the cache hit ratio and the input-output ratio. It cannot be attributed entirely to caching, and it certainly does not imply that enterprise users are necessarily cheap or individual developers necessarily expensive. What truly determines the bill is the request structure, not the user’s identity.
5. High-Cache Workloads Can Still Face Significant Price Increases
To assess the impact of a price adjustment, keep the workload’s usage structure unchanged and substitute the pre- and post-adjustment prices separately.
For the workload with a 20:1 input-output ratio and 90% hit ratio, the pre-adjustment weighted input unit price is:
[
6.5\times0.1+1.1\times0.9=1.64
]
The pre-adjustment weighted cost per million total tokens is approximately:
[
\frac{20\times1.64+27}{21}\approx2.85
]
After the adjustment it is about 8.38, roughly 2.94 times the original.
For the workload with a 5:1 input-output ratio and 20% hit ratio, the pre-adjustment weighted input unit price is 5.42, and the weighted cost per million total tokens is about 9.02; after the adjustment it is about 30.33, roughly 3.36 times the original.
Therefore, the more rigorous conclusion is: high cache reuse can soften part of a price increase, but it cannot automatically offset increases across all billing tiers.
If context length, output volume, or the number of calls changes at the same time, the actual increase on the bill will deviate further from the results above.
6. Why Caching Saves Money—and Where Its Boundaries Are
When processing input, a large model typically needs to compute internal states corresponding to the context. When implementation conditions are met, reusing the computed state of an existing prefix reduces the overhead of reprocessing identical content.
But caching is not “any repeated text will hit.” Actual effectiveness is usually affected by the following factors:
- Whether the prefix is identical, including message order, content, and serialization format.
- Whether the cache is still within its validity period.
- Whether the model version and execution configuration satisfy the reuse conditions.
- Whether service scheduling can access the corresponding cache state.
- Whether cache capacity, eviction policy, and concurrent load permit reuse.
Separating the scheduling of input processing from the incremental generation phase, and improving the storage and transfer of cache states, may help improve resource utilization. However, you cannot assert that a given workload will achieve a specific hit rate merely because a certain inference architecture is used.
It is also important to distinguish three types of caching: model computation-state caching, retrieval-result caching, and application-layer caching that directly reuses answers. They serve different purposes and should not be described with a single undifferentiated hit-rate metric.
7. Engineering Methods to Improve Cache Reuse
7.1 Stable Content First, Dynamic Content Last
For mechanisms that rely on prefix reuse, place stable rules, fixed task instructions, and long-lived interface descriptions at the front, and place user questions, timestamps, and temporary execution results at the end.
A sample structure could be:
Stable prefix:Task constraintsOutput specificationsFixed interface descriptionsDynamic suffix:Current questionRetrieval content for this callExecution results for this turn
If a constantly changing timestamp or random identifier is placed at the beginning, it can destroy prefix reuse opportunities for all the stable content that follows.
7.2 Keep Serialization Output Stable
Semantically identical content does not guarantee identical underlying representation. Changes in field order, unstable list ordering, differences in whitespace handling, and unnecessary version notes can all affect reuse.
Therefore, without changing business semantics, standardize message construction, field ordering, and text formatting.
7.3 Do Not Retain Useless Context Just to Chase Hit Rates
Cache-hit input usually still incurs charges and occupies context capacity. Keeping large amounts of outdated logs, repeated instructions, and irrelevant code can increase costs and interfere with reasoning, even if it hits the cache.
The sensible order is: first remove irrelevant content, then compress what can be compressed, and finally optimize the reuse of what remains.
7.4 Enforce Isolation and Permission Boundaries
Cross-user and cross-project cache design must follow data isolation and access control requirements. Do not mix sensitive business content into shared prefixes to save money, and do not assume that server-side caching inherently provides application-layer authorization.
8. When Output Is Expensive, Optimize Expression—Not Correctness
Under the hypothetical prices above, the share of output cost grows rapidly with output length. Therefore, optimizing output usually yields direct benefits.
Possible measures include:
- Explicitly request only the necessary conclusions, change descriptions, or structured results.
- For modification tasks, prefer returning only the changes rather than regenerating complete materials.
- Avoid restating already-confirmed background in every call.
- Set reasonable output budgets for different tasks to reduce abnormally verbose responses.
- Move completeness checks into the acceptance process to prevent output truncation from triggering additional retries.
The key is not “the shorter the output the better,” but reducing ineffective output. If excessive compression omits critical steps and produces unusable results, the subsequent repair cost may exceed the savings.
9. Ultimately, Cost Should Be Measured per Successful Task
For real workloads, what matters more than per-token pricing is:
[
\text{Cost per successful task}=
\frac{\text{Total relevant cost within the measurement period}}{\text{Number of tasks passing acceptance}}
]
Relevant costs include not only the initial call, but also failed retries, automated fixes, tool execution, and necessary human review.
For example, a higher-priced model may lower total task cost if it reduces repeated attempts; conversely, if the capability improvement does not improve acceptance outcomes, a price increase may translate directly into budget pressure. Both scenarios should be validated using the same set of representative tasks, rather than judged by impression.
It is recommended to record at least the following metrics:
- Uncached input volume, cache-hit input volume, and output volume.
- Cache hit ratio weighted by token volume.
- Number of calls and retries per task.
- Task success rate, response latency, and human handover rate.
- Cost per successful task and its distribution.
Cold-start and steady-state phases should be measured separately, and short-term experiments should not be directly compared with long-term production workloads. If the service charges separately for cache storage, cache writes, or other items, those should be added to the calculation according to the actual rules.
10. Conclusion: The Price List Is the Starting Point; Your Bill Is the Answer
Judging whether an LLM price increase is reasonable cannot be done by looking only at output prices, nor by concluding “it is actually cheaper” based solely on a high cache hit rate.
A more reliable approach is: first verify each billing tier and its increase; then compute the weighted cost using your real input-output structure; and finally evaluate cost, success rate, latency, and human effort together at the task level.
Caching determines how much repeated input can save, output volume determines how high generation costs run, and task completion efficiency determines whether that spending is worthwhile. Only by combining all three can a pricing discussion be turned into a verifiable, optimizable engineering decision.
