LLM API Cost Calculation After a Price Increase: How to Estimate Real Call Costs
This article explains why changes in per-token prices do not directly determine total LLM API spend. It breaks down input, output, and cache-hit tokens; compares token pricing with subscription quotas; provides a Python cost-estimation model; and outlines evaluation methods and engineering practices for measuring real cost per task.
1. Starting Point: Unit Price Changes Do Not Equal Bill Changes
LLM services are usually billed by token. When comparing models, people often start with the price per million tokens. If the output price of a new model generation rises noticeably, it is easy to conclude that usage costs will multiply. In practice, actual spending is determined by three variables: the number of tokens consumed per call, the price structure, and whether quota credits or cache reuse are available. Looking only at unit price cannot tell you which direction the total bill will move.
This is especially true for coding-assistant scenarios. Such applications often carry long contexts, including project file snippets, conversation history, and tool results. Input tokens are often far more numerous than output tokens, so input pricing and cache-hit pricing may affect total cost more than output pricing itself.
2. What the Three Billing Units Mean
1. Input tokens
This is the encoded length of all prompt content in a request, including system instructions, user messages, retrieved snippets, and conversation history. Longer context means more input tokens.
2. Output tokens
This is the encoded length of the model-generated content. Output prices are usually higher than input prices because generation requires decoding token by token, which has a higher per-unit computational cost.
3. Cache-hit tokens
When the prefix of a new request matches a recent request, the server can reuse intermediate results that have already been computed. The matched portion is billed at a lower price.
Whether caching takes effect depends on the server’s prefix-matching rules, cache retention period, and request structure. Rules vary widely across services, so official documentation and actual bills should be the reference. A common engineering practice is to place stable content such as system instructions, tool definitions, and long documents at the front of the prompt, and place variable content such as user questions and real-time data at the end, increasing the chance of prefix reuse.
3. Parameter Scale and Pricing
A larger model parameter count usually implies more inference computation. However, pricing is also affected by other factors, including inference optimization, batch scheduling efficiency, hardware utilization, market competition, and the provider’s commercial strategy.
As a result, a larger parameter count does not necessarily raise unit price, and a lower unit price does not necessarily mean a single task is cheaper. Two common cases are:
- A new model generation is larger, but architecture and inference optimization reduce compute cost per unit, so the final list price is lowered.
- A provider positions a high-end model as a more capable paid product and uses price to segment different levels of demand.
To judge whether a pricing adjustment is reasonable, you need to compare both capability improvements and changes in cost structure. Comparing parameter numbers alone often leads to unreliable conclusions.
4. Usage-Based Billing vs. Subscription Quotas
Many users do not pay directly by token. Instead, they buy monthly or periodic quotas. This creates two issues.
First, subscription prices and API list prices cannot be compared directly.
A subscription plan usually provides a quota cap. After exceeding it, usage may be rate-limited or converted to usage-based billing. How far the quota goes depends on usage intensity. High-frequency users may exhaust it quickly, while light users may have a remaining balance for a long time.
Second, quota refresh rules change the effective perceived cost.
Some services reset quotas weekly or daily. If conversation and coding features under the same account each have independent quota pools, total available usage should be calculated as the sum of the pools, not by looking at only one of them.
For these reasons, API list prices are closer to suggested retail prices. Real cost must be estimated together with actual usage, quota structure, and reset cycle.
5. Building a Cost Estimation Model
The function below estimates the total cost of a set of calls. The prices are placeholder parameters and must be replaced with the official prices of the service in use.
from dataclasses import dataclass@dataclassclass PriceTable:input_per_m: float # price per million input tokens (currency units)output_per_m: float # price per million output tokens (currency units)cache_hit_per_m: float # price per million cache-hit tokens (currency units)@dataclassclass UsageProfile:calls: int # number of callsavg_input: int # average input tokens per call (including cache hits)avg_output: int # average output tokens per callcache_hit_ratio: float # share of input tokens that hit cache, from 0 to 1def estimate_cost(price: PriceTable, usage: UsageProfile) -> float:if not 0 <= usage.cache_hit_ratio <= 1:raise ValueError("cache_hit_ratio must be between 0 and 1")total_input = usage.calls * usage.avg_inputhit_tokens = total_input * usage.cache_hit_ratiomiss_tokens = total_input - hit_tokenstotal_output = usage.calls * usage.avg_outputcost = (miss_tokens / 1_000_000 * price.input_per_m+ hit_tokens / 1_000_000 * price.cache_hit_per_m+ total_output / 1_000_000 * price.output_per_m)return round(cost, 4)
Use a set of hypothetical data to compare two pricing plans. These numbers are only for demonstrating the calculation method and do not represent any real service.
usage = UsageProfile(calls=10000, avg_input=20000, avg_output=2000, cache_hit_ratio=0.6)plan_a = PriceTable(input_per_m=2, output_per_m=8, cache_hit_per_m=0.2)plan_b = PriceTable(input_per_m=4, output_per_m=20, cache_hit_per_m=0.5)print(estimate_cost(plan_a, usage)) # 344.0print(estimate_cost(plan_b, usage)) # 780.0
The calculation is as follows. Total input is 200 million tokens, of which 60% hit cache: 120 million hit tokens and 80 million miss tokens. Total output is 20 million tokens. Plan A costs 80M × 2 + 120M × 0.2 + 20M × 8 = 344 currency units. Plan B totals 780 currency units, about 2.27 times Plan A.
The cache-hit ratio also has a large impact on the result. Lowering Plan A’s hit ratio from 0.6 to 0.2 raises the cost to 488 currency units, an increase of about 40%. This shows that when optimizing cost, improving cache-hit ratio may deliver more benefit than pushing down unit price.
6. Evaluating Mixed Billing Scenarios
When usage-based billing and subscription quotas are used together, evaluate them in the following steps:
- Collect historical usage: Record input, output, and cache-hit token counts by feature module, team, or user.
- Split billing pools: Distinguish quota pools for conversation, coding, API calls, and other usage, and track consumption and remaining balance separately.
- Calculate quota utilization: Divide actual consumption by available quota. If utilization remains low for a long time, the subscription may not be cost-effective.
- Estimate marginal cost: Calculate the cost per call after exceeding quota, to judge whether capacity expansion or plan adjustment is needed.
- Set a reassessment cycle: Model versions, prices, and quota rules change quickly, so quarterly review is recommended.
The core of this method is to convert “list price” into “cost per unit task.” For example, using “average cost to complete one code review” as a metric reflects real differences better than comparing price per million tokens.
7. Engineering-Level Cost Optimization
Trim context
Remove historical messages unrelated to the current task, and summarize or truncate long tool outputs. Fewer input tokens reduce both cost and latency.
Stabilize the prompt prefix
Fix system instructions and tool definitions at the front, and avoid inserting timestamps or random identifiers into every request, improving cache-hit ratio.
Limit output length
Control the generation cap through the maximum output length parameter, and require the model to return results in a structured format. This reduces output tokens and makes downstream parsing easier.
Use tiered routing
Route simple tasks such as format conversion and keyword extraction to lower-price models; route complex reasoning tasks to more capable models. Routing rules can be based on task type or input length.
def route_model(task_type: str, input_tokens: int) -> str:if task_type in {"format", "extract"} and input_tokens < 4000:return "small-model"return "large-model"
Set up usage monitoring
Record token consumption by feature and team, and set alert thresholds for daily usage and cost per call. Abnormal growth usually comes from prompt inflation, retry loops, or cache invalidation; catching it early helps avoid budget overruns.
8. Conclusion
Unit price changes for LLM services are only one component of cost. Real spending depends on the token structure per call, cache-hit conditions, subscription quota coverage, and quota refresh rules. Developers should compare options with a unified estimation model, use cost per unit task as the decision basis, and continuously reduce spending by trimming context, stabilizing prefixes, and using tiered routing. Pricing rules and service terms change over time, so the latest official documentation and actual bills should be used before each decision.
