Behind the Price Hike: Cache Pricing Logic and Cost Estimation for LLM APIs
This article dissects the recent API price adjustments from a major LLM provider, revealing that the increase is not uniform but heavily favors cached input tokens. It explains the technical foundation—prefix caching with disaggregated inference—that justifies this pricing strategy, then provides a simple mathematical model to compare real costs for enterprise agents versus low-frequency developers. Practical optimization tips for developers are included, along with a reusable cost formula.
Price Table: The Increase Is Not Even
A recent generation of APIs from a major model provider has seen notable price increases, sparking widespread discussion in the developer community. Let’s set aside the emotional reaction and break down the price tiers.
Using the previous generation’s coding-specific model as a baseline, the new generation roughly changes prices as follows:
- Output: from ~27 CNY per million tokens to ~100 CNY per million tokens—almost a 4x increase.
- Input (cache miss): from ~6.5 CNY to ~20 CNY—about a 3x increase.
- Input (cache hit): from ~1.1 CNY to ~2 CNY—less than 2x, the most moderate increase.
More telling is the shift in price ratios:
- In the previous generation, the cache-miss input price was about 6 times the cache-hit input price; the new generation widens that gap to 10 times.
- The output-to-cache-hit-input ratio jumps from about 25x to 50x.
In other words, this is not a simple across-the-board price hike. It is a deliberate re-widening of the gaps between tiers. The cache-hit tier is intentionally kept low, while the output tier is drastically raised. This is a classic price discrimination design—using the price structure, not a uniform discount, to separate different types of customers.
Why This Pricing Works: Disaggregated Inference and High Cache Hit Rates
To understand the pricing strategy, you need the technical backdrop: KV cache reuse in long-context scenarios.
In programming-agent use cases, a single conversation often carries an extremely long prefix—system prompts, tool definitions, repository context snippets, and so on. These prefixes are highly repetitive across multiple turns. If the inference system employs prefix caching with a disaggregated inference architecture (separating prefill and decode onto different compute resources), repeated contexts are computed only once. Subsequent requests hit the cache directly, saving substantial expensive prefill compute.
According to publicly available technical materials, such architectures can achieve cache hit rates above 90% in real programming scenarios. This is the foundation of the pricing confidence:
- For the provider, a cache-hit request has extremely low marginal compute cost—even at 2 CNY per million tokens, there is ample profit.
- For enterprise customers, as long as the workload is “fixed system prompt + high-frequency context reuse,” the effective input price approaches the hit price.
- The output side carries the main revenue burden, since decode phases cannot be amortized by caching and represent real compute consumption.
This is not arbitrary price inflation; it is a precise alignment between price structure and the cost structure of the inference architecture.
A Mathematical Model: How Much Different Users Actually Pay
Listed prices are not equal to effective costs. Let’s calculate with two typical scenarios.
Scenario 1: Enterprise-Level Coding Agent
Assumptions: input-to-output ratio 20:1 (common for long-context coding agents), cache hit rate 90%.
- Effective input price ≈ 2 × 0.9 + 20 × 0.1 = 3.8 CNY per million tokens
- Output: 100 CNY per million tokens
- Combined cost per million tokens ≈ (20 × 3.8 + 1 × 100) / 21 ≈ 8.4 CNY
Scenario 2: Low-Frequency Independent Developer
Assumptions: each prompt is different, cache hit rate only 20%, input-to-output ratio 5:1.
- Effective input price ≈ 2 × 0.2 + 20 × 0.8 = 16.4 CNY per million tokens
- Output: 100 CNY per million tokens
- Combined cost per million tokens ≈ (5 × 16.4 + 1 × 100) / 6 ≈ 30.3 CNY
Conclusion
Using the same model and the same price table, an enterprise customer’s effective unit cost is roughly one-third or less than that of an independent developer. On the surface, it is a “general price increase,” but in reality:
- Enterprise users with heavy prefix reuse receive an implicit discount;
- Developers with scattered prompts pay the full premium.
You can verify this with a simple code snippet:
def cost_per_mtok(hit_rate, io_ratio,in_hit=2, in_miss=20, out=100):"""Calculate effective cost per million tokens (in CNY)"""eff_in = in_hit * hit_rate + in_miss * (1 - hit_rate)return (io_ratio * eff_in + out) / (io_ratio + 1)# Enterprise scenarioprint(cost_per_mtok(0.9, 20)) # ≈ 8.4# Independent developer scenarioprint(cost_per_mtok(0.2, 5)) # ≈ 30.3
You can reuse this formula to compare price tables from any other model provider.
Practical Cost Optimization Tips for Developers
Now that you understand the pricing logic, the optimization direction is clear. The core principle is: do everything possible to increase cache hit rate, and control output token usage.
- Stabilize the prefix: Place system prompts and tool definitions at the very beginning of the prompt and keep them byte-identical. Never put timestamps, random IDs, or other variable data in the prefix—move them to a variable region at the end.
- Batch and centralize calls: Send similar requests together to avoid cache invalidation caused by frequent session context switches. For batch jobs, use fixed templates.
- Control output length: Output is the most expensive resource. Use structured output constraints (e.g., request concise JSON), set a reasonable
max_tokens, and avoid asking the model to rephrase the context. This directly cuts the largest cost component. - Use tiered model routing: Route simple classification or rewriting tasks to cheaper lightweight models. Reserve the flagship model only for requests that truly require strong reasoning.
- Continuously monitor: Set up logging and monitoring dashboards (e.g., with standard cloud observability services) to track cache hit rates and token consumption by business line. Use the data to validate optimization efforts.
Final Thoughts
This price adjustment is a signal that LLM API pricing is moving from coarse per-token billing to fine-grained tiered pricing that aligns with the cost structure of inference systems. For developers, complaining about listed prices is pointless. The right approach is to read the architectural logic behind the price structure, estimate your real cost with a simple model, and then engineer your application to land in the low-cost tier. When evaluating any model’s API in the future, do not just look at the sticker price—ask: “Given my cache hit rate and input/output ratio, how much am I actually paying per million tokens?”
