LLM Inference Pricing Explained: How Context Caching Reduces Token Costs

Key Dimensions of Inference Service Pricing

The cost of LLM inference services usually can’t be summarized as a single number. When integrating a model, developers typically need to track multiple billing dimensions at once: input, output, cache hits, and cache writes. Together, these determine the final bill.

Consider a typical set of pricing data (in CNY):

Model tier	Input price	Output price	Cache hit	Cache write (5 min)	Cache write (1 h)
Standard	¥13.00 / MTok	¥68.00 / MTok	¥1.30 / MTok	¥13.00 / MTok	¥26.00 / MTok
Speed	¥26.00 / MTok	¥136.00 / MTok	¥1.30 / MTok	¥26.00 / MTok	¥52.00 / MTok
Previous-generation Standard	¥20.00 / MTok	¥100.00 / MTok	¥2.00 / MTok	¥20.00 / MTok	¥40.00 / MTok

Several key patterns stand out:

1. Output prices are far higher than input prices: Output tokens typically cost about 5x as much as input tokens. Output tokens must be generated one at a time and cannot be parallelized, so they consume more inference compute.
2. Cache hit prices are dramatically lower than normal input prices: A cache hit costs roughly one-tenth of the normal input price, showing that context caching can substantially reduce the billing cost of repeated prefixes.
3. The speed tier costs 2x the standard tier: The speed tier delivers faster responses through higher scheduling priority and shorter queue times, but at double the cost.
4. The new generation is cheaper overall than the previous one: Input price dropped from ¥20.00 to ¥13.00, output price from ¥100.00 to ¥68.00, and cache write prices were lowered in step—a clear overall reduction.

Understanding how these dimensions relate to each other is the prerequisite for cost optimization.

How Basic Token Pricing Works

Basic tokens fall into two categories: input and output. Input tokens cover everything sent to the model—system prompts, conversation history, and the user’s current question. Output tokens are the model’s generated response.

In practice, input token counts are often underestimated. A seemingly simple question, when combined with a long system prompt, multiple turns of history, and retrieved context, can easily reach thousands or even tens of thousands of tokens. Output is relatively controllable and usually makes up only a small share of a request’s total tokens.

Because output tokens carry a much higher unit price, however, applications oriented toward long-form generation—code completion, report generation, article continuation—may find that output costs become the dominant expense. When evaluating costs, don’t just look at total token volume; estimate input and output separately and multiply each by its own unit price.

Speed Tier vs. Standard Tier: When Each Makes Sense

The speed tier costs twice as much as the standard tier, and this gap holds consistently across input, output, and cache writes. Only the cache hit price is identical for both. This indicates that reading from cache involves little compute overhead, so no surcharge applies even on the speed channel.

So when does the speed tier make sense?

For real-time interactive scenarios—online Q&A, conversational coding assistants, customer service bots that need second-level responses—the speed tier’s low latency can directly determine user experience. Here the premium is justified: every second shaved off waiting time translates into measurable gains in retention and satisfaction.

For offline batch scenarios—overnight summary generation, scheduled tagging jobs, asynchronous report generation—latency doesn’t matter, and the standard tier completes the same work at lower cost. Choosing the speed tier in these cases simply doubles inference spend.

One more detail worth noting: cache hit prices are identical across both tiers. This means that when your call pattern has a high cache hit rate, the actual cost increase of the speed tier is less than 2x. The higher the hit rate, the smaller the effective price gap between the two tiers.

How Context Cache Writes and Hits Are Billed

Context caching is the key mechanism for avoiding repeated billing on long, duplicated prefixes. The principle is simple: when the prefix of a request matches an earlier request, the system skips recomputing that portion and reads the cached computation instead, running inference only on the new tokens.

Two cache actions are billed: writes and hits.

Cache writes are usually priced the same as normal input, sometimes slightly higher. In the table, the standard tier’s 5-minute cache write costs ¥13.00 / MTok—the same as the normal input price—while the 1-hour write costs ¥26.00 / MTok, twice the 5-minute price.

Why does the write price depend on time?

Because the storage lifetimes differ. A 5-minute cache entry may be evicted or expire within a short window; a 1-hour entry must remain available much longer, consuming more storage and indexing resources—which naturally costs more.

Cache hit prices are set far below normal input prices—¥1.30 / MTok in the table, about one-tenth of the input price. The pricing intent is clear: encourage developers to maximize cache hit rates and reduce the system’s overall compute consumption.

Engineering Practices for Cache Optimization

Once you understand the pricing mechanics, how do you put caching optimization into practice? Here are several general principles.

5.1 Put Stable Content First

Place the most stable content at the front of each request. System prompts, role definitions, tool definitions, and the fixed parts of a domain knowledge base should remain unchanged and stay at the beginning of every request. The more stable the prefix, the higher the chance of a cache hit.

[System prompt: fixed]
[Tool definitions: fixed]
[Domain knowledge: relatively fixed]
[Conversation history: grows over time]
[Current user input: different each time]

If you put the current user input first and the system prompt after it, the cache will miss even when the system prompt is identical—because the prefix is occupied by user input that changes every time.

5.2 Keep Dynamic Content Out of the Stable Prefix

A common mistake is embedding timestamps, random numbers, or user IDs in the prompt, for example:

The current time is 2025-01-15 14:32:08. Please answer accordingly...

This injects different content into the prefix on every call, breaking the cache hit condition. Instead, move dynamic information to the end of the request, or inject it in a non-destructive way, keeping anything that varies as late as possible.

5.3 Share a Common Prefix Across Batch Requests

In batch processing, when the same analysis instructions apply to multiple documents or user inputs, put the shared instructions first as a common prefix and append the varying documents after it. The common prefix is written to the cache once, and all subsequent batches hit it.

5.4 Choose the Write Tier Based on Call Intervals and Cost

If calls are closely spaced—for example, multi-turn exchanges in a live conversation, typically seconds to minutes apart—the 5-minute cache write is sufficient and cheaper. For sessions spanning hours or regularly repeated calls, a 1-hour cache write may be needed to keep the cache valid. Keep in mind that the 1-hour write costs twice as much as the 5-minute write; whether it pays off depends on whether the hit rate covers the higher write cost.

Cost Estimation Method and Example

Suppose an application handles 1 million requests per day, each with a stable prefix of 2,000 tokens, a dynamic portion of 200 tokens, and an average output of 300 tokens. Using the standard tier’s 5-minute cache write:

• With a 90% cache hit rate:
  • Input cost = 2000 × 0.1 × ¥13.00 + 2000 × 0.9 × ¥1.30 + 200 × ¥13.00 (per million tokens)
  • The savings from cache hits are substantial.
• With a 30% cache hit rate:
  • Most requests are billed at the full input price, leaving significant room for optimization.

Estimates like this let developers quickly judge whether it’s worth investing effort in restructuring their prompts.

Conclusion

LLM inference pricing may look complex, but it essentially revolves around three dimensions: input/output token consumption, compute speed priority, and context cache reads/writes. Understanding the price differences among these dimensions—and how they interact—is the foundation of controlling inference costs.

The table shows that the new-generation standard tier is cheaper than the previous generation on input, output, and cache writes, lowering the barrier to upgrading. The speed tier matches the standard tier on cache hit pricing, making high-hit-rate speed-tier calls more cost-effective. And the distinction between cache write TTLs reminds developers to pick the right write tier based on task intervals.

Ultimately, cost optimization isn’t about blindly chasing the cheapest tier. It’s about matching the model tier and caching strategy to your application’s latency requirements, prefix stability, and call frequency.