K3 Pricing Explained: How Cache Hits and Output Tokens Reshape Inference Costs

    A detailed breakdown of K3's tiered pricing changes compared to K2.7, why cache-hit input is cheap while output tokens are expensive, and how developers can recalculate API budgets using cache hit rates and output length controls. Includes worked cost examples and sensitivity analysis. (Prices cited are from the original material and not verified against official pricing pages; treat them as illustrative.)

    Data Notes and Methodology

    The price figures in this article are taken from the original source material and have not been verified against the vendor’s official pricing page. They are used purely as calculation examples. Always check the vendor’s latest pricing before making decisions.

    Throughout this article:

    • Prices are in CNY per million tokens.
    • Increase = (K3 price − K2.7 price) / K2.7 price
    • Multiple = K3 price / K2.7 price

    How the Three Pricing Tiers Changed

    Billing tier K2.7 (CNY/M tokens) K3 (CNY/M tokens) Multiple Increase
    Input (cache miss) 6.5 20 ~3.1x ~208%
    Input (cache hit) 1.1 2 ~1.8x ~82%
    Output 27 100 ~3.7x ~270%

    Output saw the largest increase; cache-hit input the smallest. The gap between tiers has widened.

    The ratio changes reveal the structural shift even more clearly:

    • Cache-miss input vs. cache-hit input: from roughly 5.9x to 10x.
    • Output vs. cache-hit input: from roughly 24.5x to 50x.
    • Output vs. cache-miss input: from roughly 4.2x to 5x.

    These numbers suggest the vendor focused on widening the gap between tiers rather than raising prices uniformly. The pricing signal may be nudging developers to reduce repeated cache-miss input and keep output length in check. That said, numbers alone can’t confirm intent—it could simply reflect the vendor’s cost accounting.

    Why Cache-Hit Input Is Cheap and Output Is Expensive

    Large model inference runs in two phases. The prefill phase processes all input tokens at once and builds the KV Cache. The decode phase generates output tokens one at a time—each new token requires a forward pass and a read of the KV Cache for all preceding tokens.

    A cache hit means the server has already stored the KV Cache for that prefix, so prefill computation for those tokens can be skipped and the main cost becomes reading the cache. The cache may live in GPU memory, main memory, or lower storage tiers; read cost depends on the specific tiering scheme. That’s why cache-hit input is significantly cheaper than cache-miss input.

    Output tokens cost more because decoding is inherently serial. Each token depends on the previous one, so unlike prefill it can’t be batched and parallelized at scale—and every step consumes compute resources and reads the cache. The more tokens generated, the longer the resources are occupied.

    Context length also affects decode cost. The longer the context, the more KV Cache each decode step must read, increasing per-step memory bandwidth and attention computation. This is a change in resource consumption. API output pricing, however, is still billed purely by output token count—it doesn’t automatically scale with input length. Keep the two separate in your cost models.

    Cache Strategy: When 80% Hit Rate Is a Reasonable Target

    Using K3 pricing, let h be the cache hit rate. The effective input price becomes: 20 × (1 − h) + 2 × h = 20 − 18h.

    • 50% hit rate: effective price of 11 CNY/M tokens.
    • 70% hit rate: effective price of 7.4 CNY/M tokens.
    • 80% hit rate: effective price of 5.6 CNY/M tokens—roughly 72% savings versus fully missing cache on all input.

    80% is only a sensible optimization target when all of the following hold:

    1. Requests share substantial repeated prefixes, such as system prompts, tool definitions, few-shot examples, and stable conversation history.
    2. Cache lifetime is manageable—the platform’s expiration window and prefix stability match your business needs.
    3. If cache writes or storage incur extra fees, the input savings are enough to cover them.

    If each request injects different user data at the beginning of the prompt, the prefix can’t be reliably reused and hit rates will stay low. In that case, a more effective approach is to put static content first and dynamic content last, and remove history messages irrelevant to the current task.

    Prefill warm-up also depends on platform support. Some platforms may not offer it, may bill warm-up requests separately, or may expire caches after a set time. Validate with low traffic first—confirm the hit rate improvement and cost impact before rolling out broadly.

    Controlling Output Length

    With output prices up, the payoff from controlling generation length has grown substantially. Common techniques include:

    • Setting a sensible max_tokens cap and post-processing truncated results.
    • Using structured output such as JSON Mode or function calling to cut filler like pleasantries and explanations.
    • Breaking long tasks into multiple short steps with intermediate results stored externally, avoiding one long generation.
    • Generating a summary or key points first, then expanding on demand only when needed.

    Structured output has costs too. The schema itself consumes input tokens and may increase the model’s generation burden. Whether to use it should be decided based on measured token counts and output quality.

    Per-request costs look small. For a code-generation task producing 5,000 output tokens, output cost is roughly 0.135 CNY under K2.7 versus about 0.5 CNY under K3. At scale, the gap grows quickly.

    Request Design: Avoiding Large Cache-Miss Contexts

    With cache-miss input now weighted more heavily, dumping entire long documents into a single request has become costlier. At K3 pricing, a 100K-token document that misses the cache costs about 2 CNY per request in input alone. If you first use vector retrieval or keyword filtering to extract roughly 3,000 tokens of relevant content, input cost drops to about 0.06 CNY.

    Retrieved snippets should also be kept as stable as possible. Reusing the same snippets across requests raises cache hit rates. Note that retrieval itself carries cost and latency, and should be measured alongside model calls.

    Budget Recalculation: A Complete Worked Example

    Assumptions: 1 million requests per day, average input of 5,000 tokens, 70% cache hit rate, average output of 500 tokens.

    Total input: 1M × 5,000 = 5 billion tokens, i.e., 5,000 million tokens. Total output: 1M × 500 = 500 million tokens, i.e., 500 million tokens.

    Hit-rate-weighted effective input prices:

    • K2.7: 0.3 × 6.5 + 0.7 × 1.1 = 2.72 CNY/M tokens.
    • K3: 0.3 × 20 + 0.7 × 2 = 7.4 CNY/M tokens.

    Daily costs:

    Item K2.7 (CNY) K3 (CNY)
    Input cost 5,000 × 2.72 = 13,600 5,000 × 7.4 = 37,000
    Output cost 500 × 27 = 13,500 500 × 100 = 50,000
    Total 27,100 87,000

    Daily cost rises to about 3.2x the original, an increase of roughly 221%. Of the 59,900 CNY added, output accounts for 36,500 CNY, or about 61%. Output’s share of total cost climbs from about 50% to about 57.5%.

    A sensitivity analysis shows the impact of two variables:

    • Raising the hit rate from 70% to 80% lowers K3’s effective input price to 5.6 CNY/M tokens, cutting daily input cost by about 9,000 CNY.
    • Reducing average output from 500 to 400 tokens cuts daily output cost by about 10,000 CNY.

    Your budget model should therefore be driven by output volume, with cache hit rate as a core variable. The general effective-cost formula is:

    Effective cost = Total input (M tokens) × [miss ratio × miss price + hit ratio × hit price] + Total output (M tokens) × output price

    Tier-based pricing is one way vendors align prices more closely with actual resource consumption. Going forward, comparing per-million-token prices alone is no longer enough. An application’s output share, cache hit rate, and request structure all significantly affect final cost.

    Output-heavy applications such as code generation and long-form writing are more exposed to the output price increase. Applications with low output share and heavy repeated context—classification, extraction, and structured understanding—may fare relatively better under this pricing structure.

    This price divergence may also drive more research and engineering around decode optimization, such as speculative decoding and early stopping, aimed at reducing the number of tokens actually generated while preserving output quality. Cache-friendly architecture—tiered KV Cache, context segmentation management, and the like—will also get more attention.

    Risk Notes

    • Cache rules can change with platform versions, including hit determination, expiration times, and billing methods.
    • Cache hits usually require an exact prefix match; changing even a single character can cause a miss.
    • Features such as prefill warm-up and cache storage may incur extra fees or be unsupported on some platforms.
    • The examples above assume particular hit rates and average lengths; real business costs should be validated with actual usage data.