LLM API Price Increases Explained: Input, Output, and Cache Costs

    A practical breakdown of what large language model API price changes actually mean: how input (cache miss vs. cache hit) and output tiers are priced, why the multiplier ratios matter more than headline numbers, and how to build an effective cost model to compare real workload expenses across scenarios.

    1. This Isn’t a Straight Price Increase — It’s a Repricing of the Cost Structure

    Developers have recently noticed clear price adjustments across large language model (LLM) APIs: on the newest generation, output prices are close to four times the previous generation, input prices are up roughly threefold, and cache-hit prices have also risen, though more modestly. Taken at face value, the numbers invite a simple conclusion: calling costs have gone up across the board.

    Once you break the billing dimensions apart, the picture is less uniform. LLM API pricing is rarely settled against a single “total price.” It is normally split into at least three tiers:

    1. Cache-miss input price: context in the user request must be fully reprocessed.
    2. Cache-hit input price: shared prefixes, system prompts, or prior content in the request can be reused.
    3. Output price: generated tokens are decoded step by step and typically cost more.

    Judging the real cost of a model call therefore requires more than one unit price. Input, output, cache hit rate, context length, and request structure all combine into an effective unit price.

    For discussion, the table below uses a set of typicalized sample prices for abstraction. They do not refer to any specific product or service.

    Billing tier Previous generation (sample) New generation (sample) Approximate increase
    Input, cache miss CNY 6.5 / million tokens CNY 20 / million tokens ~3x
    Input, cache hit CNY 1.1 / million tokens CNY 2 / million tokens ~1.8x
    Output CNY 27 / million tokens CNY 100 / million tokens ~3.7x

    The increases are not uniform across tiers. Cache-hit prices rose relatively gently, while output prices rose much more sharply. The pricing strategy is therefore not a flat hike across all tiers — it pushes callers to pay attention to cache reuse, input structure, and call efficiency.

    2. The Shift in Multipliers Signals That Cache Reuse Is Now Worth More

    What matters is not only absolute price but also the ratio between tiers.

    Under the previous-generation sample prices:

    • Cache-miss input was roughly 6x the cache-hit price.
    • Output was roughly 25x the cache-hit input price.

    Under the new-generation sample prices:

    • Cache-miss input is roughly 10x the cache-hit price.
    • Output is roughly 50x the cache-hit input price.

    The relative value of a cache hit has been amplified significantly. Put differently, when a request can hit the cache, unit input cost is heavily diluted; when it cannot, input-side cost rises noticeably.

    This design logic is especially visible in long-context scenarios. In coding agents, code review, knowledge Q&A, document analysis, and agentic workflows, requests often carry large amounts of fixed or semi-fixed content: system prompts, tool definitions, code snippets, retrieval results, conversation history, template instructions. When that content can be reused from cache, the inference service does not need to recompute the same prefix every time, which improves overall throughput and cuts redundant compute.

    The message from the new price structure is explicit: API providers are shifting cost-optimization pressure from “borne unilaterally by the provider” to “shared between callers and system architecture.”

    3. Why Long-Context Coding Agents Tend to Hit the Cache Often

    Long-context coding agents are one of the most typical high-frequency, complex-reasoning workloads. A typical coding assistant request may look like this:

    1. fixed system prompt
    2. fixed tool descriptions
    3. fixed coding conventions
    4. fixed codebase context snippets
    5. fixed few-shot examples
    6. dynamic user question
    7. dynamic retrieval results
    8. dynamic execution results

    System prompts, tool descriptions, coding conventions, and few-shot examples are usually highly stable over a period of time. With a sensible request ordering, the inference system can identify them as a reusable prefix and serve them from cache.

    A common industry approach is disaggregated inference, which separates prefill from decode into different processing paths and reuses computed results for repeated context through caching. This raises throughput in long-context scenarios without sacrificing model capability.

    Cache hits matter particularly for coding agents because the workload is rarely a single short Q&A. It is multi-turn, multi-step, long-context continuous calling. If every turn carries the same large prefix and the cache cannot be hit, input-side cost accumulates quickly.

    4. Real Cost Isn’t the Listed Price — Build an “Effective Unit Price” Model

    Once the price changes are understood, developers need a cost model that is computable, comparable, and reviewable. Effective LLM API cost is generally determined by these variables:

    • Number of input tokens
    • Number of output tokens
    • Cache hit rate
    • Cache-miss input price
    • Cache-hit input price
    • Output price

    The effective input price and blended unit price can be expressed as follows:

    1. effective input unit price = cache-hit price × cache hit rate + cache-miss price × (1 - cache hit rate)
    2. blended unit price = (input tokens × effective input unit price + output tokens × output price) / (input tokens + output tokens)

    The value of this formula is that it turns a “nominal listed price” into “real business cost.”

    Comparing list prices alone leads to misjudgment. A generation of models may show a steep input price increase, yet if the cache hit rate is high, actual input-side cost can sit far below the cache-miss price. Conversely, even a cheap cache-hit price yields real cost close to the cache-miss price if requests never hit the cache.

    5. Two Typical Scenarios: Same Model, Costs That Differ by Several Times

    Scenario A: Enterprise-Grade Long-Context Coding Agent

    Assume an enterprise coding agent workload with these characteristics:

    • Input-to-output token ratio of roughly 20:1
    • System prompts, tool descriptions, and code context are highly stable
    • Cache hit rate reaches 90%
    • New-generation sample prices: cache hit CNY 2 / million tokens, cache miss CNY 20 / million tokens, output CNY 100 / million tokens

    Effective input unit price:

    1. effective input unit price = 2 × 0.9 + 20 × 0.1
    2. = 1.8 + 2
    3. = 3.8 CNY / million tokens

    Blended unit price, assuming 20 million input tokens and 1 million output tokens:

    1. blended unit price = (20 × 3.8 + 1 × 100) / 21
    2. = (76 + 100) / 21
    3. ≈ 8.4 CNY / million tokens

    In this scenario, output is priced as high as CNY 100 / million tokens, but because input volume far exceeds output volume and the cache hit rate is high, the blended cost is still diluted substantially.

    Scenario B: Low-Frequency, Dynamic Calls

    Another scenario: a developer who calls the model occasionally, with a different prompt every time.

    • Input-to-output token ratio of roughly 5:1
    • Context varies considerably from request to request
    • Cache hit rate of only 20%
    • Same new-generation sample prices

    Effective input unit price:

    1. effective input unit price = 2 × 0.2 + 20 × 0.8
    2. = 0.4 + 16
    3. = 16.4 CNY / million tokens

    Assuming 5 million input tokens and 1 million output tokens:

    1. blended unit price = (5 × 16.4 + 1 × 100) / 6
    2. = (82 + 100) / 6
    3. ≈ 30.3 CNY / million tokens

    Comparing the two:

    • Enterprise, high-hit scenario: about CNY 8.4 / million tokens
    • Low-frequency dynamic scenario: about CNY 30.3 / million tokens
    • The latter is roughly 3.6x the former

    This is the root cause of “same model, completely different experience for different users.” The price table shows nominal prices; what is actually paid is a blended cost determined by workload characteristics.

    6. Who Is Hit Hardest by the Increase? It’s Request Structure, Not Identity

    Many people read this difference as “enterprise customers get discounts while individual developers lose out.” That framing is intuitive but not rigorous. What determines cost is not user identity but the structural characteristics of the calls.

    Workloads that more easily achieve a low effective cost usually have these traits:

    1. High input-to-output ratio: input tokens far exceed output tokens, diluting output cost.
    2. High cache hit rate: many requests share a stable prefix.
    3. Stable context structure: system prompts, tool descriptions, templates, and knowledge snippets change little.
    4. High-frequency continuous calls: the same or similar prefix is reused repeatedly.
    5. Batchable work: requests can be merged, queued, or processed asynchronously.

    Workloads more likely to face high costs usually have these traits:

    1. Highly dynamic context per request: no stable prefix to reuse.
    2. Relatively high output share: the model generates a lot of content.
    3. Variable, non-fixed input length: hard to form cache reuse.
    4. Low call frequency: even occasional repeated content rarely yields stable cache benefits.
    5. Disorganized request structure: dynamic content placed first, fixed content placed last.

    So developers should not fixate only on “output prices went up.” They also need to examine whether their request structure suits caching at all.

    7. Optimization 1: Put the Stable Prefix First, the Dynamic Content Last

    The first lever for improving cache hit rate is request structure. The basic principle:

    1. the more stable and reusable the content, the earlier it should sit in the request.
    2. the more dynamic and personalized the content, the later it should sit in the request.

    A recommended request structure, abstracted:

    1. [fixed system prompt]
    2. [fixed role definition]
    3. [fixed tool descriptions]
    4. [fixed coding conventions]
    5. [fixed few-shot examples]
    6. [fixed shared knowledge snippets]
    7. [dynamic user input]
    8. [dynamic retrieval results]
    9. [dynamic execution logs]

    The benefit is that the inference service can more easily identify the shared prefix in the request and include it in cache reuse.

    By contrast, if a request always begins with a dynamic timestamp, a random ID, ad hoc user input, or ad hoc retrieval content, even a large fixed prompt later on can leave the overall prefix changing and lower the cache hit rate.

    8. Optimization 2: Control Context Length and Avoid Ineffective Input Growth

    In long-context scenarios, input cost often far exceeds output cost, so controlling input length is a major cost lever.

    Common practices:

    1. Trim ineffective history: keep only messages strongly relevant to the current task.
    2. Compress old conversations: summarize early turns instead of carrying the full text repeatedly.
    3. Limit retrieval snippet count: inject only the code or documents most relevant to the question.
    4. Use a sliding window: retain only the most recent few turns in multi-turn conversations.
    5. Structure the context into segments: split it into stable and dynamic parts to make cache reuse easier.
    6. Avoid stacking repeated templates: nesting the same instructions multiple times bloats input tokens.

    For example, in a multi-turn coding task, if every turn passes the entire codebase, all tool descriptions, and the full message history again, input tokens climb rapidly. Even with a high cache hit rate, cost can still be significant. The more sensible approach is to keep only necessary context and summarize or drop stale content.

    9. Optimization 3: Reduce the Output Share and Avoid High-Cost Generation

    Output unit prices are typically higher than input prices, and with the new generation’s output prices rising sharply, controlling output matters even more.

    Common approaches:

    1. Ask for structured output: avoid overly long explanations.
    2. Constrain the return format: reduce repeated explanations and ineffective elaboration.
    3. Split task steps: turn one long output into several short ones.
    4. Summarize first, expand later: avoid generating very long content in a single pass.
    5. Define explicit stop conditions: cut redundant wrap-ups.
    6. Post-process to compress results: generate a short result first, then let local logic fill in the rest.

    For example, in code generation, ask the model to return only the key code snippet rather than reprinting the whole file; in document summarization, generate key points first and then decide whether to expand into a longer piece.

    10. Optimization 4: Build a Tiered Calling Strategy — Different Tasks, Different Paths

    Not every task needs the strongest, most expensive model with the longest context. Enterprise applications in particular need task tiering.

    Tasks can be divided into:

    1. Simple classification tasks: intent recognition, label classification, simple routing.
    2. Moderate generation tasks: summarization, rewriting, template filling.
    3. Complex reasoning tasks: code modification, architecture analysis, multi-step planning.

    For simple tasks, prefer a lower-latency, lower-cost inference path; reserve the higher-capability model for complex tasks. This avoids funnelling every request onto the expensive path.

    Online and offline tasks can also be handled separately. Batch analysis, log summarization, and document extraction can run as asynchronous batch jobs, which makes it easier to exploit caching, merge requests, and reduce peak pressure.

    11. Optimization 5: Build a Cost Baseline from Real Traffic, Not Just Unit Prices

    After a price adjustment, the most useful thing developers can do is not judge whether “the increase is fair” but rebuild their cost baseline.

    At minimum, track these metrics:

    Metric Description
    Cache hit rate Share of requests whose reusable prefix is served from cache
    Average input length Average input tokens per request
    Average output length Average generated tokens per request
    Input-to-output ratio Ratio of input tokens to output tokens
    Cost per successful task Effective cost required to complete a task
    Share of high-cost requests Proportion of cache-miss inputs or long outputs
    Prefix reuse rate Share of requests containing the stable prefix

    Watching only the API list price easily leads to over- or underestimating real cost. A better approach is to load-test against actual business traffic and compute cost per thousand calls, cost per task, cost per user, or effective cost per request.

    1. effective cost = (input tokens × effective input unit price + output tokens × output price) / number of successfully completed tasks

    This moves cost analysis from “unit price comparison” to “business efficiency comparison.”

    12. How Should Developers Read This Round of Price Changes?

    The core of this round is not simply “models got more expensive.” It is that LLM services are moving from coarse-grained calling to fine-grained billing. Cache hits, input structure, request reuse, and output length are becoming the key factors that drive cost.

    The impact differs by user:

    1. Enterprise applications with long context, high reuse, and high call frequency: with a high cache hit rate, real cost can be diluted significantly.
    2. Applications with dynamic prompts, low call frequency, and unstable context: cache benefits are weak, so nominal increases translate more directly into higher actual cost.
    3. Applications with a high output share: focus on controlling generation length and how tasks are decomposed.
    4. Individual developer projects: efficiency gains must come more from template reuse, context compression, and task tiering.

    Facing a price adjustment, developers should not stop at an emotional verdict. They should return to engineering practice: optimize request structure, improve cache hits, control context length, set up cost monitoring, and refactor the calling chain.

    13. Conclusion: Real Cost Depends on System Structure, Not a Single List Price

    From an engineering perspective, the LLM API price changes essentially reflect a change in the cost structure of inference services. The input, output, and cache-hit dimensions jointly determine call cost, while the input-to-output ratio, cache hit rate, and how requests are organized determine how those dimensions stack up for a given workload.

    Under the same model and the same price system, real costs across applications can differ by several times. An enterprise coding agent with high cache hits, long input, and short output can achieve a lower blended cost than a dynamic low-frequency scenario. Conversely, with a loose request structure, an unstable prefix, and overly long output, real cost can rise noticeably even if headline prices barely move.

    For developers, what is needed is not “price-hike anxiety” but a quantifiable cost-optimization method: put stable content first and dynamic content last, compress ineffective context, control output length, improve cache hits, and establish a cost baseline under real traffic. Only then can technical decisions stay sound and sustainable as API prices continue to shift.