Why LLM API Prices Doubled: Billing Logic and Cost-Based Model Selection

Recently, a leading vendor’s new flagship model API pricing drew widespread discussion after a significant increase over the previous generation: published rates came in at roughly ¥100 per million output tokens, ¥20 per million input tokens, and ¥1 per million cache-hit tokens. Many developers’ first reaction was “that’s expensive,” but three more useful questions deserve attention: What billing structure sits behind these numbers? Why is output 5× the price of input? And why can cache hits be as cheap as one-twentieth of the input rate? This article breaks down LLM API billing from the perspective of inference compute cost, and offers a selection framework for subscription versus pay-as-you-go plans, along with engineering practices for cost reduction.

1. Reading the Pricing Table: Three-Tier Pricing for Input, Output, and Cache

1.1 Tokens Are the Basic Billing Unit

Nearly all LLM APIs meter usage in tokens rather than characters or request counts. For Chinese text, one token typically corresponds to roughly 0.6–1.5 characters, depending on the tokenizer implementation. In industry practice, a single request is generally split into three billable items:

• Input tokens: everything you send to the model, including system prompts, historical context, tool descriptions, and user input;
• Output tokens: everything the model generates, including reasoning traces and the final answer;
• Cache-hit tokens: the portion whose prefix exactly matches a previous request and whose computation can be reused.

1.2 Why Output Costs More Than Input

This is not a pricing tactic—it is a direct mapping of the inference cost structure. LLM inference has two stages with very different compute profiles:

• The prefill stage processes the input. The entire input sequence can be computed in parallel, completing in a single forward pass. GPU matrix operations achieve high compute utilization, so per-token cost is relatively low;
• The decode stage generates output. This stage is strictly autoregressive—each generated token requires reading the KV cache of all previous tokens, shifting the bottleneck from compute to memory bandwidth. Per-token cost is significantly higher than prefill.

There is an intuitive engineering rule of thumb: effective compute utilization per token in the decode stage is often only a fraction of that in prefill. As a result, “output priced at 3–5× input” is an industry-wide phenomenon, and a 5:1 ratio sits at the upper end of the common range, typically corresponding to flagship-tier models with more activation parameters and higher output quality.

1.3 Why Cache Hits Can Be 20× Cheaper

The principle behind context caching (prefix caching) is that a Transformer’s computation over the input (the KV cache) depends only on the input content itself. If the prefixes of two requests are identical (for example, the same system prompt and the same code repository summary), the second request does not need to recompute that portion of prefill—it can reuse the existing KV directly.

For vendors, the marginal cost of a cache hit is essentially just storage and hit validation, so pricing can be pushed down to 5%–10% of the original input price. For users, this is the most worthwhile “hidden discount” to actively cultivate in the pricing table.

Billing Item	Typical Relative Unit Price	Underlying Compute Cost
Input	1×	One prefill forward pass, high parallelism
Output	3–5×	Autoregressive decode, memory-bandwidth bound
Cache hit	0.05–0.1×	Prefill avoided, only storage and validation

2. Is the Price Increase Justified? What Makes Up Flagship Model Unit Cost

2.1 “Total Parameters” vs. “Activated Parameters” Under MoE

Today’s flagship models mostly adopt a Mixture-of-Experts (MoE) architecture: total parameter count can reach the trillion scale, but only a small fraction in the tens of billions is activated per token. Activated parameters determine per-token compute; total parameters determine memory footprint. Together they push costs up. Unit inference price can be roughly understood as:

Unit price ≈ (compute cost + memory cost + R&D and depreciation amortization + target gross margin) ÷ effective token throughput

Flagship models typically push both activation parameters and context length beyond the previous generation at the same time, so rising unit cost is structural.

2.2 The Hidden Cost of Long Context

KV cache memory grows linearly with context length, and attention computation grows quadratically with length (though the industry has developed optimizations targeting attention memory efficiency). Processing a 256K-context request costs far more than 32× an 8K request. This is also why some billing schemes charge a surcharge for very long contexts. When selecting a model, always confirm: does the advertised “up to 1M context” come with a separate pricing tier?

2.3 Beyond Unit Price, Per-Task Usage Is Also Rising

Stronger coding and agent capabilities often mean longer outputs (chain-of-thought and tool-calling traces are billed too), more interaction rounds, and larger tool-description inputs. To evaluate cost, look at the product of two factors—unit price × usage. Staring only at unit price leads to misjudgment.

3. Subscription Plans vs. Pay-as-You-Go APIs: The Economics of Two Models

For developer scenarios such as AI coding, mainstream vendors generally offer two access paths: subscription plans (a fixed monthly fee in exchange for quota) and pay-as-you-go APIs (pay for what you use). Drawing on the common forms of model services on cloud platforms—which support both post-paid usage and prepaid resource packages—the essential differences are as follows.

3.1 Subscriptions: The Price of Wholesale Is Conversion Rules

The surface appeal of a subscription is cheapness; the essence is trading “committed usage” for a “wholesale unit price.” Before committing, clarify three things:

1. Quota unit: Is it tokens, credits, or request counts? If credits, the deduction multiplier across models can vary enormously—one credit on a flagship tier may convert to only a few dozen tokens, while a lightweight tier may convert to over a thousand;
2. Whether input and output are priced the same: Many plans weight output tokens at 2–4× when converting quota, aligning with the pay-as-you-go price structure;
3. Rate and peak limits: Quota is a total-volume concept, and plans often layer on concurrency caps and per-minute request/token limits, which high-concurrency agent workloads can easily hit.

3.2 Pay-as-You-Go APIs: The Price of Transparency Is No Discount

Pay-as-you-go billing is clear and can be precisely attributed to projects and tenants, making it suitable for teams with volatile usage or a need for cost allocation and accounting. Its downside is the absence of volume discounts—at high frequency and heavy usage, cost is significantly higher than a subscription.

3.3 Break-Even Point

Let monthly input be I, monthly output be O, and monthly cache hits be C (all in millions of tokens). Then:

Monthly pay-as-you-go cost ≈ I×p_in + O×p_out + C×p_cache

When this value exceeds the plan’s monthly fee, and usage does not hit quota ceilings or frequently trigger rate limits, a subscription is more economical; otherwise pay-as-you-go wins. As a rule of thumb: light usage favors the flexibility of pay-as-you-go; heavy users running hundreds of completions or agent calls per day can typically cut costs substantially with a subscription; teams are better off with a hybrid strategy—baseline load on prepaid resource packages, overflow and bursts on pay-as-you-go.

4. A Reusable Cost-Estimation Script

def monthly_cost(input_m: float, output_m: float, cache_ratio: float,
                 p_in: float, p_out: float, p_cache: float) -> float:
    """Units: millions of tokens / currency. cache_ratio is the share of input that hits cache."""
    fresh_in = input_m * (1 - cache_ratio) * p_in
    cached   = input_m * cache_ratio * p_cache
    out      = output_m * p_out
    return round(fresh_in + cached + out, 2)
# Example: 300M monthly input, 60M monthly output, unit prices input 20 / output 100 / cache 1 (per million tokens)
print(monthly_cost(300, 60, 0.0, 20, 100, 1))   # no cache: 12000.0
print(monthly_cost(300, 60, 0.6, 20, 100, 1))   # 60% hit rate: 8580.0

This model makes three decision variables explicit: compress input, compress output, and raise hit rate—corresponding directly to the three categories of optimization below. It can also be used for subscription selection: simply compare the estimated pay-as-you-go cost against the plan’s monthly fee.

5. Engineering Cost Reduction: Five Levers That Don’t Sacrifice Quality

5.1 Compress Input: Context Engineering

Trim system prompts and tool descriptions (in agent scenarios, tool definitions often dominate input; streamlining can save 20–40% of input); in multi-turn conversations, summarize early history rather than carrying it verbatim; in coding scenarios, retrieve relevant file snippets first instead of stuffing the entire repository into context.

5.2 Raise Hit Rate: Cache-Friendly Request Structure

Caching matches by prefix, so place stable content (system prompts, tool definitions, project conventions) at the very front of the request, and volatile content (user questions, real-time data) at the end. Never insert timestamps, random numbers, or other “cache poison” into the stable prefix. At an input price of ¥20, if 60% of input hits cache, the effective input price drops to about ¥8.6 per million tokens—nearly 60% savings on the input side.

5.3 Compress Output: Make the Model “Say Less”

Explicitly constrain output format, require structured responses, and prohibit restating the question; set a reasonable output length cap to prevent runaway generation; enable deep reasoning chains only for tasks that need them.

5.4 Model Routing: Tiered Dispatch

Use lightweight models for simple tasks (format conversion, small edits), and call flagship models only for complex tasks (cross-file refactoring, architecture design). Given the real-world distribution where simple tasks dominate, mixed routing yields considerable savings versus routing everything to a flagship.

5.5 Observability and Budget Control

Treat token usage as a first-class metric and feed it into logging, monitoring, and alerting. Set budget thresholds and alert rules for each project and tenant. General-purpose log services and monitoring/alerting products on cloud platforms can directly serve these observability needs, acting as the last line of defense against “end-of-month bill shock.”

6. Conclusion: Three Judgments

First, structurally, “output 5× input and cache 20× cheaper” is a faithful reflection of inference cost structure, not simply aggressive pricing. Second, in terms of value, cost should be measured by “cost per task” rather than “cost per token”—if capability upgrades reduce interaction rounds and rework rates, a higher token price does not necessarily raise total cost. Third, strategically, heavy users should carefully calculate the break-even between subscriptions and pay-as-you-go, and systematically apply context engineering, cache optimization, and model routing on the engineering side. In a period of rising prices, what truly separates teams is not whether they can afford a flagship, but who achieves a lower unit cost for the same task.