Flagship LLM API Prices Rise 5x: Understanding Token Billing Logic and Making Cost-Smart Choices

1. The Phenomenon: Flagship Model Pricing Keeps Stepping Upward

A signal worth watching recently appeared in the industry: a newly released flagship LLM published API pricing at roughly ¥100 per million output tokens, ¥20 per million input tokens, and ¥1 per million cache-hit tokens (prices in CNY) — about 5x the previous-generation flagship of the same class. This is not an isolated event. Looking back at the commercialization of LLMs, the alternating pattern of “the older generation keeps getting cheaper while a new generation launches at a premium” has appeared repeatedly. It reflects the ongoing tension between capability upgrades and inference costs.

For developers, emotionally criticizing the price hike is pointless. The questions that actually matter are: Which cost structures does this pricing change alter? How should you choose between subscription plans and pay-per-use APIs? And which engineering levers can push your real costs down?

2. Understanding the Bill: Why Input, Output, and Cache-Hit Tokens Have Three Different Prices

LLM APIs generally charge by the token, but the three token categories carry very different unit prices, and those differences map to three completely different computational workloads.

1. Input tokens (prefill stage). The prompt you submit must be prefilled in a single pass, converting the entire context into a KV cache. Prefill is compute-intensive, but it parallelizes well and keeps GPU utilization high, so its unit price is usually low.

2. Output tokens (decode stage). Every token the model generates depends on the KV cache of all preceding tokens — a strictly autoregressive, serial process that cannot be effectively parallelized. Memory bandwidth becomes the bottleneck. Generating a given amount of text costs far more compute and memory traffic than reading the same amount of input, which is the fundamental reason output unit prices are typically several times input prices.

3. Cache-hit tokens (prefix reuse). Mainstream inference services support prompt caching: the KV cache of an identical prefix (system prompts, few-shot examples, long documents, and so on) can be reused across multiple requests. Hit portions skip prefill entirely and are billed only a small storage-and-scheduling fee. The tenfold-or-more price gaps between output and input, and between input and cache, come exactly from here.

A simplified cost estimator:

def estimate_cost(prompt_tokens, completion_tokens, cached_tokens,
                  p_in=20, p_out=100, p_cache=1):
    # Unit prices: CNY per million tokens
    uncached = prompt_tokens - cached_tokens
    return (uncached / 1e6) * p_in \
         + (cached_tokens / 1e6) * p_cache \
         + (completion_tokens / 1e6) * p_out
# Example: 100K input tokens, 80K of which hit the cache, 20K output tokens
print(estimate_cost(100_000, 20_000, 80_000))

Every 10-percentage-point increase in cache hit rate cuts input-side costs by roughly 19%. The billing structure itself is an optimization opportunity.

3. Why New Flagships Can Command Premium Prices: The Inference-Cost Logic Behind the Hike

1. Parameter scale and activation structure. New-generation flagships generally adopt MoE architectures with far larger total parameter counts — trillions of total parameters, with tens of billions activated per forward pass. A larger expert pool means higher memory footprint and scheduling complexity in every layer’s forward computation, fewer concurrent requests each GPU can host, and therefore a higher hardware cost amortized per token.

2. The non-linear cost of long context. Once context windows extend to hundreds of thousands or even around a million tokens, attention computation and KV cache storage grow with sequence length. Ultra-long requests amplify memory and bandwidth consumption super-linearly, and the inference clusters that serve long context have a much higher unit cost than those serving ordinary-length requests.

3. Capability premium: a higher unit price does not mean a higher total cost. Pricing is set per token, but what engineering teams should really measure is the effective cost of completing a task. A stronger model has a higher success rate on complex coding tasks, needs fewer retries, and requires less manual patching time. If the unit price rises 5x but the first-pass success rate climbs from 40% to 90%, the effective cost per task may actually fall. The evaluation metric should shift from “how much per million tokens” to “how much per solved problem.”

4. Subscription Plans vs. Pay-As-You-Go APIs: A Selection Framework for Two Billing Models

Vendors typically offer both payment models. The essential difference is who carries the risk.

1. Subscription plans (coding-plan style). A fixed monthly fee buys an agreed quota; exceed it and you are throttled or must purchase add-ons. Costs are budgetable, which suits individuals and small teams with stable usage and a monthly iteration cadence. Caveats: quotas are often denominated in credits or request counts, and the tokens you can actually consume are coupled with the model tier and context length — before signing, work out how much quota a typical task consumes per round. Low-priced tiers often restrict which models or context lengths you can use; flagship capability usually starts from a mid-tier plan.

2. Pay-as-you-go APIs. Pay only for what you use, with transparent unit prices — a good fit for volatile usage, burst traffic, and evaluation/testing phases. The trade-off: you must build your own quota management, budget alerts, and usage monitoring, or costs can easily spiral. Details such as caching and batch discounts significantly affect the total bill, so verify them item by item.

3. Selection advice. If usage is predictable and dominated by everyday coding-assistant work: choose a subscription first and transfer price-volatility risk to the provider. If traffic is steep and dominated by production integrations and batch processing: choose pay-as-you-go first and avoid paying for peak capacity. A hybrid strategy is also common — the subscription covers daily development, while pay-as-you-go absorbs peaks and production canary rollouts.

5. Five Engineering Practices That Push Costs Down

1. Stabilize your prompt prefixes to fully capture the caching dividend. Put stable content — system prompts, tool definitions, project conventions — at the very front of the prompt, and dynamic content after it. The more stable the prefix, the higher the hit rate:

messages = [
  {"role": "system", "content": STABLE_SYSTEM_PROMPT},  # Cacheable prefix
  {"role": "user",   "content": dynamic_task_input},     # Dynamic part
]

2. Tiered model routing. Route simple completions, format conversions, and summarization tasks to lightweight model tiers; invoke the flagship only for complex reasoning and repository-scale refactoring. A router can significantly dilute overall costs.

3. Context trimming and retrieval augmentation. Do not stuff the entire codebase or the full conversation history into context: use retrieval to locate relevant files, keep rolling summaries of history, and clean up stale tool output promptly. This lowers input fees and improves quality.

4. Control output length. Output carries the highest unit price. Use structured output constraints, explicit formatting instructions, and sensible stop conditions to avoid verbose generation.

5. Build a usage-monitoring and budget loop. Count token consumption and cost for every request at the gateway layer, tag it by project and by person, and set monthly budget thresholds using logging services and monitoring alerts. Major cloud providers — including Baidu AI Cloud — support this kind of usage accounting in their LLM services and monitoring/alerting products. The key is to establish a mechanism that makes costs visible to the whole team.

6. Build Your Own Cost Evaluation in Three Steps

1. Fix a benchmark task set. Choose 5–10 real development tasks (bug fixes, feature implementation, refactoring) and record the success rate and token consumption of each model or tier.

2. Compute effective cost. Effective cost = (input tokens × input unit price + output tokens × output unit price + human intervention time × labor cost) / number of successful tasks.

3. Re-test regularly. Models and prices are both changing fast. Re-run the benchmark quarterly so you are not locked into a stale conclusion.

Conclusion

A 5x increase in a flagship model’s unit price is, in essence, the market repricing the balance between capability supply and inference cost. Rather than agonizing over unit prices, get three things straight: the leverage that cache and output pricing hold within the billing structure, the risk boundary between subscriptions and pay-as-you-go, and the continuous cost-reduction space on the engineering side. Replace the “price per token” lens with “cost per effective task,” and you can keep making rational decisions as LLMs iterate at full speed.