5x LLM API Price Hike: Cost Logic and Model Selection Strategies
A recent 500% price increase for a next-generation LLM API has sparked discussions about the real cost of AI. This article delves into the technical cost structure behind API pricing, analyzes the discrepancy between list prices and actual task-level costs, and provides practical strategies for enterprise model selection and cost optimization.
Recently, a leading LLM provider announced pricing for its new generation API that caught the attention of many developers: 100 CNY per million tokens for output, 20 CNY per million tokens for input, and 1 CNY per million tokens for cache hits. Compared to the previous generation, this represents an overall price increase of about five times. The model supports long-context input, file comprehension, content generation, and data analysis, ranking highly in comprehensive evaluations. On the surface, this seems like a natural result of “stronger capabilities, higher prices.” However, when analyzing API unit prices, task consumption, subscription quotas, and caching strategies together, the conclusion is far from simple. This article explores the issue from both technical cost and actual usage cost perspectives.
1. The Cost Structure Behind the Price List
Why is Output Much More Expensive Than Input?
LLM inference is divided into two phases: prefill and decode. Prefill allows for parallel computation; the more input tokens there are, the denser the matrix operations processed at once, making it easier to dilute the unit cost per token. Decode generates tokens sequentially, where each step depends on the result of the previous one. Memory bandwidth and access latency become the primary bottlenecks, making the unit cost of output tokens much higher than input. This explains why the new model’s output price is set at 5 times the input price: output tokens indeed consume significantly more computational power.
Why is Cache Hit So Cheap?
In every request’s input, system prompts, fixed contexts, and the beginning portions of the same document are often repeatedly used. If the server supports prefix caching and reuses the KV cache across requests, this part of the input no longer needs to be prefilled, significantly reducing computation. The cache hit pricing is only 1 CNY per million tokens, 95% cheaper than the 20 CNY per million tokens for uncached input. For scenarios like long-text processing, file comprehension, and data analysis—where there is a “large input, multi-turn querying the same document” pattern—the cache hit rate directly determines the final cost.
The Impact of Long-Context and Multimodal Inputs
The next-generation model supports long-context input and file comprehension, meaning the input tokens per request could surge from a few thousand to hundreds of thousands or more. Even if the unit price remains unchanged, long context will multiply the cost per task. Conversely, this makes caching even more critical: if fixed documents, knowledge base prefixes, and system prompts are designed as reusable prefixes, a massive amount of input costs can be reduced from 20 CNY to 1 CNY.
2. The Discrepancy Between List Price and Real Cost
API Unit Price is Just the “MSRP”
The API pricing table represents the official standard rate, but what developers actually pay depends on how many tokens each task consumes. Different models vary greatly in answer quality, reasoning depth, and task completion. A model adept at deep thinking might output longer reasoning chains and more intermediate steps for each query; a model that answers simply and directly might use fewer output tokens but require multiple calls to complete a task. Therefore, simply comparing “price per million tokens” is meaningless; you should compare “the average cost to complete a task.”
The Concept of “Expensive” from a Task-Level Cost Perspective
In similar benchmark tests, the average token consumption per task for the new generation model has approached the level of leading international models. That means, if your task is to “have the model write a report,” it may not use significantly fewer tokens than leading international models. Since international models often provide more converted quotas at the same price point in their subscription packages, competing models might actually feel more expensive in practice despite having lower list prices. This is a classic case of “cheap unit price but not cheap total price.”
Subscription Quotas and Reset Rules
Many providers offer both pay-as-you-go API billing and subscription packages. Subscriptions typically convert “available quota” based on the API list price, rather than being unlimited. Suppose a provider’s monthly subscription is 199 CNY, but the converted API usage is roughly between 1,200 and 2,800 CNY; users will hit the quota quickly if they run heavy tasks. More critical is the quota reset frequency: some reset weekly, others daily; different product lines may have independent quota pools that cannot be shared. These rules can significantly alter actual user costs, sometimes making a “low-price package” not cheap at all.
3. Why Doesn’t Price Increase Linearly with Model Scale?
Some argue that as model parameters grow from hundreds of billions to trillions, inference costs must double; coupled with feature enhancements, a fivefold price hike is “reasonable.” However, from an engineering perspective, the relationship between parameter scale and inference cost is not linear. Current LLMs widely adopt sparsely activated architectures (MoE). Although the total parameter count is high, the number of parameters actually activated per token may not change much. Optimization techniques like activated parameters, KV cache size, batching efficiency, quantization precision, and speculative decoding all decouple inference costs from total parameters. Therefore, doubling parameters does not mean the provider’s costs double.
So why the fivefold price increase? Beyond actual compute costs, one must consider R&D cost amortization, long-context KV cache optimization, backend support for file parsing and data analysis, and marketing/channel costs. At the same time, price acts as a “filter”: if the new model significantly leads in comprehensive evaluations, the provider can recoup investments early and lower prices later as industry competition intensifies. This is also why many predict that once open-source competitors complete training their subsequent versions and the performance gap is closed, this pricing system will be quickly disrupted. The market will force service providers back to a “performance-price” equilibrium.
4. Enterprise Model Selection and Cost Control Practices
Build a Task-Level Cost Estimation Model
Don’t just look at token unit prices; factor in “input length, output length, cache hit rate, and call frequency” for each scenario. You can maintain a simple budget function:
def estimate_cost(requests, avg_input, avg_output, cache_rate):uncached_input = avg_input * (1 - cache_rate)cached_input = avg_input * cache_rateprefill_cost = uncached_input * INPUT_PRICE / 1_000_000cache_cost = cached_input * CACHE_PRICE / 1_000_000output_cost = avg_output * OUTPUT_PRICE / 1_000_000return (prefill_cost + cache_cost + output_cost) * requests
Use this function to compare different models. The “monthly total cost” calculated under a fixed task volume is the truly comparable metric.
Treat Cache Hit Rate as a KPI
- Place system prompts and fixed regulatory/knowledge base prefixes at the very beginning of the input to ensure a consistent cached prefix.
- Avoid frequently changing the prefix order in the same session; standardize the extraction of document content.
- For long texts, perform chunked retrieval first and only send key fragments instead of sending the full text every time.
Control Output Length and Reasoning Depth
For tasks like content writing and data analysis, explicitly require “conclusion first, then steps,” and set a maximum output token limit. For tasks requiring deep reasoning, you can have the model output only the final answer without displaying the full reasoning chain. Output tokens are the bulk of the cost; every bit saved counts.
Task-Tiered Routing
Use lightweight models for simple rewriting, summarization, and classification tasks; reserve expensive models for complex planning and long-document analysis. Through routing strategies, you can reduce overall costs by an order of magnitude without sacrificing core task quality.
Monitoring and Budget Alerts
Log the input/output token count, cache hit status, and cost for each request at the API call layer, and perform aggregated analysis by user, project, and scenario. Set daily/monthly budget alerts to avoid “running a report and exhausting the monthly quota.”
Choose the Right Cloud Provider and Billing Model
Different model service providers have vastly different billing strategies for caching, resource packages, and batch tasks. When selecting a model, besides focusing on model quality, incorporate “list price, cache discounts, package quotas, and quota reset frequency” into your evaluation. For example, when selecting model services on cloud platforms like Baidu AI Cloud, you should integrate token unit price, cache discounts, and resource packages into your cost model before deciding between pay-as-you-go or subscription models.
5. Conclusion
A fivefold price hike in LLM APIs cannot be simply judged as “expensive” or “not expensive.” For users, what truly matters is “how much it costs to complete the same task.” Stronger models often “think more,” leading to higher token consumption per task; coupled with subscription quotas, cache hits, and reset rules, there is often a massive deviation between list price and real cost. Only by breaking costs down to the task and scenario level can developers avoid being misled by unit prices and make sound selection decisions. Prices will fluctuate with competition, but “task-level cost accounting” will always be the core methodology for controlling LLM expenditures.
