Flagship LLM API Pricing Logic and How to Evaluate It

1. Why API pricing cannot be judged by “cost per million tokens” alone

LLM API billing is becoming increasingly granular. Many teams used to focus only on the price per million tokens, but actual invoices often differ significantly from expectations. The reason is that a complete model call typically involves three distinct billing categories:

1. Input tokens: system prompts, conversation history, retrieved context, tool call results, and similar content sent to the model.
2. Output tokens: content actually generated by the model.
3. Cache-hit tokens: when a prefix or segment of the input has been cached by the platform, sending the same content again may be billed at a lower rate.

Focusing only on the input price makes it easy to overlook output costs. In practice, some scenarios generate far more output tokens than input tokens. In long conversations, document Q&A, and code completion, input tokens often dominate the total cost. A more reasonable approach is to calculate the proportion of the three token types based on your own traffic patterns, then estimate costs using the corresponding unit prices.

The flagship model discussed in this article has API pricing of 6 yuan per million input tokens, 18 yuan per million output tokens, and 0.3 yuan per million cache-hit tokens. The three billing categories use the same token accounting method, making it easier for developers to estimate costs directly. This pricing is relatively low among comparable flagship models. By comparison, the output unit price of some overseas flagship models can be several times higher after currency conversion.

2. Input, output, and cache-hit tokens: differences and impact of the three billing categories

2.1 Input tokens: the main source of hidden costs

Input tokens are the part that many teams most easily underestimate. A typical RAG application may send several KB or even tens of KB of retrieval results with every request. If a user question contains only a dozen tokens but the context includes thousands of tokens of document fragments, input costs will far exceed output costs.

More importantly, input tokens affect not only cost but also latency and throughput. Longer input increases the time required for the model prefilling phase and raises time-to-first-token latency. Therefore, compressing input context is not only a cost optimization measure but also a performance optimization measure.

2.2 Output tokens: the highest unit price and the need for precise control

In the pricing structure above, the output unit price is three times the input unit price. This means that verbose, repetitive, or unconstrained model output can quickly drive up costs. In some open-ended generation scenarios, output tokens may even be several times the input volume, making output charges the dominant part of the bill.

Common engineering controls include:

• Setting a reasonable max_tokens limit.
• Explicitly requesting concise output in the prompt.
• Using structured output to reduce unnecessary explanations.
• Processing long-form generation tasks in segments to avoid ineffective generation.

2.3 Cache hits: the core lever for cost optimization

The cache-hit unit price is one twentieth of the input price, making it a key cost-control mechanism. In long conversations, historical messages are usually sent repeatedly. If the platform supports prefix caching, all content except the latest turn may be eligible for cache hits.

However, the cache hit rate depends on how calls are structured. If each request reorganizes the message order, dynamically inserts timestamps, or adds random markers, the cache hit rate will drop significantly. Developers who want to take full advantage of caching should keep prefixes as stable as possible and place changing content at the end.

3. A unified cost estimation model: how to compare different pricing systems

Different providers do not always use the same pricing approach. Some bill input and output separately, some distinguish between offline and online inference, some apply different caching policies, and some charge different rates for different model versions. Therefore, directly comparing “price per million tokens” is often not rigorous enough.

The following simplified model is recommended for estimation:

Cost per call ≈ 
    input price × average input tokens
  + output price × average output tokens
  + cache price × average cache-hit tokens

This can be extended with traffic metrics:

Monthly cost ≈ 
    cost per call × average monthly calls

If the workload has meaningful input cache-hit potential, the cache hit rate can be introduced as a variable:

Effective input cost = 
    miss ratio × input price + hit ratio × cache price

For example, suppose an application averages 10,000 input tokens and 1,000 output tokens per call, with a cache hit rate of 80%. Based on the prices above:

• Non-cached input: 2,000 tokens × 6 yuan per million = 0.012 yuan
• Cached input: 8,000 tokens × 0.3 yuan per million = 0.0024 yuan
• Output: 1,000 tokens × 18 yuan per million = 0.018 yuan
• Cost per call: approximately 0.0324 yuan

If the cache hit rate drops to 0%, the cost per call becomes:

• Input: 10,000 × 6 per million = 0.06 yuan
• Output: 0.018 yuan
• Cost per call: 0.078 yuan

Increasing the cache hit rate from 0% to 80% reduces the per-call cost by approximately 58%. This shows that for long-input workloads, caching strategy can affect cost even more than the model unit price itself.

4. Understanding flagship model pricing logic from a cost perspective

Current flagship models generally offer strong complex reasoning capabilities, but high performance often comes with higher resource consumption. As a result, flagship models typically use differentiated pricing to balance the following tensions:

• Inference costs are high and need to cover compute and memory usage.
• Enterprise customers demand high output quality and are willing to pay for lower error rates.
• Developers are cost-sensitive and need caching mechanisms to reduce long-context burdens.

A low input price, high output price, and extremely low cache price reflect a typical engineering trade-off: platforms want to encourage developers to reuse context and reduce redundant computation. At the same time, output is the part where the model actually generates new tokens and incurs the highest computational cost, so it is priced highest.

For developers, this leads to two important conclusions:

1. Optimizing input structure is more effective than simply chasing a lower unit price. If your workload has long and highly repetitive input, cache hits may deliver far greater savings than switching to a cheaper model.
2. Output control is essential for avoiding runaway costs. Without sacrificing quality, limiting output length and improving prompts can directly reduce expenses.

5. Engineering practices for cost control

5.1 Structure prompts into separate components

Managing system prompts, tool definitions, conversation history, and knowledge base content as distinct blocks can improve cache hit rates. System prompts and tool definitions are usually stable and should be placed at the beginning. Conversation history and retrieved content may change and should be placed in the middle or at the end.

5.2 Choose different models for different scenarios

Not every task requires a flagship model. Simple classification, extraction, and summarization tasks can use lightweight models, while complex reasoning, long-document understanding, and code generation can call a flagship model. Although this article does not compare specific models in detail, this kind of tiered routing is common industry practice.

5.3 Establish cost monitoring and alerting

On the engineering side, record the number of input, output, and cache-hit tokens for every call, and connect this data to monitoring and alerting systems. When the per-call cost of a particular task type rises abnormally, teams can quickly identify whether the cause is prompt expansion, uncontrolled output, or a drop in cache hit rate.

5.4 Evaluate long-term architectural costs

In addition to model API charges, consider:

• Network request and retry costs.
• Result caching and knowledge base construction costs.
• Log storage and compliance audit costs.
• Engineering complexity introduced by multi-model routing.

Only by evaluating API costs within the overall architecture can teams make more informed decisions.

6. Summary

The unit price of flagship LLM APIs is important, but developers should pay more attention to their own call structure. The price differences among input tokens, output tokens, and cache-hit tokens mean that cost composition can vary significantly across different business patterns. The pricing system discussed in this article—6 yuan per million input tokens, 18 yuan per million output tokens, and 0.3 yuan per million cache-hit tokens—is representative among flagship models and provides a clear path for developers to reduce costs substantially through cache optimization.

When evaluating options, it is better to first build an internal cost estimation model based on the token distribution and cache hit rate of real production traffic than to simply compare headline unit prices. At the same time, engineering practices such as prompt structuring, output constraints, tiered routing, and cost monitoring can achieve better cost efficiency without sacrificing model performance.