LLM Inference Pricing Logic and Cost Optimization Practices

1. The Starting Point: Does a Higher Unit Price Mean a Higher Total Cost

Recently, the inference pricing for a certain generation of large language model services has increased noticeably compared with the previous generation. On the output side, the price reaches the range of one hundred yuan per million tokens. On the input side, it is around twenty yuan per million tokens. For cache hits, the price drops to about one yuan per million tokens. Looking at these numbers alone, the increase is significant. However, in engineering practice, the question of whether a model is expensive cannot be answered by unit price alone. What matters is the effective cost per unit of work.

The effective cost can be expressed as:

Cost per task = Σ(input tokens × input unit price) + Σ(output tokens × output unit price) - cache savings

Unit price is only one factor in the formula. What ultimately determines the bill is how many tokens are consumed to complete a task, what proportion of those tokens can hit the cache, and the ratio between plan quota and actual call volume.

2. Horizontal Comparison: Why “Expensive” Is Relative

When discussing pricing, a common reference point is comparable leading models. Across the industry, the general pattern is that more capable models tend to have higher per-token prices, and the leverage ratio between plan quota and actual usable spending varies widely.

A simplified comparison framework looks like this:

Dimension	High-leverage plan model	Low-leverage plan model
Monthly fee tier	Low to mid	Mid to high
Quota leverage	20x and above	5x to 10x
Suitable scenario	High-frequency lightweight calls	Low-frequency heavy tasks

Here, “leverage” refers to how much usage, calculated at the unit price, a user can get for the monthly fee they pay. Higher leverage makes users more inclined to run the plan at full capacity. Lower leverage forces users to control their calls more carefully.

If a service has a high unit price but low plan leverage, high-frequency users will end up paying significantly more than the unit price alone suggests. This is exactly the key difference between “expensive per token” and “expensive in total cost.”

3. The Overlooked Variable: Cache Hit Rate

The cache hit price dropping to roughly one-twentieth of the input price is a highly meaningful engineering signal. It means that the more repeated context a workload has, the more the cost drops.

Typical high cache hit scenarios include:

• Repeated calls with long system prompts
• Reuse of historical messages in multi-turn conversations
• Retrieval-augmented generation with a fixed knowledge base prefix
• Repository-level context in code completion

A simplified cost model looks like this:

def cost(input_tokens, output_tokens, cached_tokens, price_in, price_out, price_cache):
    return (
        (input_tokens - cached_tokens) * price_in
        + cached_tokens * price_cache
        + output_tokens * price_out
    ) / 1_000_000

Assume a task has 8,000 input tokens and 2,000 output tokens, with 6,000 input tokens hitting the cache. Using an input price of 20 yuan, an output price of 100 yuan, and a cache price of 1 yuan:

• Without cache: 8,000×20 + 2,000×100 = 360,000, or 0.36 yuan
• With cache: 2,000×20 + 6,000×1 + 2,000×100 = 246,000, or 0.246 yuan

The cost drops by about 32%. If the cache ratio is higher, the reduction becomes even larger. This shows that a high unit price on the pricing sheet can be significantly diluted under cache-friendly workloads.

4. Token Consumption per Task: A More Critical Metric Than Unit Price

Industry evaluations often use “average tokens consumed per task” to measure model efficiency. This metric reflects how much a model needs to “say” to complete a standard task.

If a model consumes a similar number of tokens per task as leading models, then even a slightly lower unit price may not translate into a lower total cost. Conversely, if consumption is lower, the total cost may still be more favorable even with a higher unit price.

Therefore, when evaluating the economics of a model, it is recommended to proceed in the following order:

1. Measure token consumption per task: Sample on real business workloads rather than relying only on public benchmarks.
2. Measure the cache hit ratio: Distinguish between reusable prefixes and non-reusable content.
3. Calculate the effective unit price: Factor cache discounts into the weighted average unit price.
4. Compare plan leverage: Confirm whether the usage available for the monthly fee can cover actual demand.

Only after completing these four steps can a meaningful conclusion be drawn about whether a model is expensive or economical.

5. Engineering Approaches to Cost Reduction

5.1 Context Layering

Divide the context into a stable layer and a dynamic layer. Put the stable layer (system prompts, tool definitions, knowledge prefixes) at the front so it can be reused through caching. Put the dynamic layer (user input, real-time retrieval results) at the back. This maximizes the cache hit rate.

5.2 Tiered Model Routing

Not every request needs the strongest model. Routing can be based on task complexity:

def route(task):
    if task.complexity == "low":
        return light_model
    elif task.complexity == "medium":
        return standard_model
    else:
        return strong_model

Send simple tasks to a lightweight model and reserve the high-priced model for scenarios that genuinely require deeper reasoning.

5.3 Output Length Constraints

Output tokens are usually more expensive than input tokens. Prompt-based constraints, structured output, and maximum token limits can effectively reduce output-side spending.

5.4 Batching and Asynchronous Processing

For offline tasks, batch APIs often come with additional discounts. For online tasks, asynchronous processing can help avoid extra costs caused by resource contention during peak periods.

5.5 Monitoring and Alerting

Set up token-level monitoring and break down costs by business line, API, and user. Without measurement, there is no optimization. The following metrics are worth tracking:

• Average input/output tokens per request
• Cache hit rate
• Effective cost per task
• Plan quota consumption speed

6. Evaluating Whether a Plan Is Reasonable

From a user’s perspective, whether a plan is reasonable depends on whether the ratio of payment amount to usable quota matches the user’s calling pattern.

If a plan’s leverage is significantly lower than comparable products, high-frequency users will feel that it is not enough. If the leverage is too high, it may attract many low-frequency users to stock up, causing resource misallocation. The ideal design is to make high-frequency users feel it is worthwhile and low-frequency users feel it is flexible.

For developers choosing a plan, the first step should be to estimate monthly token consumption and then work out the required leverage. A simple formula can be used:

Required leverage = monthly consumption amount / monthly fee

If the required leverage is higher than what a plan provides, the plan is not suitable for the current workload. In that case, consider adjusting the calling strategy or choosing a different tier.

7. Conclusion

Evaluating LLM inference pricing should not stop at comparing unit prices. What truly determines cost is token consumption per task, cache hit rate, plan leverage, and the degree of engineering optimization on the application side.

A high unit price does not necessarily mean a high total cost, and a low unit price does not necessarily mean economical. For developers, building a quantifiable cost evaluation framework, combined with cache optimization, model routing, and output constraints, is the way to find the optimal balance between capability and budget. Pricing is the provider’s choice, but cost control is the engineering team’s own responsibility.