Demystifying LLM API Pricing: A Developer’s Guide to Evaluating Real Inference Costs

Recently, a new generation of LLMs was released, with output prices increased by about 5 times compared to its predecessor. The input price is set at around 20 CNY per million tokens, while cache hits are as low as 1 CNY per million tokens. Within the developer community, two perspectives emerged: one argues that “top-tier benchmark scores justify the premium,” while the other points out that after converting subscription quotas into actual usable value, the effective price is higher than monthly subscription plans of comparable overseas models. As a developer who regularly audits inference costs, I believe the core of this debate is not whether the model is “expensive,” but rather that most teams are using the wrong cost metric.

I. Unit Price Is Not Cost: Three Essential Conversions

When comparing model prices, directly looking at “cost per million tokens” will almost certainly lead to the wrong conclusion. You need to perform at least three layers of conversion:

Layer 1: Input-Output Ratio Conversion. In typical conversational and Agent tasks, output tokens often account for 20% to 40% of the total. If Model A’s output unit price is 5 times its input price, while Model B’s output price is only 3 times its input, Model B’s weighted cost might be lower in scenarios with a 30% output ratio. Looking solely at the output unit price will systematically overestimate the former.

Layer 2: Per-Task Token Consumption Conversion. This is the most critical metric. Two models with similar benchmark scores might consume 2 to 3 times the amount of tokens to complete the exact same task. Longer reasoning chains, more tool-calling rounds, and frequent self-corrections all drive up consumption. Industry-standard evaluation methods (such as calculating average consumption per task) are designed specifically for this reason. If Model A’s unit price is 1.5 times that of Model B, but its per-task consumption is 2 times that of Model B, then Model A’s actual cost is 3 times higher. The observation that “average consumption per task is similar” in the original discussion precisely proves that unit price comparisons must be multiplied by a consumption coefficient to be meaningful.

Layer 3: Cache Hit Rate Conversion. A cache hit price as low as 1 CNY per million tokens sounds highly attractive, but the hit rate depends entirely on your business use case. In multi-turn conversations where system prompts and historical context are frequently reused, the hit rate can exceed 50%. However, for one-off long document processing, the hit rate is nearly zero. Calculating cost using “cache price × ideal hit rate” only gives you a theoretical lower limit, not your actual billing amount.

You can solidify these three conversions using a simple estimation function:

def effective_cost(input_price, output_price, cache_price,
                   input_tokens, output_tokens, cache_hit_rate,
                   task_multiplier=1.0):
    """Returns the effective cost per task, task_multiplier is the relative consumption coefficient"""
    cached = input_tokens * cache_hit_rate
    uncached = input_tokens * (1 - cache_hit_rate)
    cost = (uncached * input_price
            + cached * cache_price
            + output_tokens * output_price) / 1_000_000
    return cost * task_multiplier

When you plug the parameters of two different models into this function, the resulting difference often points in the opposite direction of their listed price difference. Conclusion: Horizontal comparisons must use “effective cost per task,” not “unit price per million tokens.”

II. The Pitfalls of Monthly Subscription Conversion

The most valuable observation from the original discussion concerns the conversion of subscription quotas: a $20 overseas model plan can support hundreds of dollars worth of usage, and a $200 plan can support tens of thousands of dollars in usage, yielding a conversion multiplier of about 20x to 50x. In contrast, the domestic model’s 199 CNY plan converts to only about 1,200 to 2,800 CNY worth of usage, a multiplier of around 6x to 14x. This means that a lower-priced plan actually buys less computing power per unit of currency.

This stems from two different subscription design philosophies:

• High-multiplier plans: These use low prices to attract users and lock them in with massive quotas. The essence is to dilute marginal costs through scale, making them suitable for high-frequency, heavy users.
• Low-multiplier plans: The quota is kept close to the cost line, suitable for light users. However, heavy users will quickly find their actual expenditures exceeding expected pay-as-you-go rates.

Developers should conduct a “subscription break-even test” when selecting a model:

def plan_break_even(plan_price, plan_quota_value, monthly_usage_value):
    """Determine if a monthly subscription is worth it"""
    return plan_price < monthly_usage_value * (plan_price / plan_quota_value)

A more intuitive approach is to take your actual token bills from the past three months and calculate the cost using both “pay-as-you-go” and “monthly subscription with overage” methods, then choose the smaller value. Do not be anchored by the raw number of a “XX dollars plan.”

III. Four Engineering Techniques to Reduce Costs

Regardless of which model you choose, the following techniques can reduce your bill by 30% to 60% without sacrificing output quality:

1. Context Layering and Pruning. Manage system prompts, few-shot examples, historical summaries, and current-turn inputs in layers. Place fixed content in the prefix to increase cache hits. Use rolling summaries for historical conversations instead of full concatenation. Dynamically select few-shot examples based on task type rather than injecting them all at once. In practice, simply replacing full historical context with summaries can cut input tokens by over 40%.

2. Task Tiered Routing. Not all requests require the most powerful model. Categorize requests into three types: simple classification/extraction goes to a small model, regular Q&A goes to a mid-tier model, and complex reasoning and code generation are routed to the flagship model. The routing layer can be implemented using rules (keywords, input length) or a lightweight classifier. This immediately reduces costs with almost zero perceptible impact on the user.

3. Output Length Constraints. Output unit prices are typically 3 to 5 times the input price, making output the largest cost center. Compress outputs using structured output (JSON Schema constraints), explicit maximum length instructions, and prohibiting the repetition of the prompt. A common waste is the model restating the question before answering—adding a simple instruction like “Give the conclusion directly, do not restate the question” can save a considerable amount of output tokens.

4. Batching and Asynchrony. Route non-real-time tasks (data labeling, batch summarization, offline evaluation) through batch processing channels, which typically offer significant discounts. Concurrently, keep concurrency within quotas to avoid triggering rate limits, which results in retries that amplify costs.

IV. Model Selection Checklist

Here is an actionable checklist for teams currently evaluating LLMs:

1. Run evaluations with real business samples. Track average input/output tokens per task and task completion rates, rather than relying on public benchmark scores.
2. Calculate the effective cost per task. Include the consumption coefficient and cache hit rate, rather than just looking at the listed price.
3. Calculate the subscription break-even point. Back-calculate using historical bills, not the face value of the subscription.
4. Evaluate migration costs. This includes prompt adaptation, output format parsing, and tool-calling protocol differences—these hidden costs are often underestimated.
5. Maintain a multi-model abstraction layer. Encapsulate API calls behind a unified interface, allowing you to dynamically switch models based on cost and quality.

Returning to the original question: is a 5x price increase for a new generation model justified? From a purely technical perspective, benchmark improvements correspond to higher training and inference investments, so a price increase has its underlying cost logic. But from a developer’s perspective, whether it is worth paying a premium for these capabilities depends on whether your tasks genuinely utilize them, and whether you have implemented cost engineering. For the vast majority of business scenarios, the reasonable approach is not to chase the lowest unit price or the highest benchmark scores. Instead, establish a dual-metric evaluation system based on “effective cost per task + quality pass rate” to ensure every cent of your inference budget is well spent. Price lists will change, but this methodology will not.