How to Evaluate LLM API Pricing Scientifically: True Cost Accounting from Tokens to Usage

1. Why Looking Only at Unit Price Is Often Misleading

Whenever a new generation of models is released, pricing inevitably sparks discussion: the price table may be noticeably higher than the previous generation, and input, output, and cache hits are often billed at separate unit prices. Many people’s first reaction is “it got more expensive,” but this conclusion often does not hold up. The reason: a price table is just a unit price, while true cost = unit price × actual consumption volume.

Consumption volume is affected by multiple factors: model behavior (context length, the number of tokens produced by reasoning-style thinking, verbosity of output), application architecture (whether caching is used, whether multi-model tiered calling is done), and task success rate (failed retries multiply costs). Two models with similar unit prices can differ by several times in unit cost on the same task. Therefore, evaluating whether “a model is expensive” should be an accounting question, not an impression-based one. This article follows the order of “billing structure → task-level true cost → subscription plan value → ultimate selection benchmark” to provide a reusable accounting framework.

2. Breaking Down the Billing Structure: Why Input, Output, and Cache Are Three Prices

2.1 The Physical Cost Difference Between Input and Output

LLM inference has two stages with very different cost characteristics:

• Prefill (input processing): The input sequence is sent into the model for parallel computation, fully utilizing GPU matrix operations, with high hardware efficiency and low amortized compute cost per token;
• Decode (output generation): Tokens are generated autoregressively one by one, and each step is constrained by memory bandwidth, resulting in low hardware utilization and significant latency cost.

This is the technical root of why output unit prices are usually several times higher than input unit prices in the industry’s common pricing systems. When you see a “price increase,” the first thing to check is which line item increased: for tasks with long input and short output (document classification, information extraction), the input unit price has a greater impact; for long-output tasks (code generation, long-form writing), the output unit price is the dominant variable.

2.2 The Economics of Cache Hits

“Cache hit” corresponds to a prefix-level KV Cache (context caching) mechanism: when the prefixes of consecutive requests such as system prompts, few-shot examples, and long document context are completely identical, repeated computation can be skipped and the cached intermediate state can be reused directly. The cache hit unit price is usually an order of magnitude lower than the regular input unit price.

But the value of this mechanism depends on two engineering conditions:

1. Prefix stability: Fix the content that remains unchanged across multiple calls (role settings, tool definitions, knowledge snippets) at the very front of the prompt;
2. Time window and call frequency: Caches usually have a validity period, and high-frequency, continuous calls are more likely to hit.

For Agent-type applications with high-frequency calls and fixed prefixes, caching can save considerable input costs; for occasional single calls, caching is almost imperceptible. Comparing “whether cache pricing is cheap” without considering your own calling pattern is not very meaningful.

2.3 Key Points for Dissecting Generational Price Changes

When a new generation model’s pricing is noticeably higher than the previous generation, rather than making an emotional judgment, break the change into three questions:

1. Which line item increased: Input, output, or cache? Compare this with your task structure to determine which line item accounts for the largest share of cost;
2. Has the capability improvement changed usage volume: Fewer reasoning steps, fewer retries, more restrained output, and fewer multi-turn interactions all shrink the “usage” factor in the multiplication, potentially offsetting or even outweighing the unit price increase;
3. Is the old version still on sale: If the old model is still available, you can migrate gradually in tiers based on task sensitivity to capability, rather than switching all at once.

3. From Unit Price to True Cost: Task-Level Accounting Methods

3.1 True Cost Formula for a Single Task

For a task containing multiple rounds of interaction (such as an Agent executing tool calls), the true cost can be approximated as:

True cost per task ≈ Σ(for each round) [
    (input tokens − cache hit tokens) × input unit price
  + cache hit tokens × cache unit price
  + output tokens × output unit price
]

3.2 The Two Most Underestimated Variables

• Token consumption differences: Different models completing the same task vary greatly in thinking chain length, tool-calling structure, and output detail. At the same unit price, a “chatty” model’s true cost may be several times higher;
• Number of interaction rounds: Agent workflows often require multiple back-and-forth steps of “model → tool → model,” and each round must carry the context again. Cost grows approximately linearly or even superlinearly with the number of rounds.

3.3 Example Calculation (Numbers Are for Demonstration Only)

Assume a hypothetical pricing: input 4 yuan per million tokens, output 16 yuan per million tokens, cache hit 0.4 yuan per million tokens. A task averages 3 rounds, each round has 20,000 input tokens (50% cache hit rate) and 2,000 output tokens:

• Input cost per round: 10,000 × 4/1,000,000 + 10,000 × 0.4/1,000,000 = 0.04 + 0.004 = 0.044 yuan
• Output cost per round: 2,000 × 16/1,000,000 = 0.032 yuan
• Total per task: (0.044 + 0.032) × 3 = 0.228 yuan

def task_cost(input_tokens, output_tokens, rounds,
              price_in, price_out, price_cache, cache_hit_rate=0.0):
    """Estimate cost per task (yuan); unit prices are per million tokens"""
    total = 0.0
    for _ in range(rounds):
        hit = input_tokens * cache_hit_rate
        miss = input_tokens - hit
        total += miss / 1e6 * price_in + hit / 1e6 * price_cache
        total += output_tokens / 1e6 * price_out
    return total
cost = task_cost(input_tokens=20_000, output_tokens=2_000, rounds=3,
                 price_in=4, price_out=16, price_cache=0.4,
                 cache_hit_rate=0.5)
print(f"True cost per task: {cost:.3f} yuan")

Multiplying the cost per task by the monthly task volume gives the budget baseline; conversely, dividing the budget by the cost per task yields the maximum affordable number of rounds, which can serve as a hard constraint for architecture design.

4. Subscription Plans vs. Pay-as-You-Go: Calculating the Implied Usage Multiplier

Major platforms generally offer both monthly subscriptions and pay-as-you-go billing. The core of evaluating subscription value is the implied usage multiplier:

Implied usage multiplier = plan usable quota (converted at list price) / plan price

For example (hypothetical), a plan costs 200 yuan per month and allows consumption equivalent to 1,000 yuan of list-price usage, giving an implied multiplier of 5x. But two conditions must be layered on top:

1. Quota depletion rate: The multiplier only holds if most of the quota is consistently used. If only 30% is used, the actual discount becomes 5 × 0.3 = 1.5x, which may be worse than pay-as-you-go tiered discounts;
2. Scope of quota applicability: Some quota only covers specific model tiers or specific scenarios, and flagship capabilities may not be included or may have a lower discount rate. The billing terms need to be checked item by item.

From an engineering perspective, the recommendation is: after project initiation, first run a gray-scale trial for two to four weeks, accumulate real call frequency, token distribution, and success rate data, and then decide whether to go with subscription or pay-as-you-go, rather than betting all at once based on intuition.

5. The Ultimate Benchmark: Cost per Successful Task

The most easily overlooked and most critical variable in the accounting formula is task success rate. A low unit price with a low success rate leads to manual review and repeated retries, and the true cost plus labor cost actually rises. Therefore, the selection criterion should be revised to:

Cost per successful task = true cost per task / task success rate

Using this for generational comparison makes conclusions much clearer: if a new generation model significantly improves the success rate on your core tasks and produces more restrained output, then even if the list price rises several times, the cost per successful task may decline rather than increase; conversely, if the capability improvement is irrelevant to your scenario, the price increase is pure cost deterioration. “Capability” and “price” must be placed under the same denominator for comparison to be meaningful.

The practical approach is to build a small internal evaluation set: extract 50 to 200 real tasks from the business, have the old and new models (as well as several general comparable models) run batches in groups, record token consumption, interaction rounds, success rate, and manual review rate, and uniformly convert them into cost per successful task. The long-term value of this data asset far exceeds a one-time subjective evaluation.

6. Engineering Practices for Cost Optimization

1. Tiered calling: Route simple tasks such as intent recognition and format conversion to low-cost small models or rule engines, and reserve flagship models only for critical steps;
2. Cache-friendly prompt structure: Fix system prompts, tool definitions, and knowledge snippets into stable prefixes to improve context cache hit rates;
3. Output constraints: Use length limits, structured output (JSON Schema), and instructions such as “output only the conclusion” to suppress token bloat;
4. Context slimming: Periodically trim historical conversations and compress tool return results to avoid carrying the full context in every round;
5. Monitoring and alerting: Use the usage dashboards, per-call token statistics, and budget alerting capabilities generally provided by model service platforms (including Baidu AI Cloud) to detect anomalies such as “sudden token spikes” on the same day;
6. Retry strategy design: Set exponential backoff and a maximum retry count for failed calls to eliminate extreme cases such as “rerunning a wrong answer ten times.”

7. Conclusion

Behind LLM pricing tables is the economics of inference hardware: separate pricing for input and output comes from differences in computational characteristics, and cache discounts come from the reuse of prefix computation. But a price table is always only the starting point of cost. The real basis for decision-making is task-level accounting—multiply unit price by real usage, divide by task success rate, and then compare horizontally across generations and competitors. Turn this accounting into an internal tool and continuously calibrate it with real business data, and “whether it is expensive” will have a definite answer. Being price-sensitive is not a bad thing; it pushes the entire industry from stacking parameters toward pursuing cost efficiency per unit task. And developers who can clearly do the math always hold the initiative.