LLM API Cost-Effectiveness: How to Build a Verifiable Cost Model
Published unit prices for large language model (LLM) APIs are not enough to choose a model or channel. This article explains how to build a verifiable cost model that accounts for input and output token mix, cache billing, failed requests and retries, model versions, regions, currency conversion, and price snapshot timing. It also shows how to compare cost and quality using cost per successful task and outlines a reproducible workflow for usage collection, price mapping, aggregation, and reporting.
1. Why Unit Price Tables Cannot Directly Determine Model Selection
The list price of the same model may differ by several times across different channels. The differences usually come from deployment region, channel markup or discount policies, whether cache billing is included, and whether the versions are the same. Many teams first look at the official price and then assume that other channels are more expensive. This assumption is not always valid. Pricing must be checked item by item rather than judged by impression.
A reliable cost-effectiveness evaluation should answer three questions: which version does this unit price correspond to, what billing unit is used, and what is the total cost under an actual workload?
2. Decomposing the Cost Model
Usage-based LLM APIs typically price input tokens, output tokens, and input tokens that hit the cache separately. The cost of a single request can be written as:
C = T_in × p_in + T_out × p_out + T_cache × p_cache
where T is the number of tokens and p is the corresponding unit price. All prices need to be normalized to the same billing unit, such as price per million tokens, before calculation.
2.1 Input/Output Ratio
When comparing the total cost of different channels, a representative consumption ratio is needed. Expressing the ratio as weights, the weighted unit price is:
weighted unit price = w_in × p_in + w_out × p_out, with w_in + w_out = 1
The weights depend on workload characteristics. For coding agents, each round of requests often carries a long context, and the input volume is usually large. If the model outputs long reasoning content, the share of output tokens may also increase significantly. The ratios in these two cases can be very different, so a fixed value should not be applied blindly. The most reliable approach is to collect real input and output token counts from request logs and then calculate the ratio.
The numbers below are for demonstration only and are not actual prices of any channel.
| Channel | Input price (CNY per million tokens) | Output price (CNY per million tokens) |
|---|---|---|
| Channel A | 2 | 8 |
| Channel B | 4 | 6 |
When w_in = 0.3 and w_out = 0.7, the weighted unit price for Channel A is 0.3×2 + 0.7×8 = 6.2, and for Channel B it is 0.3×4 + 0.7×6 = 5.4. Channel B is cheaper. If the input share rises to 0.8, Channel A becomes 0.8×2 + 0.2×8 = 3.2, and Channel B becomes 0.8×4 + 0.2×6 = 4.4. The ranking then reverses. The ranking is sensitive to the consumption mix, so the ratio must be determined before comparison.
2.2 Cache Billing
Multi-turn conversations and agent tasks repeatedly send the same prefix, such as system instructions, tool definitions, and historical context. If the service supports prefix caching, the hit portion is usually billed at a lower unit price. When calculating costs, cache hits must be counted separately; otherwise, the cost will be overestimated. Cache storage and hit rules vary by channel and should follow the billing documentation of the relevant channel. They should not be inferred from the rules of another channel.
2.3 Failed Requests and Retries
Timeouts, retries after rate limiting, and truncated outputs can all incur charges. When calculating costs, use the token counts returned by the bill or usage API rather than inferring from the number of successful business-level requests.
3. Ensuring Comparability
- Version alignment: Compare only the same version of the same model. Models with the same name released at different times may have different capabilities and prices. If a channel does not offer the corresponding version, mark it as “no corresponding version”; do not substitute an older version or a specialized version.
- Region and site: Prices, settlement currencies, and tax treatment may differ across sites. Record site information for every data point.
- Exchange rates: Normalize to a single currency and record the exchange rate and date. In cross-currency comparisons, exchange-rate changes may change the ranking.
- Unverifiable prices: Mark prices that cannot be confirmed from public pricing pages as unpublished. Do not fill in estimated values, or the conclusion will lack a basis.
- Capture time: Prices change frequently, so record a snapshot time for every price list.
4. Looking at Cost and Quality Together
A low price per token does not necessarily mean a low cost per task. If a model needs more rounds or more retries to complete a task, the total spend may be higher. Therefore, record pass rate, average rounds, and total spend on the same set of tasks, and calculate:
cost per successful task = total spend / number of successful tasks
This metric combines cost and quality into a single number. If pass rates differ significantly across models, an advantage in unit price may be offset.
5. Engineering Implementation: Building a Reproducible Costing Workflow
A reliable costing workflow typically has four steps: collect usage, map prices, aggregate calculations, and generate reports. Price tables should be stored as versioned configuration. Format example:
Price table (versioned configuration)Channel: channel_aModel: model_xVersion: 2026-08 snapshotInput price: price per million tokensOutput price: price per million tokensCache hit price: price per million tokensCurrency: CNYCapture time: record date
Calculation logic example:
def estimate_cost(usage_records, price_table):total = 0.0for r in usage_records:p = price_table[r.channel][r.model]uncached_input = r.input_tokens - r.cached_tokenstotal += uncached_input * p.input / 1e6total += r.cached_tokens * p.cache / 1e6total += r.output * p.output / 1e6return total
