Multi-Model API Cost Comparison: Pricing Analysis and Selection Best Practices
Evaluating large language model API costs requires more than just comparing input token prices. This article explains how to accurately interpret pricing tables by factoring in output tokens, cache hits/writes, batch discounts, and speed tiers. It introduces a reproducible cost calculation model with pseudocode, analyzes typical load scenarios, and provides governance best practices to help teams make informed, cost-effective multi-model API selection decisions.
Multi-Model API Cost Comparison: Pricing Analysis and Selection Best Practices
In large model API selection, pricing tables are often the first resource reviewed but also the most easily misunderstood. Many teams only compare “input price” or “output price,” overlooking variables like cache hits, cache writes, batch discounts, speed tiers, and output length distributions. This leads to a significant discrepancy between estimated costs and actual bills. A better approach is to abstract the public pricing table into a set of billing parameters, build a reproducible cost model based on business load profiles, and finally compare multiple models using unified metrics.
1. Why You Shouldn’t Compare Costs Based Solely on Input Price
Input price is intuitive, but it only covers a portion of the request-side cost. In actual calls, output tokens are usually billed at a higher rate, and scenarios like long responses, code generation, or structured extraction will significantly amplify output costs. Comparing only input prices might underestimate the true cost of models with long outputs or overestimate cost differences in short-output, high-cache-hit scenarios.
Furthermore, cache-related billing items are becoming crucial components of pricing tables. Cache hits usually correspond to lower prices, while cache writes might be billed separately by short-term or long-term periods. For businesses with stable system prompts, repeated knowledge base contexts, or consistent multi-turn prefixes, the cache hit ratio directly affects per-request cost. For contexts that change frequently, cache write costs cannot be ignored. Therefore, cost comparison must evolve from “unit price comparison” to “billing dimension comparison.”
Batch discounts also require attention. Some pricing tables offer discounts for asynchronous batch processing, suitable for non-real-time tasks like offline evaluation, data cleaning, or batch annotation. If the business tolerates latency, batch discounts can be more effective than simply choosing a cheaper model; if low latency is required, the premium for high-speed or low-latency tiers must be evaluated. Cost optimization isn’t just about picking the cheapest option, but balancing latency, quality, and cost.
2. Key Fields in Pricing Tables and Unified Metrics
When reading public pricing tables, it is recommended to standardize units and metrics first. Common fields include standard input price, output price, cache hit price, cache write price, batch discount ratio, and tier differences. Different tables might price per thousand tokens or per million tokens, and might differentiate cache writes by validity period. Before comparing, unify conversions to “cost per million tokens” or “cost per thousand requests,” and clarify whether taxes are included, whether regions are differentiated, and whether tiered pricing applies.
It is recommended to establish the following field mapping:
- Input price: Standard input token cost without a cache hit.
- Output price: Model-generated token cost.
- Cache hit price: Input-side cost when hitting an existing cache.
- Cache write price: Cost of writing to the cache. If the pricing table differentiates short/long term, model them separately.
- Batch discount: Discount coefficient for batch or asynchronous calls (1 if none).
- Tier factor: Price coefficient between high-speed, standard, and economy tiers.
It is important to emphasize that the billing boundaries for cache hits and cache writes may differ across vendors. From an engineering perspective, strictly split tokens according to the official documentation to avoid double billing or omission. Do not assume a vendor supports a specific caching strategy; rely on public pricing tables and documentation. For unsupported dimensions, set them to 0 or mark as N/A in the model, rather than substituting with another vendor’s rules.
3. Cost Calculation Model and Pseudocode
To make multi-model comparison reproducible, you can define a price vector for each model and a load profile for the business. The load profile should at least include: average input tokens, average output tokens, cache hit tokens, short-term cache write tokens, long-term cache write tokens, batch call proportion, and high-speed tier proportion. Then, use a unified formula to calculate the per-request cost.
The schematic formula is as follows:
Per-request cost =Normal input tokens * Input price+ Cache hit tokens * Cache hit price+ Short-term cache write tokens * Short-term write price+ Long-term cache write tokens * Long-term write price+ Output tokens * Output priceThen multiplied by the batch discount coefficient and tier factor
Note that normal input tokens should subtract cache hit and cache write parts according to pricing metrics to avoid double counting. Below is a vendor-agnostic pseudocode example:
def request_cost(profile, price):normal_input = max(profile["input_tokens"]- profile["cache_hit_tokens"]- profile["cache_write_short_tokens"]- profile["cache_write_long_tokens"],0)cost = (normal_input * price["input"]+ profile["cache_hit_tokens"] * price["cache_hit"]+ profile["cache_write_short_tokens"] * price["cache_write_short"]+ profile["cache_write_long_tokens"] * price["cache_write_long"]+ profile["output_tokens"] * price["output"])cost *= price.get("batch_discount", 1.0)cost *= price.get("speed_tier_factor", 1.0)return cost
The value of this model lies in its explainability: when the output proportion rises, you can quickly see the impact of the output price; when the cache hit rate improves, you can quantify the savings; when batch discounts change, you can evaluate whether offline tasks are worth migrating. When comparing multiple models, use the same load profile and the same set of unit metrics, otherwise the conclusions are not comparable.
4. Typical Load Scenario Analysis
The first category is the “Long Context, Low Output” scenario, such as document QA, Retrieval-Augmented Generation (RAG), and log summarization. In such scenarios, input tokens dominate, and the cache hit ratio is often high. The evaluation focus should be on input price, cache hit price, and cache write amortization. If system prompts and retrieval templates are stable, the cost reduction from cache hits can be very significant; if the context changes every time, cache write costs need to be included in the amortization calculation.
The second category is the “Short Input, Long Output” scenario, such as code generation, proposal writing, and structured expansion. The output price has a greater impact on total cost, and long outputs will amplify latency. In this case, do not just look at low input-side prices; evaluate output price, maximum output limits, and retry rates. If the model requires multiple retries or post-processing, the actual cost will be higher than the theoretical single-run cost.
The third category is the “Batch Offline” scenario, such as data annotation, offline evaluation, and content cleaning. These tasks are insensitive to latency, allowing prioritization of batch discounts and queue scheduling. When comparing, calculate the discounted per-task cost, factoring in failed retries, storage, and scheduling overhead. Batch discount isn’t the only factor; throughput stability, result consistency, and task orchestration complexity also affect the Total Cost of Ownership (TCO).
The fourth category is the “Low-Latency Interactive” scenario, such as online customer service and real-time assistants. High-speed tiers might carry a premium but can reduce timeouts and retries. The cost model should include timeout rates, retry rates, and concurrent resource costs. If a low-priced model causes more retries or longer tail latency, its actual cost might be higher than the surface price suggests.
5. Selection and Governance Practices
After establishing the cost comparison table, it is recommended to implement governance. First, monitor token distribution: log input, output, cache hit, and cache write tokens by business line, model, and interface to form weekly trends. Second, set budget alerts: set cost thresholds per project or tenant, triggering investigations upon abnormal growth. Third, establish routing policies: route simple tasks to low-cost tiers, complex tasks to high-quality tiers, and batch tasks to offline queues. Fourth, conduct A/B evaluations: cost reductions shouldn’t come at the expense of unacceptable quality; track accuracy, manual review rates, and user feedback simultaneously.
Fifth, regularly recalculate pricing tables. Public prices, discount rules, and tiers may change. The cost model should be parameterized to avoid hardcoding prices into business code. Sixth, differentiate “theoretical cost” and “billed cost”: theoretical models are for selection, while bills are for calibration; when discrepancies are large, check cache hit statistics, batch discount applicability, and retried or failed requests.
6. Common Pitfalls and Conclusion
Common pitfalls include: comparing only input prices while ignoring output costs; treating the cache hit price directly as the price for all inputs; ignoring cache write amortization; applying batch discounts to all requests; substituting monthly bills with single-request costs; and comparing directly under different token metrics. The key to avoiding these pitfalls is unified metrics, parameterized modeling, and calibration with real loads.
In summary, multi-model API cost comparison should follow the path of “fielding pricing tables — profiling business loads — formulating cost models — regularizing governance.” Public pricing tables provide billing parameters, but actual costs are determined by business loads and invocation strategies. Only by calculating input, output, cache, batch, and tier factors within the same model can you achieve explainable, reproducible, and actionable selection conclusions.
