Multi-Model API Cost Analysis and Selection Guide
A practical framework for comparing multi-model API costs beyond headline input prices. This guide covers cache hits, cache writes and storage, peak and off-peak pricing, batch discounts, output-token costs, request limits, and mutually exclusive pricing benefits. It also provides anonymized pricing examples, estimation formulas, a selection workflow, optimization strategies, and pseudocode for validating costs against real usage.
Multi-Model API Cost Analysis and Selection Guide
Procurement decisions for large language model APIs are often driven by a simple input/output price table. In practice, however, the final bill depends on several additional factors: cache hit rates, cache writes and storage, peak and off-peak pricing, batch discounts, request-length limits, the proportion of output tokens, and how requests are distributed over time.
Looking only at the input price can produce misleading conclusions. Focusing only on the cost of an individual call may also overlook storage charges that accumulate hourly or rules that prevent batch discounts and cache discounts from being combined.
This article uses four anonymized pricing samples to demonstrate a reusable cost-analysis method. The samples are intended only to illustrate billing structures and estimation techniques, not to recommend any specific service. Actual prices, limits, and discount rules should always be verified against the applicable contract and billing documentation.
1. Why You Should Look Beyond the Price per Million Tokens
API pricing is usually not a single linear function. Input tokens may be divided into cache hits and cache misses, while output tokens are often priced considerably higher. Some pricing models use time-of-use rates, with peak prices that are twice the off-peak rates. Others offer discounts for batch workloads, but those discounts may not be combinable with contextual caching.
Some services charge for cache writes based on a five-minute or one-hour TTL and continue to charge for cache storage by the million tokens per hour.
If a team estimates cost using only:
input price × input volume + output price × output volume
it may miss three major cost categories:
- Cache-write and cache-storage charges
- Higher prices during peak periods
- The opportunity cost created when batch and cache discounts are mutually exclusive
A reliable estimate should include at least the following variables:
- Uncached input tokens
- Cache-hit input tokens
- Output tokens
- Cached tokens
- Cache-storage hours
- Cache-write tokens and TTL
- The pricing tier determined by the request completion time
- Whether batch pricing is enabled
- Whether batch and cache discounts are mutually exclusive
Only after these variables are modeled can a simple price comparison become a realistic scenario-based cost comparison.
2. Breaking Down the Billing Dimensions
2.1 Separate Cache Hits from Cache Misses
Cache-hit pricing is often substantially lower than uncached input pricing. The effective input price can be expressed as:
Effective input price =hit rate × cache-hit price+ (1 - hit rate) × cache-miss price
The hit rate should be measured from actual prefix reuse rather than estimated from the theoretical maximum.
System prompts, tool definitions, fixed knowledge segments, and stable prefixes in multi-turn conversations are common sources of cache reuse. However, multi-tenant workloads, frequently changing templates, and dynamically assembled long contexts can significantly reduce the hit rate.
At a minimum, estimates should include conservative, expected, and optimistic hit-rate scenarios.
2.2 Output Is Frequently Underestimated
Output tokens are often priced several times higher than uncached input tokens. For workloads that generate long answers, code, or reports, output costs may exceed input costs.
Practical ways to control this cost include:
- Limiting output length
- Using structured-output constraints
- Setting a reasonable maximum output-token limit
- Removing repetitive explanations
The response token count must be included in the estimate; measuring only the request-body size is insufficient.
2.3 Cache Writes and Cache Storage
Caching is not free acceleration. Some pricing models charge for cache writes according to the TTL, with different rates for five-minute and one-hour retention. Cache storage may also be billed by the million tokens per hour.
When the cache is large or retained for a long period, storage charges can accumulate significantly.
To determine whether caching is worthwhile, compare:
Input cost saved through cache hits
with:
Cache-write cost + cache-storage cost
A short TTL may suit frequent reuse over a brief period, while a longer TTL may be appropriate for stable prefixes. Both approaches should be validated using actual reuse frequency.
2.4 Peak and Off-Peak Pricing
In time-of-use pricing models, specified weekday periods are charged at peak rates, while other periods—including weekends and holidays—are charged at off-peak rates. In some examples, the off-peak price is half the peak price.
A key rule is that pricing is determined by the time when the request completes. A long-running request that crosses a peak/off-peak boundary may be assigned to the tier associated with its completion time.
Batch processing, offline summarization, and data-cleaning workloads that are not latency-sensitive can be scheduled during off-peak periods. Real-time interactive workloads should be budgeted using peak pricing.
2.5 Batch Discounts and Exclusivity Rules
Batch pricing is useful for workloads that can tolerate additional latency, but it cannot always be combined with caching discounts.
Sample C explicitly illustrates a rule in which batch discounts and contextual-cache discounts are mutually exclusive. The correct approach is not to assume that the two discounts can be stacked. Instead, calculate both paths:
- Path 1: Apply batch pricing
- Path 2: Apply cache-hit pricing plus cache-write and storage charges
Then select the less expensive option.
Sample D offers batch pricing at 60% of the standard rate. This option must also be compared with the cache-based alternative.
3. Anonymized Pricing Samples
The following four samples are anonymized and are provided only to demonstrate billing structures. Prices are in currency units per million tokens unless otherwise noted.
Sample A: Time-of-Use Pricing and Cache Hits
| Tier | Input: Cache Hit | Input: Cache Miss | Output |
|---|---|---|---|
| Lightweight · Off-peak | 0.02 | 1 | 4 |
| Lightweight · Peak | 0.04 | 2 | 8 |
| Enhanced · Off-peak | 0.15 | 4.5 | 13.5 |
| Enhanced · Peak | 0.30 | 9.0 | 27.0 |
Time-of-use rules: Weekdays from 9:00–12:00 and 14:00–18:00 are peak periods. All other times, including weekends and holidays, are off-peak periods charged at half the peak rate. Pricing is determined by request completion time.
Sample B: Cache Hits and Cache Storage
| Tier | Input | Output | Cache Hit | Cache Storage (per million tokens/hour) |
|---|---|---|---|---|
| Lightweight | 0.80 | 2.70 | 0.16 | 0.017 |
| Standard | 3.00 | 15.00 | 0.60 | 0.017 |
| Enhanced | 6.00 | 30.00 | 1.20 | 0.017 |
Sample C: Mutually Exclusive Batch and Cache Discounts
| Tier | Input | Output | Notes |
|---|---|---|---|
| Standard | 2.4 | 9.6 | Standard mode only; batch is half price; contextual caching is discounted, but the two discounts are mutually exclusive |
| Flagship | 12 | 36 | Single request ≤ 1M tokens; batch is half price; contextual caching is discounted, but the two discounts are mutually exclusive |
Sample D: Cache-Write TTL and Batch Pricing
| Tier | Input | Output | Cache Hit | Cache Write: 5 min | Cache Write: 1 hour |
|---|---|---|---|---|---|
| Standard | 20 | 100 | 2 | 20 | 40 |
| Enhanced | 13 | 68 | 1.3 | 13 | 26 |
| Enhanced High-Speed | 26 | 136 | 1.3 | 26 | 52 |
The batch discount is 60% of the standard price.
4. Cost Estimation Models and Formulas
The base cost of a request can be represented as:
C = (Tin_miss / 1e6) × P_in_miss+ (Tin_hit / 1e6) × P_in_hit+ (Tout / 1e6) × P_out+ (C_cache / 1e6) × P_store × H+ (Twrite / 1e6) × P_write
Where:
Tin_miss,Tin_hit,Tout,C_cache, andTwriteare token countsPrepresents the corresponding unit priceHis the number of cache-retention hours
When batch pricing is enabled, calculate the base input and output cost first, then apply the batch factor.
If batch and cache discounts are mutually exclusive, calculate the batch plan and cache plan separately and choose the lower-cost option.
For time-of-use pricing, P must be selected according to the pricing tier associated with the request completion time.
Example 1: Sample A
Assume a lightweight off-peak request with:
- 1 million input tokens
- A 60% cache-hit rate
- 500,000 output tokens
Input cost:
0.6 × 0.02 + 0.4 × 1 = 0.412
Output cost:
0.5 × 4 = 2
Total cost:
2.412
If the same request completes during a peak period, the prices double and the total becomes approximately:
4.824
Example 2: Sample D
Assume a standard-tier request with:
- 1 million uncached input tokens
- 500,000 output tokens
The base cost is:
20 + 50 = 70
With a 60% batch discount:
70 × 0.6 = 42
If the input has a 50% cache-hit rate, the input cost becomes:
0.5 × 2 + 0.5 × 20 = 11
Adding the output cost:
11 + 50 = 61
Cache-write and storage charges must then be added, and the resulting cache plan should be compared with the batch plan costing 42.
5. Important Rules That Are Easy to Miss
Pricing may be based on completion time.
Long-running requests that cross pricing periods may be charged according to the tier active when they complete. Scheduling systems should reserve sufficient time to avoid assuming that a request started during an off-peak period will necessarily receive off-peak pricing.Cache storage accumulates hourly.
The larger the cache and the longer its TTL, the more important storage costs become.Cache writes are not free.
The write price may approach or equal the regular input price. If the cache is not reused often enough, caching can increase total cost.Batch and cache discounts may be mutually exclusive.
Batch workloads should be estimated using both pricing paths rather than assuming that the discounts can be stacked.Output tokens can be a major cost multiplier.
Long responses may dominate the total bill.Per-request length limits affect batching.
Very long contexts may not qualify for certain discounted pricing paths or may require different batch-splitting strategies.Cache-hit rates fluctuate.
Template changes, tenant isolation, and version releases can all affect reuse. Budget models should include a safety margin.
6. A Practical Selection Workflow
A six-step process can make model selection more systematic:
Step 1: Profile the Workload
Define:
- Latency SLAs
- Throughput requirements
- Context-length requirements
- Expected output length
- Tolerance for batch processing delays
Step 2: Sample Real Requests
Measure:
- Input and output token distributions
- Reusable prefixes
- The proportion of requests during peak and off-peak periods
Step 3: Build a Unified Pricing Model
Represent each candidate price list in a common structure that distinguishes:
- Cache-hit and cache-miss prices
- Cache storage
- Cache writes
- Batch factors
- Exclusivity rules
Step 4: Estimate Multiple Scenarios
At a minimum, cover combinations of:
- Peak and off-peak periods
- Cache-hit rates of 0%, 50%, and 80%
- Batch enabled and disabled
- Short and long cache TTLs
Step 5: Replay Real Traffic
Use token counts from real requests to replay the billing model and compare the estimate with actual invoices or usage records.
Step 6: Monitor and Regress
Track continuously:
- Cache-hit rate
- Cache-storage volume
- Batch-processing ratio
- Peak/off-peak traffic ratio
- Cost percentiles per request
Set budget alerts for abnormal changes.
7. Cost Optimization Strategies
Optimization should target the actual cost drivers:
- Compress prompts and stabilize reusable prefixes to improve cache hits.
- Route simple and complex workloads to different model tiers, with quality evaluation as a safeguard.
- Schedule offline tasks during off-peak periods.
- For latency-insensitive workloads, compare batch pricing with cache-based pricing.
- Control output length and use structured responses where appropriate.
- Set a suitable cache TTL to avoid unnecessary long-term storage charges.
- Attribute costs and apply quotas by tenant, project, and API interface.
- Alert on unusually long requests, duplicate requests, and sudden increases in cache misses.
Every optimization should be validated through replay against real token usage and billing rules rather than by comparing theoretical unit prices alone.
8. Illustrative Cost-Estimation Pseudocode
type Price = {inMiss, inHit, out,cacheStoreHour, cacheWrite5m, cacheWrite1h,batchFactor}function tokenCost(tokens, pricePerMillion) {return tokens / 1_000_000 * pricePerMillion}function estimate(req, p, opt) {missCost = tokenCost(req.inMiss, p.inMiss)hitCost = tokenCost(req.inHit, p.inHit)outCost = tokenCost(req.out, p.out)cacheStore = tokenCost(req.cacheTokens, p.cacheStoreHour) * req.hourswritePrice = req.ttl == 5 ? p.cacheWrite5m : p.cacheWrite1hcacheWrite = tokenCost(req.cacheWriteTokens, writePrice)cachePlan = missCost + hitCost + outCost + cacheStore + cacheWritebatchPlan = (missCost + hitCost + outCost) * p.batchFactorif (opt.batch && opt.cacheDiscount && opt.exclusive) {return min(batchPlan, cachePlan)}if (opt.batch) return batchPlan + cacheStore + cacheWritereturn cachePlan}
9. Conclusion
The key to comparing multi-model API costs is not memorizing a single price table. It is building a closed loop that connects workload profiling, billing rules, scenario estimation, and monitoring-based validation.
Input cache-hit rates, output length, peak/off-peak traffic distribution, cache writes and storage, and mutually exclusive batch discounts can all materially change the final bill.
Use anonymized samples to practice the breakdown, apply a consistent estimation formula, and calibrate the model with real token-level request replays. This approach reduces the risk of being misled by a single unit price and supports model-selection decisions that are explainable, measurable, and reviewable across performance, latency, and budget constraints.
