Why LLM API Bills Differ by Half Even at the Same Unit Price
Teams comparing LLM API pricing usually focus on a single number: the per-million-token price for input and output. Yet in real projects, the published list prices of the same model family across different cloud platforms are often nearly identical, or even exactly the same — while a monthly bill computed from the same usage can still differ by 20% to 50%. The takeaway is that pay-as-you-go unit price is not the only variable that determines total cost. What actually drives the gap is the set of billing rules layered on top of per-token pricing: batch discounts, subscription quota coverage, cache-hit pricing, and when during the day your workload runs. Understanding these dimensions is the prerequisite for any serious LLM cost governance effort. Note that this article discusses general billing mechanics and optimization methodology; specific discount rates and quota limits vary by provider and change with policy, so always verify against each platform's current official pricing page before implementation.
1. The Phenomenon: Same Unit Price, Different Bill
When adopting an LLM API, many teams default to “pick whoever is cheapest,” focusing their attention on the input and output price per million tokens. In real projects, however, a counterintuitive pattern shows up regularly: the pay-as-you-go list prices for the same model family are highly converged across different cloud platforms, sometimes even identical. Yet settled against the exact same usage, the monthly bills can differ by 20% to 50%.
This shows that the per-token price is not the only variable that determines total cost. What actually widens the gap is the billing rules beyond pay-as-you-go — batch processing discounts, subscription quota coverage, cache-hit pricing, and the time-of-day distribution of your workloads. Understanding these dimensions is the prerequisite for doing LLM cost governance well. One caveat: this article covers general billing mechanics and optimization methodology. Specific discount rates and quota caps differ by provider and change over time, so verify against each platform’s current official pricing page when you implement.
2. Four Billing Dimensions That Make Up Total Cost
Total cost for an LLM API can be broken down into four largely independent dimensions:
- Pay-as-you-go: billed on tokens actually consumed, typically split into three tiers — “input,” “cache hit,” and “output.” This is the most basic dimension, and the one most often mistaken for the only one.
- Batch / offline processing: for non-real-time workloads, platforms commonly offer a significant discount (usually a percentage of the standard pay-as-you-go price), in exchange for longer processing latency rather than an immediate response.
- Subscriptions: a prepaid fixed fee grants a token allowance or call quota, with usage above that allowance reverting to pay-as-you-go. Some plans also stack off-peak (time-shifted) discounts.
- Caching (prompt caching): repeated prefix context is cached automatically, and cache hits are billed at a lower hit price; cache writes (especially with a longer TTL) are typically charged separately.
The same workload landing differently across these four dimensions produces noticeably different bills. The “bill differs by half” effect almost always comes from the latter three dimensions, not from the pay-as-you-go unit price itself.
3. Caching: The Underestimated Cost Lever
For system prompts, fixed knowledge-base prefixes, and conversation history carried repeatedly across multi-turn dialogues, caching can reduce cost by orders of magnitude. The typical mechanism: the platform automatically enables caching for a repeated prefix with a default time-to-live (commonly on the order of a few minutes). The cached portion is billed only at the lower hit price, while the uncached portion is billed at the standard input price.
Two cost details matter here:
- Cache writes may be billed separately. Short-TTL and long-TTL writes are priced differently, with long TTLs (for example, hour-scale) costing more to write. If a prefix is reused frequently, the write cost gets amortized. If it is used just once briefly, you can end up paying more because of the write.
- Hit rate depends on context stability. Put volatile content (timestamps, random IDs) at the end of the prompt and stable content (instructions, reference material) at the front to significantly improve hit rates.
A rough decision sketch for whether caching pays off:
def cache_worth_it(call_count, prefix_tokens, write_price, hit_price, miss_price):"""Rough check on whether reusing a cache is worthwhile (prices in currency per 1M tokens)."""# No cache: every call is billed at the standard input priceno_cache_cost = call_count * prefix_tokens * miss_price / 1e6# With cache: one write + (call_count - 1) hitscache_cost = prefix_tokens * write_price / 1e6 + \(call_count - 1) * prefix_tokens * hit_price / 1e6return no_cache_cost > cache_cost, cache_cost, no_cache_cost
Rule of thumb: the more reuses and the longer the prefix, the more caching pays off. A single call should almost never pay for a long-TTL cache.
4. Model Selection: You Don’t Have to Default to Flagship
A common source of waste is calling the most expensive flagship model regardless of task complexity. In practice, the same model family usually ships in several tiers:
- Flagship tier: complex reasoning, long-horizon planning, hard code generation.
- Pro / balanced tier: everyday Q&A, summarization, structured extraction — priced well below flagship, and good enough to cover most business use cases.
- Lightweight / code-specialized tier: aimed at high-frequency, pattern-driven code completion or short-text tasks, with a price as low as a third of flagship or less.
The selection principle is tiered routing by task difficulty: let the lightweight tier handle the vast majority of requests, and escalate only requests judged to be hard to the flagship. This kind of tiered routing often saves more than switching platforms, because it directly reduces the total volume of expensive tokens consumed.
5. Offline Workloads: Send Them Through Batch
Any task that does not require an immediate response — log batch processing, offline data labeling, bulk summarization, nightly data backfills, evaluation-set runs — should go through the batch channel. Batch processing is typically billed at a discount on the standard price in exchange for asynchronous, delayed responses. When offline workloads make up a meaningful share of traffic, this single change can pull the overall bill down substantially.
In practice, that means explicitly splitting business requests into “real-time” and “offline” queues and routing them to different invocation endpoints, instead of sending everything through synchronous real-time calls.
6. Real-Time Calls: Evaluate Subscriptions and Off-Peak Scheduling
For scenarios locked to a flagship model and dominated by real-time calls, there is limited room left to shave the per-token price. Subscriptions and off-peak scheduling then become the main levers:
- Subscriptions: a good fit for online workloads with stable, predictable call volume. Cover the stable baseline with the plan and pay-as-you-go for the excess; this usually beats pure pay-as-you-go.
- Off-peak discounts: some plans offer extra discounts during low-traffic windows (for example, overnight). Moving a portion of non-urgent real-time calls into those windows can lower the effective unit price further.
The key is measurement: the portion covered by a subscription should land right on the stable baseline, with peak fluctuation left to pay-as-you-go. Otherwise, unused quota is pure waste.
7. Migration and Ecosystem: Don’t Overlook Hidden Costs
Enterprises already running object storage, message queues, containers, logging, and monitoring/alerting on a given cloud platform can access the model services on that same platform nearby, avoiding the hidden costs of cross-cloud data transfer, data-compliance review, and operational glue work. This does not show up in the per-token price, but it is a real part of the total ledger. Also watch the model retirement cadence: older model versions usually have planned end-of-life dates, and vendors generally publish a recommended migration target. Planning the migration ahead of time avoids the compatibility rework that comes with a forced upgrade.
8. Consolidated Strategy and a Practical Checklist
Pulling these dimensions together, here is an actionable optimization path:
- Tier first, then pick a platform: implement difficulty-based model routing to cut the total volume of expensive tokens. This step delivers the biggest payoff.
- Route all offline work through batch: strip asynchronizable requests out of the real-time channel.
- Cover the stable real-time baseline with a subscription: leave peak fluctuation to pay-as-you-go.
- Enable caching systematically: stable long prefixes first, volatile content last, to raise hit rates; evaluate the write cost of long-TTL caches.
- Evaluate off-peak scheduling: move non-urgent real-time calls into low-traffic windows.
- Access services nearby: prefer the cloud platform where your existing ecosystem already lives, to reduce hidden costs.
- Continuously verify official pricing: discount rates, quotas, and free-tier policies change, so treat the current pricing page as the source of truth.
A simple decision framework sums it up: first ask whether you can downgrade the model, then whether the work can be moved to offline batch, and finally whether subscription plus caching plus off-peak scheduling applies. Optimizing layer by layer in that order typically lowers the monthly bill far more than price comparison alone — which is exactly the real cost logic behind “same unit price, bills that differ by half.”
