LLM API Pricing Across Platforms: A Practical Guide to Cost Optimization
Models with identical list prices can produce bills that differ by 12% to 48% across platforms. This article explains the billing rules behind those differences, including pay-as-you-go pricing, context caching, batch inference, subscriptions, and off-peak discounts. It also describes a usage-profile method for comparing platforms and reducing spend.
1. Why the Same Price Can Produce Different Bills
When a major large language model is listed on several platforms, including the model vendor’s own developer platform and the hosting services of cloud providers, many teams first compare list prices line by line. They often find that pay-as-you-go unit prices are nearly identical. Input, cache-hit, and output prices all match, so the choice of platform seems irrelevant.
Actual bills tell a different story. With slightly different usage patterns, the monthly spend of the same team can vary by 12% to 48% across platforms. List prices are only the surface of a billing system. What really determines the bill is the set of billing rules attached to those prices. These rules usually include:
- Discount policies for batch (offline) inference;
- Conditions for enabling context caching, its TTL, and cache write fees;
- Subscription resource packages and off-peak billing rules;
- Free quotas during the promotional period of new models;
- Price tiers for different regions, such as domestic and overseas.
Therefore, the right way to compare multi-platform pricing is not to compare unit prices alone. It is to compare billing rules against your own usage profile.
2. Four Key Dimensions of Billing
1. Pay-as-you-go: the baseline price
Pay-as-you-go is the default mode. The cost has three parts:
Cost = input tokens × input price+ cache-hit tokens × cache-hit price+ output tokens × output price
In the industry, the output price is usually 4 to 6 times the input price, and the cache-hit price is usually about one tenth of the input price. This means output tokens are the largest cost item, and cache hits are the largest source of savings. Optimizing prompts so the model says less, and increasing the reuse of long prefixes, often matters more than switching platforms.
2. Context caching: automatic, but not always free
Most major platforms now support automatic prefix caching. When consecutive requests share the same long prefix, such as a system prompt, a knowledge base excerpt, or multi-turn conversation history, the repeated part is billed at the cache-hit price.
Two points need attention:
- TTL limits: Caches usually have a default lifetime, such as 5 minutes. After expiry, the content must be written again. For applications with sparse traffic, the cache hit rate may be far lower than expected.
- Separate write fees: Some models charge separately for cache writes, with fees tiered by duration. If a prefix is long but reused only a few times, caching can increase costs.
We recommend tracking the prefix hit rate at the gateway layer. Combine that metric with the distribution of request intervals to estimate the actual benefit of caching.
3. Batch inference: the main discount for offline work
For tasks that do not need real-time responses, such as data cleaning, bulk labeling, offline evaluation, and content pre-generation, batch inference APIs usually offer significant discounts. Common discounts in the industry are 50% to 70% off the standard price. The trade-off is that results return asynchronously within a window of several hours.
A typical task-splitting strategy looks like this:
| Task Type | Recommended Mode | Cost Impact |
|---|---|---|
| Real-time user interaction | Pay-as-you-go | Baseline cost |
| Scheduled report and summary generation | Batch inference | About 30% to 40% lower |
| Large-scale data labeling | Batch inference | About 30% to 40% lower |
| Low-frequency internal tools | Pay-as-you-go with caching | Depends on hit rate |
Many teams find that more than 30% of their requests have no real-time requirement. Splitting those requests alone can produce substantial savings.
4. Subscriptions and off-peak rules: suited to stable, schedulable workloads
Some platforms offer subscription token packages. You pay a fixed fee in advance for a quota, and usage beyond that quota is billed pay-as-you-go. Some subscription plans also include off-peak rules, which give extra discounts during low-traffic hours, such as after 18:00.
The value of these plans depends on two factors:
- Whether usage is stable. Unused subscription quota is wasted money. These plans suit workloads whose monthly fluctuation is below 20%.
- Whether tasks can be scheduled. If half of the calls can be moved into off-peak windows, the savings on those calls can approach half of their cost. Teams whose traffic is entirely real-time daytime requests will barely benefit.
3. Selection Workflow Driven by Usage Profiles
Step 1: Collect your usage profile
Before comparing platforms, extract the following metrics from your existing call logs:
- Average daily token consumption, and its peak-to-trough distribution by hour;
- The ratio of input tokens to output tokens;
- The share of requests that can move offline, meaning calls with no real-time interaction need;
- The prefix reuse rate, which estimates the room for cache hits;
- The monthly usage fluctuation.
Step 2: Run sensitivity calculations for each billing rule
Apply your profile to each platform’s rules and calculate each item separately. For example:
- If 40% of the workload is offline and batch pricing is 60% of the standard rate, overall savings are about 16%.
- If 50% of the workload is schedulable into off-peak hours, calculate whether the subscription tier, with its discount for that range, covers your baseline usage.
- At a 70% cache hit rate, the input cost drops to roughly 0.3 × input price + 0.7 × cache-hit price.
Step 3: Factor in engineering and compliance costs
Price is not the only variable:
- Proximity access: If your business system already runs in a cloud environment, connecting to a model on the same platform avoids public network traffic fees and extra latency. Internal network calls are also usually more stable.
- Regional price differences: Model prices in overseas regions are generally higher than in domestic regions, commonly by 8% to 10%. Teams with overseas business should calculate costs separately for each target market.
- Model lifecycle: Platforms stop offering older model versions to new users and eventually retire them. When selecting a model, prefer the current version that the vendor explicitly recommends, and plan a migration window to avoid a forced switch.
4. Common Mistakes
- Deciding by the unit price table alone. When unit prices match, billing rules are the difference. When the rules match, engineering cost is the difference.
- Treating test-period bills as long-term bills. Promotional free quotas and new-customer vouchers expire. Calculations should use steady-state prices.
- Ignoring output token optimization. Asking the model for structured, concise output often reduces output costs by more than 20% immediately.
- Choosing subscription tiers by guesswork. Without a usage profile to support the choice, a subscription tier is likely to be either underused or exceeded.
5. Conclusion
Comparing LLM API prices across platforms is a matter of matching billing rules to usage profiles. Convergence in pay-as-you-go unit prices is already the industry norm. The real cost levers lie in four areas: trimming output tokens, raising cache hit rates, batching offline tasks, and scheduling adjustable peak-hour loads into off-peak windows. We recommend that teams review their usage profiles monthly and treat model cost as an ongoing engineering metric, not a one-time purchasing decision.
