LLM API Billing Breakdown: Why Identical Token Prices Can Cause API Bills to Differ by Half
This article explains why teams using the same LLM model at the same published token price can receive very different API bills. It breaks down the main billing dimensions, including input and output token metering, prefix cache hits and writes, batch versus real-time channels, subscription plans, off-peak discounts, free quotas, and regional pricing. It also provides a reproducible calculation example, scenario-based selection guidance, and an engineering checklist for keeping LLM API costs under control.
1. The question: same unit price, why are the bills so different?
When choosing a large language model service, many teams start by opening the pricing pages of several model platforms and comparing input and output prices side by side. They often reach a seemingly reasonable conclusion: the pay-as-you-go price for the same model is identical across platforms, so it does not matter which one they choose.
That conclusion is true at the list-price level. Model providers publish official pricing for a model, and hosting or distribution platforms usually inherit the same pay-as-you-go rates. This is why the price sheets for the same model look almost identical across platforms. However, identical list prices do not mean identical settlement amounts. An API bill is not a simple multiplication of usage by unit price. It is the result of a set of billing rules applied to your calling pattern. In simplified form:
Monthly bill = (usage per channel × corresponding unit price, adjusted for region, free quota, subscriptions, and other factors) − plan deductions
The calling pattern is determined by your business, while the unit price and adjustments are determined by platform rules. In practice, teams with similar usage volumes can still differ by half in their bills because of the answers to four questions:
- Can the workload be split into offline calls and routed through a batch channel?
- How much of each request is a reusable prefix, and what is the cache hit rate?
- Is the traffic curve smooth and predictable enough for a subscription plan?
- Where are the stored data and the business deployed, and are cross-region or cross-border calls required?
Therefore, “which platform should we choose?” should not be the first question. The better approach is to profile your business calls first, then review each platform’s billing rules, and finally run the same usage data through the calculation formula. This article follows that approach.
2. Break down the bill: a usage-driven cost formula
No matter how a platform displays prices, inference cost can ultimately be broken down into “token metering × unit price.” Five billing dimensions deserve particular attention in real projects.
2.1 Token metering: input and output are priced separately
Most LLM services meter usage in tokens, the smallest text chunks processed by the model. In Chinese, one Chinese character usually corresponds to about 1–2 tokens, depending on tokenization. Structured content such as code, JSON, and tables usually consumes noticeably more tokens than natural-language text of the same length. Input and output are generally priced separately, and the output price is often several times the input price. As a result, long generated outputs are a cost that is easy to underestimate.
The same token can have different prices depending on its role:
- Input without a cache hit: billed at the standard input price.
- Input with a cache hit: billed at a cache-hit price that is significantly lower than the standard input price.
- Input written to cache: some services charge separately for cache writes, usually close to the standard input price.
- Output tokens: billed at the output price.
2.2 Context caching: repeated prefixes are the biggest cost variable
Applications with fixed prompts, long-document summarization, and multi-turn conversations often carry a large, completely identical prefix with every request. Most inference services support prefix caching: the same prefix does not need to be recomputed, and the cached portion is billed at the cache price. These services usually require the cached portion to match character for character from the beginning of the request, and they set a cache time-to-live. If the cache expires and has to be rewritten, write fees may occur.
As a result, the cache hit rate becomes one of the largest cost variables outside the contract terms. For the same application, increasing the cache hit rate from 30% to 70% can reduce input-side cost by more than half.
2.3 Batch and asynchronous channels: trading latency for price
In addition to the real-time channel, most platforms offer asynchronous batch submission. A batch of requests is submitted together and returned within an agreed time window, usually at a lower unit price than real-time calls. This is suitable for offline data processing, bulk text processing, bulk labeling, and model evaluation. The trade-off is that it cannot meet interactive scenarios that require second-level responses.
2.4 Subscription plans and off-peak discounts
In addition to pay-as-you-go billing, some platforms offer monthly Token subscription plans or time-window-based off-peak discounts. In essence, they exchange committed usage for a lower price. For stable and predictable workloads, the marginal cost can be significantly lower than pure pay-as-you-go. However, once usage exceeds the plan or falls outside the discount window, the excess may return to standard pricing or even a higher tier.
2.5 Regional differences and free quota
The price for the same model may vary by region, and cross-region calls can add data transfer costs and extra latency. In addition, new models often come with free quotas or limited-time promotions during the early launch period. Once usage crosses the quota, standard billing resumes. These costs may not appear in a unit-price comparison table, but they still end up on the bill.
3. The same usage record under four settlement rules: a reproducible estimate
To show the impact of rule differences, the following example constructs a typical usage profile and calculates the result under different assumptions. The usage values below are in millions of tokens. The unit prices are hypothetical and are used only to demonstrate the calculation logic. They do not represent real quotes:
| Item | Usage | Assumed unit price (CNY per million tokens) |
|---|---|---|
| Input tokens | 3,000 | Cache miss: 20; cache hit: 2; cache write: 20 |
| Output tokens | 600 | 100 |
Assume that the batch channel price factor is 0.6 relative to the real-time channel, and the off-peak discount factor is 0.5.
| Settlement rule | Calculation note | Assumed monthly bill (CNY) |
|---|---|---|
| ① Pure pay-as-you-go, no cache | 3000 × 20 + 600 × 100 | 120,000 |
| ② 50% prefix-cache hit rate | Input is split into hit and miss portions | 87,000 |
| ③ On top of ②, 60% of input and output moved to the batch channel | Billable portion × 0.6 | ~59,880 |
| ④ On top of ③, half of the deferrable calls scheduled during off-peak hours | Corresponding portion × 0.5 | ~44,910 |
With the same usage, the bill falls from 120,000 CNY to about 44,910 CNY, a reduction of more than 60%. The unit prices on each platform are identical; the difference comes entirely from the combination of rules. This chain can be written as a platform-agnostic calculation script. To compare settlement methods, replace the price table and usage profile with real data:
from dataclasses import dataclass, fieldfrom typing import OptionalPRICE = {# Hypothetical values for demonstration only. Replace with live platform prices."realtime": {"input": 20.0, "output": 100.0, "cache_hit": 2.0, "cache_write": 20.0},"batch": {"input": 12.0, "output": 60.0, "cache_hit": 1.2, "cache_write": 12.0},"offpeak": {"input": 10.0, "output": 50.0, "cache_hit": 1.0, "cache_write": 10.0},}@dataclassclass Usage:"""Monthly usage profile, in millions of tokens (MPT)."""input: float = 0.0 # Total input tokensoutput: float = 0.0 # Total output tokenshit_ratio: float = 0.0 # Share of input that hits the cachewrite_ratio: float = 0.0 # Share of input written to cachegeo_factor: float = 1.0 # Regional adjustment factorfree_quota: float = 0.0 # Remaining free quota (MPT, converted at standard input price)discount: float = 1.0 # Settlement discount for batch, off-peak, and similar options@dataclassclass Plan:"""Subscription plan; unit_price is in CNY per MPT."""included: floatused: float = 0.0over_price: Optional[float] = None # Excess-unit price; defaults to pay-as-you-go pricingunit_price: Optional[float] = None # Effective plan price, used for comparing with non-subscription optionsdef monthly_cost(u: Usage, price: dict, plan: Optional[Plan] = None) -> float:"""Calculate the monthly bill (CNY) using metering × unit price × adjustments."""hit = u.input * u.hit_ratiowrite = u.input * u.write_ratiomiss = max(u.input - hit - write, 0.0)mpt = miss + write + u.output # Plan allowance is deducted using all billable tokensif plan is not None and plan.over_price is None:covered = min(mpt, max(plan.included - plan.used, 0.0))over = max(mpt - covered, 0.0)# The plan is prepaid, so covered usage is not charged againreturn (plan.included * plan.unit_price + over * price["input"]) * u.geo_factor# Non-subscription path, or subscriptions that require explicit excess rulescost = (miss * price["input"] + hit * price["cache_hit"]+ write * price["cache_write"] + u.output * price["output"])cost *= u.discount * u.geo_factorreturn max(cost - u.free_quota, 0.0)if __name__ == "__main__":plan = Plan(included=4000, unit_price=9.0) # Assumed: 4,000 MPT allowance, equivalent to CNY 9 per MPTbase = dict(input=3000, output=600, hit_ratio=0.5, discount=0.6)scenarios = {"① Pure pay-as-you-go": dict(base, hit_ratio=0.0, discount=1.0),"② Prefix cache": dict(base, discount=1.0),"③ + Batch channel": base,"④ + Off-peak window": dict(base, discount=0.3),"⑤ Subscription plan": dict(base, hit_ratio=0.0, discount=1.0),}for name, params in scenarios.items():p = plan if name == "⑤ Subscription plan" else Noneprint(f"{name}: {monthly_cost(Usage(**params), PRICE['realtime'], p):,.0f} CNY")
The fifth scenario in the script shows the subscription-plan path. Assume a subscription plan provides a fixed monthly allowance of 4,000 million tokens at an effective CNY 9 per million tokens, while the same workload would cost 120,000 CNY under pure pay-as-you-go without caching or discounts. In that case, a subscription plan is more suitable for a workload that is steady, does not use caching, and is almost entirely real-time. If only part of the calls are suitable for the plan, split the workload into two Usage objects and use plan.over_price to specify the excess-unit price, then settle them separately and sum the results.
Pay special attention to the input-to-output ratio. A price sheet only gives a price per million tokens, but there is a multiple-times price gap between input and output. If output accounts for a large share of total tokens, or if the cache hit rate is low, a pay-as-you-go bill can quickly be pushed up by output and uncached input. Pay-as-you-go can only be on par with a subscription plan for workloads where output accounts for a small share and caching is largely ineffective. This is the typical cause of “four platforms have identical unit prices, but monthly bills differ by half”: one side lowers the effective unit price with caching and batch processing, while the other pays the standard price for full usage.
4. Match billing methods to business scenarios: five typical cases
4.1 Scenario 1: The model can be replaced by outcome, with cost as the priority
If the business has no strong dependency on a specific model—for example, text classification, extraction, or summarization—you can compare output quality across different tiers in the same model family using a small evaluation set. If the acceptance metrics are met, choose the tier with the lower unit price. Unit prices across tiers may differ by several times, and this factor usually has a greater impact than differences between channels. A practical approach is to build a candidate table with a quality threshold and a cost ranking: remove options that fail the threshold, then compare total cost among the remaining options.
4.2 Scenario 2: Offline batch tasks
Tasks such as log analysis, bulk document summarization, data cleaning and labeling, and offline evaluation usually do not need second-level responses. These workloads should prioritize the batch channel, and object storage can be used for task distribution and result collection when necessary. When evaluating them, calculate three items: the batch price discount, the deadline for task completion, and the extra cost caused by retries. If the batch channel’s return window conflicts with the business deadline, use a hybrid approach of “real-time guarantee plus batch gain,” sending only deferrable calls to batch.
4.3 Scenario 3: Real-time online, with the main model already locked
For core paths that are latency-sensitive and cannot easily change models because of quality or compliance requirements, consider a subscription plan plus pay-as-you-go for excess usage. The plan allowance should be estimated from the P50 and P95 usage percentiles over the past three to six months. If it is too low, you will frequently exceed it; if it is too high, the allowance will be wasted. At the same time, identify calls that can be deferred, such as non-real-time notification copy, offline reports, and asynchronous summaries, and move them to off-peak windows or the batch channel.
Using the assumptions in this article, when 60% of calls can be moved to off-peak windows (scenario ④ in the script), the effective discount drops from 0.6 to 0.3, and the bill is reduced by about 64% compared with pure pay-as-you-go. Generalizing, there is a simple estimation relationship:
Overall savings ratio ≈ share of migratable calls × (1 − discount factor of the migratable portion)
This also explains why, at the same unit price, teams that design their call schedule well may have bills that are only half those of other teams.
4.4 Scenario 4: Existing business already runs on a cloud
If data, services, and operations are already deployed on a cloud platform, connecting to model services on the same platform can avoid the extra latency and data transfer costs of cross-region calls. Token prices alone may be identical across platforms, but after including hidden costs such as transfer, retries, and failover, same-region access is often cheaper. For example, a team whose existing data and business systems already run on Baidu AI Cloud can prioritize evaluating direct model-service access within the cloud to reduce cross-network overhead and potential failure points.
4.5 Scenario 5: Multi-region and overseas business
Cross-border or cross-region calls may face different regional pricing, compliance reviews, and data-export requirements. Include the region dimension in the selection matrix. List latency, price, and compliance requirements together, and compare cost only after compliance is satisfied.
4.6 Quick selection guide
| Business characteristic | Preferred billing method | Key action |
|---|---|---|
| Meets quality requirements; model not locked | Pay-as-you-go on a lower-priced tier | Quality-threshold evaluation |
| Can tolerate minute-level latency | Batch channel | Task orchestration and retry control |
| Real-time, steady traffic, model locked | Subscription plan + pay-as-you-go for excess | Estimate allowance by P50/P95 |
| Real-time and highly variable | Pay-as-you-go first + off-peak for deferrable calls | Call-pool scheduling |
| Existing business on a cloud | Same-region access | Include transfer and operations costs |
5. Engineering implementation: build cost control into the system
5.1 Build your own metering; do not wait for the monthly bill
Platform usage details are useful for reconciliation, but cost governance must happen at the moment a request is made. Log structured data for every request in a gateway or wrapper layer. At minimum, record:
- Model name and version;
- Input, output, cache-hit, and cache-write token counts;
- Channel (real-time, batch, off-peak) and region;
- Unit price and discount applied to the request;
- Retry count and final status.
With this data, you can break down cost by business line, scenario, and tenant, calculate budget consumption in real time, and automatically downgrade or rate-limit when a threshold is reached, instead of discovering problems only when the monthly bill arrives.
5.2 Make prompts “cache-friendly”
To improve prefix-cache hit rates, organize prompts with stable content first and volatile content last:
# ── Stable prefix: keep character-for-character unchanged to hit the prefix cache ──[System instructions and role settings][Tool or function definitions][Fixed knowledge snippets / few-shot examples]# ───────────── Volatile section ─────────────[Current user input][Request time, session random ID, and other volatile fields, always at the end]
Supporting practices include moving volatile fields such as timestamps, random IDs, and temporary request identifiers out of the prefix; version-managing system instructions to avoid frequent changes; and setting a reasonable cache TTL to balance hit benefits against write overhead. The benefit is greatest when many requests share the same prefix. If every conversation round uses the complete history as the prefix, the hit rate usually decreases as rounds increase. In that case, use session summaries or similar methods to compress history.
5.3 Separate calls into pools: real-time pool and deferrable pool
def dispatch(req):if req.interactive: # User is waiting for the responsepool = "realtime"elif req.deadline_hours >= 1: # Can tolerate a longer completion timepool = "batch"else: # Near-real-time, but does not require second-level responsepool = "offpeak"return submit(req, pool=pool, max_retries=2, timeout_s=30)
Pair this with monitoring that distinguishes two failure types: retryable throttling and timeouts, and non-retryable parameter errors. The retry limit and backoff strategy must be set explicitly. Otherwise, a single incident can multiply call volume several times.
5.4 Use lightweight controls to block low-value calls
Not every request needs an LLM. If a question can be answered directly with rules, cached results, or retrieval results, return it at the gateway layer. Also set an upper limit on generated length to prevent output-side costs from spiraling out of control.
5.5 Keep a replaceable model access layer
Wrap model calls behind a unified internal interface, and manage model names, parameters, and billing conventions through configuration. This allows you to switch between different tiers or platforms as the business changes, and to recalculate costs quickly when pricing or rules change, avoiding lock-in to a single access method.
5.6 Migration and canary release: include switching costs in the calculation
When an older model version stops accepting new users and is gradually retired, assess the quality difference and price changes of the new version early. Complete the migration with a small-traffic canary and dual-track comparison to avoid concentrated switching that causes quality fluctuations and bill spikes.
6. Common misconceptions
- Comparing only unit prices, not rules. On platforms with identical unit prices, any one of batch discounts, cache billing, subscription plans, or regional factors can produce a double-digit percentage cost difference.
- Ignoring cache writes and expiration. If prefixes change frequently or caches are repeatedly rewritten after expiration, caching may not save money and can even cost more than not enabling it.
- Estimating plan allowances too coarsely. Once a plan allowance is exceeded, excess usage is billed at standard rates. During promotions or traffic spikes, this can end up more expensive than pay-as-you-go.
- Ignoring input/output structure. When output share is high or cache hit rate is low, any discount on the input side has limited effect.
- Looking only at model cost, not integration and operations cost. Hidden costs such as cross-region calls, multi-provider authentication and monitoring, and compliance audits can sometimes exceed the token cost itself.
7. Conclusion: an actionable checklist
“Same unit price, different bills” is not contradictory. A unit price is only a number on a price sheet. A bill is the result of rules and calling patterns working together. When implementing, check the following items one by one:
- Collect daily and scenario-level call volumes and input/output token distributions for the past three months;
- List the billing rules of each platform in the target region, including batch discounts, cache billing, subscription plans, off-peak discounts, and free quotas;
- Plug the same usage data into the calculation script and compare settlement methods side by side;
- Choose an appropriate billing method for each business scenario, and design a pool-based scheduling strategy for real-time and deferrable calls;
- Build your own metering and budget alerts so cost governance happens at the moment a request is made.
After completing these five steps, “it is the same no matter which platform you choose” becomes a question that can be answered quantitatively: the difference is not in the platform, but in whether you apply the billing rules in the right places.
