LLM API Billing Breakdown: Why Identical Token Prices Can Cause API Bills to Differ by Half

    This article explains why teams using the same LLM model at the same published token price can receive very different API bills. It breaks down the main billing dimensions, including input and output token metering, prefix cache hits and writes, batch versus real-time channels, subscription plans, off-peak discounts, free quotas, and regional pricing. It also provides a reproducible calculation example, scenario-based selection guidance, and an engineering checklist for keeping LLM API costs under control.

    1. The question: same unit price, why are the bills so different?

    When choosing a large language model service, many teams start by opening the pricing pages of several model platforms and comparing input and output prices side by side. They often reach a seemingly reasonable conclusion: the pay-as-you-go price for the same model is identical across platforms, so it does not matter which one they choose.

    That conclusion is true at the list-price level. Model providers publish official pricing for a model, and hosting or distribution platforms usually inherit the same pay-as-you-go rates. This is why the price sheets for the same model look almost identical across platforms. However, identical list prices do not mean identical settlement amounts. An API bill is not a simple multiplication of usage by unit price. It is the result of a set of billing rules applied to your calling pattern. In simplified form:

    Monthly bill = (usage per channel × corresponding unit price, adjusted for region, free quota, subscriptions, and other factors) − plan deductions

    The calling pattern is determined by your business, while the unit price and adjustments are determined by platform rules. In practice, teams with similar usage volumes can still differ by half in their bills because of the answers to four questions:

    1. Can the workload be split into offline calls and routed through a batch channel?
    2. How much of each request is a reusable prefix, and what is the cache hit rate?
    3. Is the traffic curve smooth and predictable enough for a subscription plan?
    4. Where are the stored data and the business deployed, and are cross-region or cross-border calls required?

    Therefore, “which platform should we choose?” should not be the first question. The better approach is to profile your business calls first, then review each platform’s billing rules, and finally run the same usage data through the calculation formula. This article follows that approach.

    2. Break down the bill: a usage-driven cost formula

    No matter how a platform displays prices, inference cost can ultimately be broken down into “token metering × unit price.” Five billing dimensions deserve particular attention in real projects.

    2.1 Token metering: input and output are priced separately

    Most LLM services meter usage in tokens, the smallest text chunks processed by the model. In Chinese, one Chinese character usually corresponds to about 1–2 tokens, depending on tokenization. Structured content such as code, JSON, and tables usually consumes noticeably more tokens than natural-language text of the same length. Input and output are generally priced separately, and the output price is often several times the input price. As a result, long generated outputs are a cost that is easy to underestimate.

    The same token can have different prices depending on its role:

    • Input without a cache hit: billed at the standard input price.
    • Input with a cache hit: billed at a cache-hit price that is significantly lower than the standard input price.
    • Input written to cache: some services charge separately for cache writes, usually close to the standard input price.
    • Output tokens: billed at the output price.

    2.2 Context caching: repeated prefixes are the biggest cost variable

    Applications with fixed prompts, long-document summarization, and multi-turn conversations often carry a large, completely identical prefix with every request. Most inference services support prefix caching: the same prefix does not need to be recomputed, and the cached portion is billed at the cache price. These services usually require the cached portion to match character for character from the beginning of the request, and they set a cache time-to-live. If the cache expires and has to be rewritten, write fees may occur.

    As a result, the cache hit rate becomes one of the largest cost variables outside the contract terms. For the same application, increasing the cache hit rate from 30% to 70% can reduce input-side cost by more than half.

    2.3 Batch and asynchronous channels: trading latency for price

    In addition to the real-time channel, most platforms offer asynchronous batch submission. A batch of requests is submitted together and returned within an agreed time window, usually at a lower unit price than real-time calls. This is suitable for offline data processing, bulk text processing, bulk labeling, and model evaluation. The trade-off is that it cannot meet interactive scenarios that require second-level responses.

    2.4 Subscription plans and off-peak discounts

    In addition to pay-as-you-go billing, some platforms offer monthly Token subscription plans or time-window-based off-peak discounts. In essence, they exchange committed usage for a lower price. For stable and predictable workloads, the marginal cost can be significantly lower than pure pay-as-you-go. However, once usage exceeds the plan or falls outside the discount window, the excess may return to standard pricing or even a higher tier.

    2.5 Regional differences and free quota

    The price for the same model may vary by region, and cross-region calls can add data transfer costs and extra latency. In addition, new models often come with free quotas or limited-time promotions during the early launch period. Once usage crosses the quota, standard billing resumes. These costs may not appear in a unit-price comparison table, but they still end up on the bill.

    3. The same usage record under four settlement rules: a reproducible estimate

    To show the impact of rule differences, the following example constructs a typical usage profile and calculates the result under different assumptions. The usage values below are in millions of tokens. The unit prices are hypothetical and are used only to demonstrate the calculation logic. They do not represent real quotes:

    Item Usage Assumed unit price (CNY per million tokens)
    Input tokens 3,000 Cache miss: 20; cache hit: 2; cache write: 20
    Output tokens 600 100

    Assume that the batch channel price factor is 0.6 relative to the real-time channel, and the off-peak discount factor is 0.5.

    Settlement rule Calculation note Assumed monthly bill (CNY)
    ① Pure pay-as-you-go, no cache 3000 × 20 + 600 × 100 120,000
    ② 50% prefix-cache hit rate Input is split into hit and miss portions 87,000
    ③ On top of ②, 60% of input and output moved to the batch channel Billable portion × 0.6 ~59,880
    ④ On top of ③, half of the deferrable calls scheduled during off-peak hours Corresponding portion × 0.5 ~44,910

    With the same usage, the bill falls from 120,000 CNY to about 44,910 CNY, a reduction of more than 60%. The unit prices on each platform are identical; the difference comes entirely from the combination of rules. This chain can be written as a platform-agnostic calculation script. To compare settlement methods, replace the price table and usage profile with real data:

    1. from dataclasses import dataclass, field
    2. from typing import Optional
    3. PRICE = {
    4. # Hypothetical values for demonstration only. Replace with live platform prices.
    5. "realtime": {"input": 20.0, "output": 100.0, "cache_hit": 2.0, "cache_write": 20.0},
    6. "batch": {"input": 12.0, "output": 60.0, "cache_hit": 1.2, "cache_write": 12.0},
    7. "offpeak": {"input": 10.0, "output": 50.0, "cache_hit": 1.0, "cache_write": 10.0},
    8. }
    9. @dataclass
    10. class Usage:
    11. """Monthly usage profile, in millions of tokens (MPT)."""
    12. input: float = 0.0 # Total input tokens
    13. output: float = 0.0 # Total output tokens
    14. hit_ratio: float = 0.0 # Share of input that hits the cache
    15. write_ratio: float = 0.0 # Share of input written to cache
    16. geo_factor: float = 1.0 # Regional adjustment factor
    17. free_quota: float = 0.0 # Remaining free quota (MPT, converted at standard input price)
    18. discount: float = 1.0 # Settlement discount for batch, off-peak, and similar options
    19. @dataclass
    20. class Plan:
    21. """Subscription plan; unit_price is in CNY per MPT."""
    22. included: float
    23. used: float = 0.0
    24. over_price: Optional[float] = None # Excess-unit price; defaults to pay-as-you-go pricing
    25. unit_price: Optional[float] = None # Effective plan price, used for comparing with non-subscription options
    26. def monthly_cost(u: Usage, price: dict, plan: Optional[Plan] = None) -> float:
    27. """Calculate the monthly bill (CNY) using metering × unit price × adjustments."""
    28. hit = u.input * u.hit_ratio
    29. write = u.input * u.write_ratio
    30. miss = max(u.input - hit - write, 0.0)
    31. mpt = miss + write + u.output # Plan allowance is deducted using all billable tokens
    32. if plan is not None and plan.over_price is None:
    33. covered = min(mpt, max(plan.included - plan.used, 0.0))
    34. over = max(mpt - covered, 0.0)
    35. # The plan is prepaid, so covered usage is not charged again
    36. return (plan.included * plan.unit_price + over * price["input"]) * u.geo_factor
    37. # Non-subscription path, or subscriptions that require explicit excess rules
    38. cost = (miss * price["input"] + hit * price["cache_hit"]
    39. + write * price["cache_write"] + u.output * price["output"])
    40. cost *= u.discount * u.geo_factor
    41. return max(cost - u.free_quota, 0.0)
    42. if __name__ == "__main__":
    43. plan = Plan(included=4000, unit_price=9.0) # Assumed: 4,000 MPT allowance, equivalent to CNY 9 per MPT
    44. base = dict(input=3000, output=600, hit_ratio=0.5, discount=0.6)
    45. scenarios = {
    46. "① Pure pay-as-you-go": dict(base, hit_ratio=0.0, discount=1.0),
    47. "② Prefix cache": dict(base, discount=1.0),
    48. "③ + Batch channel": base,
    49. "④ + Off-peak window": dict(base, discount=0.3),
    50. "⑤ Subscription plan": dict(base, hit_ratio=0.0, discount=1.0),
    51. }
    52. for name, params in scenarios.items():
    53. p = plan if name == "⑤ Subscription plan" else None
    54. print(f"{name}: {monthly_cost(Usage(**params), PRICE['realtime'], p):,.0f} CNY")

    The fifth scenario in the script shows the subscription-plan path. Assume a subscription plan provides a fixed monthly allowance of 4,000 million tokens at an effective CNY 9 per million tokens, while the same workload would cost 120,000 CNY under pure pay-as-you-go without caching or discounts. In that case, a subscription plan is more suitable for a workload that is steady, does not use caching, and is almost entirely real-time. If only part of the calls are suitable for the plan, split the workload into two Usage objects and use plan.over_price to specify the excess-unit price, then settle them separately and sum the results.

    Pay special attention to the input-to-output ratio. A price sheet only gives a price per million tokens, but there is a multiple-times price gap between input and output. If output accounts for a large share of total tokens, or if the cache hit rate is low, a pay-as-you-go bill can quickly be pushed up by output and uncached input. Pay-as-you-go can only be on par with a subscription plan for workloads where output accounts for a small share and caching is largely ineffective. This is the typical cause of “four platforms have identical unit prices, but monthly bills differ by half”: one side lowers the effective unit price with caching and batch processing, while the other pays the standard price for full usage.

    4. Match billing methods to business scenarios: five typical cases

    4.1 Scenario 1: The model can be replaced by outcome, with cost as the priority

    If the business has no strong dependency on a specific model—for example, text classification, extraction, or summarization—you can compare output quality across different tiers in the same model family using a small evaluation set. If the acceptance metrics are met, choose the tier with the lower unit price. Unit prices across tiers may differ by several times, and this factor usually has a greater impact than differences between channels. A practical approach is to build a candidate table with a quality threshold and a cost ranking: remove options that fail the threshold, then compare total cost among the remaining options.

    4.2 Scenario 2: Offline batch tasks

    Tasks such as log analysis, bulk document summarization, data cleaning and labeling, and offline evaluation usually do not need second-level responses. These workloads should prioritize the batch channel, and object storage can be used for task distribution and result collection when necessary. When evaluating them, calculate three items: the batch price discount, the deadline for task completion, and the extra cost caused by retries. If the batch channel’s return window conflicts with the business deadline, use a hybrid approach of “real-time guarantee plus batch gain,” sending only deferrable calls to batch.

    4.3 Scenario 3: Real-time online, with the main model already locked

    For core paths that are latency-sensitive and cannot easily change models because of quality or compliance requirements, consider a subscription plan plus pay-as-you-go for excess usage. The plan allowance should be estimated from the P50 and P95 usage percentiles over the past three to six months. If it is too low, you will frequently exceed it; if it is too high, the allowance will be wasted. At the same time, identify calls that can be deferred, such as non-real-time notification copy, offline reports, and asynchronous summaries, and move them to off-peak windows or the batch channel.

    Using the assumptions in this article, when 60% of calls can be moved to off-peak windows (scenario ④ in the script), the effective discount drops from 0.6 to 0.3, and the bill is reduced by about 64% compared with pure pay-as-you-go. Generalizing, there is a simple estimation relationship:

    Overall savings ratio ≈ share of migratable calls × (1 − discount factor of the migratable portion)

    This also explains why, at the same unit price, teams that design their call schedule well may have bills that are only half those of other teams.

    4.4 Scenario 4: Existing business already runs on a cloud

    If data, services, and operations are already deployed on a cloud platform, connecting to model services on the same platform can avoid the extra latency and data transfer costs of cross-region calls. Token prices alone may be identical across platforms, but after including hidden costs such as transfer, retries, and failover, same-region access is often cheaper. For example, a team whose existing data and business systems already run on Baidu AI Cloud can prioritize evaluating direct model-service access within the cloud to reduce cross-network overhead and potential failure points.

    4.5 Scenario 5: Multi-region and overseas business

    Cross-border or cross-region calls may face different regional pricing, compliance reviews, and data-export requirements. Include the region dimension in the selection matrix. List latency, price, and compliance requirements together, and compare cost only after compliance is satisfied.

    4.6 Quick selection guide

    Business characteristic Preferred billing method Key action
    Meets quality requirements; model not locked Pay-as-you-go on a lower-priced tier Quality-threshold evaluation
    Can tolerate minute-level latency Batch channel Task orchestration and retry control
    Real-time, steady traffic, model locked Subscription plan + pay-as-you-go for excess Estimate allowance by P50/P95
    Real-time and highly variable Pay-as-you-go first + off-peak for deferrable calls Call-pool scheduling
    Existing business on a cloud Same-region access Include transfer and operations costs

    5. Engineering implementation: build cost control into the system

    5.1 Build your own metering; do not wait for the monthly bill

    Platform usage details are useful for reconciliation, but cost governance must happen at the moment a request is made. Log structured data for every request in a gateway or wrapper layer. At minimum, record:

    • Model name and version;
    • Input, output, cache-hit, and cache-write token counts;
    • Channel (real-time, batch, off-peak) and region;
    • Unit price and discount applied to the request;
    • Retry count and final status.

    With this data, you can break down cost by business line, scenario, and tenant, calculate budget consumption in real time, and automatically downgrade or rate-limit when a threshold is reached, instead of discovering problems only when the monthly bill arrives.

    5.2 Make prompts “cache-friendly”

    To improve prefix-cache hit rates, organize prompts with stable content first and volatile content last:

    1. # ── Stable prefix: keep character-for-character unchanged to hit the prefix cache ──
    2. [System instructions and role settings]
    3. [Tool or function definitions]
    4. [Fixed knowledge snippets / few-shot examples]
    5. # ───────────── Volatile section ─────────────
    6. [Current user input]
    7. [Request time, session random ID, and other volatile fields, always at the end]

    Supporting practices include moving volatile fields such as timestamps, random IDs, and temporary request identifiers out of the prefix; version-managing system instructions to avoid frequent changes; and setting a reasonable cache TTL to balance hit benefits against write overhead. The benefit is greatest when many requests share the same prefix. If every conversation round uses the complete history as the prefix, the hit rate usually decreases as rounds increase. In that case, use session summaries or similar methods to compress history.

    5.3 Separate calls into pools: real-time pool and deferrable pool

    1. def dispatch(req):
    2. if req.interactive: # User is waiting for the response
    3. pool = "realtime"
    4. elif req.deadline_hours >= 1: # Can tolerate a longer completion time
    5. pool = "batch"
    6. else: # Near-real-time, but does not require second-level response
    7. pool = "offpeak"
    8. return submit(req, pool=pool, max_retries=2, timeout_s=30)

    Pair this with monitoring that distinguishes two failure types: retryable throttling and timeouts, and non-retryable parameter errors. The retry limit and backoff strategy must be set explicitly. Otherwise, a single incident can multiply call volume several times.

    5.4 Use lightweight controls to block low-value calls

    Not every request needs an LLM. If a question can be answered directly with rules, cached results, or retrieval results, return it at the gateway layer. Also set an upper limit on generated length to prevent output-side costs from spiraling out of control.

    5.5 Keep a replaceable model access layer

    Wrap model calls behind a unified internal interface, and manage model names, parameters, and billing conventions through configuration. This allows you to switch between different tiers or platforms as the business changes, and to recalculate costs quickly when pricing or rules change, avoiding lock-in to a single access method.

    5.6 Migration and canary release: include switching costs in the calculation

    When an older model version stops accepting new users and is gradually retired, assess the quality difference and price changes of the new version early. Complete the migration with a small-traffic canary and dual-track comparison to avoid concentrated switching that causes quality fluctuations and bill spikes.

    6. Common misconceptions

    1. Comparing only unit prices, not rules. On platforms with identical unit prices, any one of batch discounts, cache billing, subscription plans, or regional factors can produce a double-digit percentage cost difference.
    2. Ignoring cache writes and expiration. If prefixes change frequently or caches are repeatedly rewritten after expiration, caching may not save money and can even cost more than not enabling it.
    3. Estimating plan allowances too coarsely. Once a plan allowance is exceeded, excess usage is billed at standard rates. During promotions or traffic spikes, this can end up more expensive than pay-as-you-go.
    4. Ignoring input/output structure. When output share is high or cache hit rate is low, any discount on the input side has limited effect.
    5. Looking only at model cost, not integration and operations cost. Hidden costs such as cross-region calls, multi-provider authentication and monitoring, and compliance audits can sometimes exceed the token cost itself.

    7. Conclusion: an actionable checklist

    “Same unit price, different bills” is not contradictory. A unit price is only a number on a price sheet. A bill is the result of rules and calling patterns working together. When implementing, check the following items one by one:

    1. Collect daily and scenario-level call volumes and input/output token distributions for the past three months;
    2. List the billing rules of each platform in the target region, including batch discounts, cache billing, subscription plans, off-peak discounts, and free quotas;
    3. Plug the same usage data into the calculation script and compare settlement methods side by side;
    4. Choose an appropriate billing method for each business scenario, and design a pool-based scheduling strategy for real-time and deferrable calls;
    5. Build your own metering and budget alerts so cost governance happens at the moment a request is made.

    After completing these five steps, “it is the same no matter which platform you choose” becomes a question that can be answered quantitatively: the difference is not in the platform, but in whether you apply the billing rules in the right places.