LLM API Costs: Why Identical Token Prices Can Produce Very Different Monthly Bills
Identical token prices do not guarantee identical LLM API bills. This article explains how input and output usage, batch processing, prompt caching, subscription allowances, and overage rates affect total spending. It also provides break-even calculations, a workload-based decision framework, and Python code for estimating request and monthly costs.
1. Why the Same Token Price Can Lead to Different Bills
Comparing listed prices alone does not reveal the true cost of an LLM API. Your bill depends on three factors: unit prices, billing rules, and workload composition. Even when unit prices are identical, differences in billing rules and usage patterns can substantially change total spending.
Consider a team accessing the same model through two providers with identical standard token rates. Through the first provider, every request uses the real-time API. Through the second, half of the tasks that can tolerate a delay use an offline batch API, while stable system instructions appear at the beginning of requests to enable prefix caching.
The resulting difference in monthly bills comes from how the work is organized—not from differences in model capability.
This article focuses on billing differences for the same model, not comparisons between models.
2. Breaking Down an LLM API Bill
LLM APIs typically charge by the token. A request can generate several types of charges:
- Input tokens: Prompts, conversation history, retrieved reference material, and other content sent to the model.
- Output tokens: Content generated by the model. Output rates are typically higher than input rates because tokens are generated sequentially, requiring more time and computation.
- Cache-related tokens: Repeated prefixes may qualify for caching, with cache hits typically billed at a lower rate. Some platforms also charge separately for cache writes.
A request’s cost can be estimated as follows:
Request cost = Uncached input tokens × Input rate+ Cache-hit tokens × Cache-hit rate+ Output tokens × Output rate+ Cache-write charges, if billed separately
Use consistent units when applying this formula. If rates are quoted per million tokens, divide the corresponding token counts by one million.
Because output tokens often cost more, controlling output length and format can directly reduce spending. However, the total also depends on token volumes. With long prompts, input tokens may greatly outnumber output tokens, making input the largest expense.
Calculate each component rather than assuming which one dominates.
3. Billing Rules That Affect Total Cost
3.1 Pay-as-You-Go: Flexible, but Often at Standard Rates
Pay-as-you-go billing offers flexibility without an upfront commitment. It suits workloads with uncertain demand and projects still being validated.
The trade-off is that usage is generally billed at standard rates, without batch or commitment discounts unless the platform offers promotions, reserved capacity, or volume tiers. For large, stable monthly workloads, standard pay-as-you-go billing is often not the least expensive option.
3.2 Offline Batch APIs: Trading Turnaround Time for Lower Cost
Offline batch APIs accept groups of tasks for asynchronous processing, often using available server capacity. Results are returned within a specified completion window.
Suitable workloads include:
- Data labeling
- Bulk document summarization
- Evaluation runs
- Historical data cleaning
Before using a batch API, check:
- Completion SLA: Providers generally specify a completion window rather than an exact processing duration.
- Result delivery: Results may be retrieved through polling, callbacks, or downloadable files.
- Model availability: Not every model supports batch processing.
- Concurrency and file-size limits: Larger workloads may need to be split.
Batch discounts vary by platform. Use the official pricing page rather than assuming a fixed discount.
3.3 Subscription Plans and Overage Charges
A usage-based subscription typically includes a fixed monthly fee and an allowance, with additional usage billed at an overage rate. Evaluating it requires a piecewise calculation.
Define:
U: Monthly usage, in millions of tokensF: Monthly subscription fee, in currency unitsQ: Included allowance, in millions of tokensp: Pay-as-you-go rate, in currency units per million tokensp_over: Overage rate, in currency units per million tokens
The pay-as-you-go cost is:
C_pay = U × p
The subscription cost is:
C_sub = F when U ≤ QC_sub = F + (U − Q) × p_over when U > Q
Define the difference:
Δ(U) = C_sub − C_pay
The subscription is cheaper when Δ(U) < 0.
A higher overage rate does not automatically make a subscription unattractive. Even if p_over ≥ p, the included allowance may still offer savings when F < Q × p. Start by comparing F with Q × p, then evaluate each usage range.
Within the allowance: U ≤ Q
Δ(U) = F − U × p
The difference decreases as usage increases.
- If
F ≥ Q × p, the subscription is never cheaper within the allowance. - If
F < Q × p, the break-even point is:
U* = F ÷ p
Above this point, the subscription is cheaper within the included allowance.
Above the allowance: U > Q
Δ(U) = (F − Q × p) + (p_over − p) × (U − Q)
First calculate:
Δ(Q) = F − Q × p
Then examine the slope, p_over − p.
If p_over < p:
The difference decreases as usage grows.
If
Δ(Q) ≥ 0, the break-even point is:U* = (F − Q × p_over) ÷ (p − p_over)
This point is at or above
Q. The subscription becomes cheaper aboveU*.If
Δ(Q) < 0, the subscription remains cheaper throughout the overage range.
If p_over = p:
The difference remains constant:
Δ(U) = F − Q × p
A negative value favors the subscription, a positive value favors pay-as-you-go, and zero means equal costs.
If p_over > p:
The difference increases as usage grows.
If
Δ(Q) < 0, the subscription remains cheaper until:U_x = (Q × p_over − F) ÷ (p_over − p)
Above
U_x, pay-as-you-go becomes cheaper.If
Δ(Q) ≥ 0, the subscription is never cheaper in the overage range.
Always verify that a calculated break-even point falls within the usage range being evaluated. Otherwise, use the applicable case.
Example
Assume the following illustrative prices, using a consistent currency:
F = 800Q = 100 million tokensp = 10 per million tokensp_over = 12 per million tokens
Since 800 < 100 × 10, the break-even point within the allowance is:
U* = 800 ÷ 10 = 80 million tokens
The overage rate is higher than the pay-as-you-go rate, so the subscription eventually loses its advantage:
U_x = (100 × 12 − 800) ÷ (12 − 10)= 200 million tokens
The subscription is cheaper for monthly usage above 80 million and below 200 million tokens. Costs are equal at those endpoints. Above 200 million tokens, pay-as-you-go is cheaper.
Also check:
- Coverage: Does the plan include the model and version you use?
- Off-peak rules: Some plans offer additional discounts outside peak hours. Whether you can schedule work during those periods affects actual savings.
- Overage rates: Rates substantially above standard pay-as-you-go pricing reduce the plan’s flexibility.
Use the platform’s current pricing page and service terms for the actual parameters.
3.4 Prompt Caching: Activation, Expiration, and Write Costs
Some platforms enable prefix caching automatically; others require explicit configuration. Check the relevant API documentation.
In either case, place stable content—such as system instructions, tool definitions, and fixed reference documents—at the beginning of the request. Put changing content, such as the user’s question, at the end so it does not interrupt the reusable prefix.
Three factors deserve particular attention:
- Time to live (TTL): Check how long cached content remains valid without being accessed. Short retention periods can make cache hits unlikely for infrequent requests.
- Write charges: Some providers charge for writing content to the cache. Subsequent savings must cover that cost.
- Hit rate: Monitor the proportion of input tokens billed as cache hits.
A simple break-even calculation is:
Break-even cache hits = W ÷ S
Here, W is the additional cache-write cost and S is the saving from each cache hit. Caching produces net savings when the number of hits exceeds this threshold.
For example, assume:
Fixed prefix: 3,000 tokensStandard input: 2.0 per million tokensCache write: 2.5 per million tokensCache hit: 0.2 per million tokens
In this example, the cache-write rate replaces the standard input rate for the prefix when it is written. The additional write cost is:
W = 3,000 × (2.5 − 2.0) ÷ 1,000,000= 0.0015
The saving per cache hit is:
S = 3,000 × (2.0 − 0.2) ÷ 1,000,000= 0.0054
Therefore:
Break-even cache hits = 0.0015 ÷ 0.0054 ≈ 0.28
One cache hit is enough to cover the additional write cost in this example. A smaller write premium requires fewer hits to recover.
3.5 Regional and Time-Based Pricing
The same model may have different prices across deployment regions. Cross-border access can also involve higher costs and greater network latency. Compare pricing, compliance requirements, and latency together rather than selecting a region on price alone.
Transferring personal data across borders may trigger applicable data-transfer requirements.
Some platforms also vary prices by time of day or offer off-peak discounts. If your workload has clear peaks and troughs, consider scheduling offline tasks during lower-cost periods.
3.6 Model Updates and Retirement
Model versions change frequently. An older version may first become unavailable to new users and later be scheduled for retirement.
Migration costs can include:
- Changes to API request parameters
- Prompt adaptation
- Output-format validation
- Quality regression testing
During migration, old and new versions may run in parallel. Include this duplicate processing in the budget; otherwise, monthly estimates may be too low.
4. Choosing the Right API and Billing Approach
Before selecting an approach, classify tasks by response-time requirements and usage patterns.
| Workload | Suggested approach | Main reason | What to check |
|---|---|---|---|
| Real-time user conversations | Pay-as-you-go or subscription, depending on usage | Immediate responses are required | Peak concurrency and overage rates |
| Offline background processing | Offline batch API | Immediate results are unnecessary, and discounts may be available | Completion window and result retrieval |
| Long prompts with stable prefixes | Cache-friendly request structure | Reusable prefixes reduce input costs | TTL and cache-write charges |
| Large, stable workloads | Subscription plan | Can reduce costs within the favorable usage range | Piecewise break-even calculations, model coverage, off-peak rules, and overage rates |
| Experiments with uncertain usage | Pay-as-you-go | No fixed monthly commitment; easy to stop | Budget alerts |
Use the following process:
- Classify tasks. Separate real-time, offline, and mixed workloads, and count requests for each.
- Model usage. Estimate input tokens, output tokens, cache-hit rates, and peak concurrency for each category.
- Compare plans. Calculate monthly costs for standard pay-as-you-go, offline batch processing, and subscriptions with overage charges.
- Validate with limited traffic. Run real requests for a week, compare actual charges with estimates, and adjust the model.
Two common mistakes deserve attention.
The first is routing latency-sensitive work to a batch API, where the completion window exceeds the application’s tolerance.
The second is disrupting a reusable prefix with request-specific content. For example, placing a timestamp or username at the beginning of system instructions changes the prefix on every request and can prevent cache hits.
If the billed share of cache-hit tokens remains close to zero, inspect the prefix structure first.
5. Estimating Costs in Python
The following functions estimate per-request and monthly costs. They can support a comparison between:
- Plan A: All requests use the real-time API without caching.
- Plan B: Half of the requests use offline batch processing, with caching enabled.
All prices and discounts are illustrative. Replace them with values from the official pricing page.
The calculation assumes that batch discounts and caching can be combined, and that the batch discount applies to every cost component shown. Verify those rules before using it for a real estimate.
It also assumes that a prefix cache miss is billed at the cache-write rate, rather than as a separate cache-write fee added to standard input charges.
# Units: currency units per million tokens.# Illustrative values only; use the official pricing page.PRICE = {"input": 2.0, # Uncached input rate"cache_write": 2.5, # Cache-write rate for prefix tokens"cache_hit": 0.2, # Cache-hit rate"output": 8.0, # Output rate}def request_cost(prefix_tok, dynamic_tok, output_tok, price,use_cache=False, hit_rate=0.0, batch_discount=0.0):"""Estimate the expected cost of one request.prefix_tok: Number of tokens in the fixed prefix.dynamic_tok: Number of input tokens that change per request.hit_rate: Fraction of prefix tokens served from cache, from 0 to 1.batch_discount: Batch discount fraction, from 0 to 1."""m = 1_000_000if use_cache:prefix_price = (hit_rate * price["cache_hit"]+ (1 - hit_rate) * price["cache_write"])else:prefix_price = price["input"]cost = (prefix_tok * prefix_price+ dynamic_tok * price["input"]+ output_tok * price["output"]) / mreturn cost * (1 - batch_discount)def monthly_cost(requests, **kwargs):return requests * request_cost(**kwargs)
For Plan A, calculate the full request volume without caching or a batch discount. For Plan B, calculate the real-time and batch portions separately using their applicable cache-hit rates and discounts, then add the two costs.
The central lesson is that the listed token rate is only one part of the bill. Accurate estimates require the actual input/output mix, cache behavior, processing mode, and any subscription or overage rules.
