LLM Token Billing: A Practical Guide to Calculating API Usage Costs

Why does LLM token billing need a separate calculation?

When an AI application moves from a demo to stable production, LLM token billing shifts from a minor concern to a hard business metric. In the early stage, with low call volume and scattered bills, teams often judge cost by rough intuition. Once the system enters production, requests accumulate by the day or even by the hour, and the scale effect of API costs becomes obvious. If teams continue to rely on vague impressions, the bill can easily become painful once the project grows to a certain volume.

More importantly, the cost structure influences product design in reverse. Technical choices such as enabling long context, performing full retrieval augmentation for every request, or always calling the strongest model directly affect the final bill. Therefore, understanding LLM token billing is not only a finance task. It is a basic skill for every architect and developer.

How do you convert tokens to characters or words?

Large language models usually do not charge by “number of characters” or “number of requests.” Instead, they use tokens as the standard unit. A token is the smallest semantic unit after the model splits text. A common approach is subword tokenization, such as BPE (Byte Pair Encoding) methods. One token may correspond to one Chinese character, one English word, or only part of a word.

Two basic points need to be understood:

• Tokens and characters are not a one-to-one mapping. Under a common rule of thumb, one Chinese character is usually equivalent to about 0.6 to 1 token. Conversely, one token usually corresponds to about 1 to 2 Chinese characters, depending on the tokenizer. Punctuation, numbers, code, spaces, and line breaks may also consume tokens.
• Input and output are usually billed separately, and their unit prices are different. Most providers charge more for output tokens than input tokens because generation has a higher computational cost. Before doing formal estimates, use the provider tokenizer or the official token-counting API. Estimating by character count alone can create bias.

The following Python example is useful for quick estimation. It calls both estimate_tokens and build_prompt, closing the loop between context assembly and token estimation:

def estimate_tokens(text: str, ratio: float = 0.6) -> int:
    """
    Roughly estimate the number of tokens.
    ratio represents the average number of tokens per character.
    Chinese can use 0.6 to 1.0, English can use 0.25 to 0.4.
    In production, use the real tokenizer to get exact values.
    """
    return max(1, int(len(text) * ratio))
def build_prompt(system_prompt: str, history: list[dict], user_input: str) -> str:
    """Build the final context sent to the model. Its length directly determines the number of input tokens."""
    parts = [system_prompt]
    for msg in history:
        parts.append(f"{msg['role']}: {msg['content']}")
    parts.append(f"user: {user_input}")
    return "\n".join(parts)
system_prompt = "You are a customer support assistant."
history = [
    {"role": "user", "content": "Where can I check my order?"},
    {"role": "assistant", "content": "Go to the order page."},
]
user_input = "Help me check the shipment."
final_prompt = build_prompt(system_prompt, history, user_input)
system_tokens = estimate_tokens(system_prompt)
history_tokens = sum(estimate_tokens(f"{m['role']}: {m['content']}") for m in history)
user_tokens = estimate_tokens(user_input)
total_tokens = estimate_tokens(final_prompt)
print({
    "system": system_tokens,
    "history": history_tokens,
    "user": user_tokens,
    "total": total_tokens,
})
# Example output: values vary with text length. In production, use the provider tokenizer.
# {'system': 4, 'history': 18, 'user': 3, 'total': 32}

Only after understanding tokens does it make sense to calculate cost.

Subscription or pay-as-you-go? How to choose

Looking at billing forms, there are two common models: subscription and pay-as-you-go. Their cost curves are different.

Subscription: The user pays a fixed fee for a billing period and receives a certain usage allowance. Cost is prepaid and predictable, making budgets easier to control. As long as usage stays within the allowance, the marginal cost of a single call is close to zero, which encourages frequent use. However, if usage is far below the allowance, resources are idle. If usage exceeds the allowance, it may trigger rate limiting, degraded service, or additional charges.

Pay-as-you-go: You are charged based on the tokens actually consumed. There is no fixed threshold, making it suitable for workloads with large fluctuations or early validation stages. The unit price is clear, which makes unit-cost accounting easier. However, cost grows with usage, so active usage governance is required.

Dimension	Subscription	Pay-as-you-go
Cost predictability	High; fixed periodic fees are clear	Medium; changes with call volume
Suitable usage pattern	Stable, high-frequency, predictable	Highly variable, early validation, low-frequency exploration
Consequence of exceeding usage	May trigger rate limiting, degradation, or extra charges	Usually continues to be billed by usage; watch for peaks
Comparability of unit cost	Needs to be converted to actual usage	Quotes are directly comparable
Typical stage	Stable and scaled-up stage	Validation, ramp-up, or hybrid strategy

A simple rule for choosing: compare estimated monthly usage × pay-as-you-go unit price directly with the subscription fee. If monthly usage is stable and close to or exceeds the subscription allowance, a subscription is usually more controllable. If usage is low, highly variable, or the business is not yet validated, pay-as-you-go is more flexible.

How do you convert subscription fees into unit prices?

Converting a subscription package into a “price per million tokens” makes subscription and pay-as-you-go models comparable. The conversion logic can be fixed in a function:

def subscription_to_unit_price(monthly_fee: float, monthly_used_tokens: int) -> float:
    """
    Convert a subscription fee to CNY per million tokens.
    Args:
        monthly_fee: amount actually paid for one billing period
        monthly_used_tokens: total billable tokens consumed in that period
    """
    if monthly_used_tokens == 0:
        raise ValueError("Usage is zero; cannot calculate unit price")
    return monthly_fee / (monthly_used_tokens / 1_000_000)

There is an easily overlooked prerequisite: the denominator must be the tokens actually consumed or the equivalent tokens under the billing model, not the allowance limit. If you divide the fee by the allowance limit, the unit price is artificially lowered, hiding resource waste.

At the same time, pay attention to the difference between average cost and marginal cost. The converted subscription unit price is the average cost for that period. When the allowance is not fully used, the marginal cost of an additional call within the subscription may be close to zero, but it is still constrained by the allowance and rate limits. Once usage exceeds the allowance, it may trigger rate limiting or switch to pay-as-you-go billing. Therefore, the marginal price of a single call should not be directly compared with the pay-as-you-go unit price.

How do you calculate real cost? Four steps and an example

Real-cost calculation can be divided into four steps:

1. Separate input and output usage. Count input tokens and output tokens separately, because their unit prices are usually different.
2. Plug the numbers into the weighted formula to calculate the base cost. Use input, output, cache hit rate, and cache discount.
3. Compare with the converted subscription unit price. Convert the subscription fee into CNY per million tokens using the same basis.
4. Apply hidden-variable corrections. Cache hits, batch discounts, failed retries, and structured output can all change the actual bill.

A runnable example is shown below. It adds cache_rate, which represents the ratio of the cached-read unit price to the input unit price. A cache hit is not free; it is usually billed at a lower rate. The exact ratio depends on the provider documentation.

def effective_input_tokens(
    input_tokens: int,
    cache_hit_ratio: float,
    cache_rate: float,
) -> float:
    """
    Calculate the effective number of input tokens after caching.
    cache_rate represents the ratio of the cached-read unit price to the input unit price, for example 0.1.
    """
    return input_tokens * ((1 - cache_hit_ratio) + cache_hit_ratio * cache_rate)
def blended_price(
    input_tokens: float,
    output_tokens: float,
    in_rate: float,
    out_rate: float,
) -> float:
    """
    Calculate the weighted average unit price in CNY per million tokens.
    """
    total_tokens = input_tokens + output_tokens
    if total_tokens == 0:
        raise ValueError("Total token count is zero; cannot calculate unit price")
    return (input_tokens * in_rate + output_tokens * out_rate) / total_tokens
def subscription_to_unit_price(monthly_fee: float, monthly_used_tokens: float) -> float:
    """
    Convert a subscription fee to CNY per million tokens.
    """
    if monthly_used_tokens == 0:
        raise ValueError("Usage is zero; cannot calculate unit price")
    return monthly_fee / (monthly_used_tokens / 1_000_000)
# Example assumptions. Rates are in CNY per million tokens.
input_tokens = 900_000_000
output_tokens = 321_000_000
in_rate = 1.0
out_rate = 5.0
cache_hit_ratio = 0.4
cache_rate = 0.1
monthly_fee = 2000.0
# Effective input tokens after caching
effective_input = effective_input_tokens(input_tokens, cache_hit_ratio, cache_rate)
# Pay-as-you-go blended price after caching
p2p_price = blended_price(effective_input, output_tokens, in_rate, out_rate)
# Total billable tokens used for subscription comparison
billed_tokens = effective_input + output_tokens
# Total pay-as-you-go cost for the period
raw_cost = p2p_price * billed_tokens / 1_000_000
# Converted subscription unit price and equivalent period cost
sub_unit_price = subscription_to_unit_price(monthly_fee, billed_tokens)
sub_equivalent_cost = sub_unit_price * billed_tokens / 1_000_000
print(f"Effective input after caching: {effective_input:,.0f} tokens")
print(f"Pay-as-you-go blended price after caching: {p2p_price:.2f} CNY per million tokens")
print(f"Pay-as-you-go cost for the period: {raw_cost:.2f} CNY")
print(f"Converted subscription unit price: {sub_unit_price:.2f} CNY per million tokens")
print(f"Equivalent subscription cost: {sub_equivalent_cost:.2f} CNY")
# Example output:
# Effective input after caching: 576,000,000 tokens
# Pay-as-you-go blended price after caching: 2.43 CNY per million tokens
# Pay-as-you-go cost for the period: 2181.00 CNY
# Converted subscription unit price: 2.23 CNY per million tokens
# Equivalent subscription cost: 2000.00 CNY

In this example, the pay-as-you-go blended price after caching is about 2.43 CNY per million tokens, while the converted subscription unit price is about 2.23 CNY per million tokens. Since the units are the same, they can be compared directly. If the equivalent subscription cost is lower than the pay-as-you-go cost, the subscription is more cost-effective at this usage level.

Which hidden variables make the bill higher?

The real bill often does not match theoretical estimates. The reasons are usually hidden in factors that are easy to underestimate:

• Context expansion. Multi-turn conversations continuously accumulate historical messages in the input. Without truncation or summarization, the input tokens of a single request can grow rapidly as the conversation gets longer.
• Retries and failed requests. Network jitter, rate limiting, and timeouts can trigger retries. Each retry may generate real consumption, and it often does not appear in the statistics for “successful requests.”
• Structured output overhead. When the model is asked to return strict JSON, tables, or long formatted content, output tokens may be much higher than for natural-language answers.
• Cold starts and peaks. Under pay-as-you-go, sudden traffic spikes may trigger rate limiting or queuing. If throughput capacity is reserved for peaks, idle capacity raises the effective unit cost.

In addition, different providers have different discount rules for cache hits, batch processing, reserved capacity, and model versions. When making a budget, rely on the official pricing page and your own measured bills, rather than a single rule of thumb.

How do you reduce token costs? 6 optimization methods

1. Model tiering and routing
   Classify requests by complexity. Send simple tasks to lightweight models and call stronger models only for complex reasoning. In many scenarios, model tiering is one of the most cost-effective methods with relatively low implementation cost. The prerequisite is that requests can be classified by complexity and that there is a quality regression mechanism to prevent higher error rates from lightweight models.

2. Context and prompt caching
   Cache frequently reused system prompts or knowledge-base snippets. After a cache hit, they are billed at a lower price. Caching is not free; it is discounted according to provider rules. Also pay attention to cache expiration, minimum cache length, and billing definitions. If cached content involves user data or third-party interfaces, perform desensitization, permission checks, and compliance review first.

3. Batch processing
   For offline tasks that are not sensitive to latency, batch APIs usually provide a lower unit price at the cost of slower responses. This is suitable for scenarios such as log analysis, batch summarization, and offline generation.

4. Context compression
   Summarize or use sliding-window truncation for historical conversations, keeping only necessary information to control input volume at the source. Compression strategies must balance information loss and cost savings, avoiding the loss of critical context just to save tokens.

5. Result reuse
   Build a cache layer for identical or semantically equivalent requests to avoid calling the model repeatedly. However, semantic-equivalence caches may return stale or incorrect results, so TTL, version invalidation, and sampling checks are required. Model upgrades may also change tokenization and billing rules, so costs and cache hit rates should be reviewed after an upgrade.

6. Budget guardrails
   Set daily or monthly usage limits and alerts on the application side. Once a threshold is approached, automatically degrade service, rate-limit, or switch to a lightweight model to prevent the bill from getting out of control.

Common mistake checklist

• Using the allowance limit instead of actual usage as the denominator for subscription cost allocation.
• Treating cache hits as zero cost and ignoring that cached reads may still be billed at a low price.
• Treating the converted average subscription unit price as the marginal cost of a single call.
• Counting only successful requests and ignoring consumption from retries, failures, and queuing.
• Not separating input and output statistics, which makes the cost definition unclear.
• Ignoring changes to tokenizers, cache rules, or billing rules after model upgrades.

Summary and FAQ

The cost of using large language models is not a vague question of “expensive or not.” It is an engineering problem that can be accurately decomposed and governed. The core path is: first understand tokens as the billing unit, then use a unified formula to compare subscription and pay-as-you-go on the same basis, and finally reduce unit costs through model tiering, caching, batch processing, context compression, and budget guardrails.

FAQ

How do you convert tokens to characters or words?
Usually, one token corresponds to about 1 to 2 Chinese characters, but punctuation, numbers, code, English, and spaces can affect the result. For formal estimates, use the provider tokenizer or the official token-counting API.

How much does context caching actually save?
After a cache hit, the request is usually billed at a certain percentage of the input unit price, not for free. The savings depend on the hit rate, cache price ratio, and provider billing rules, so combine official documentation with measured data.

When is a subscription more cost-effective?
When monthly usage is stable, predictable, and close to or above the subscription allowance, a subscription is usually more cost-effective. If usage is low, highly variable, or the business is not yet validated, pay-as-you-go is more flexible.

Why does the bill not match?
Common reasons include ignoring retries and failed requests, continuously accumulating conversation history, longer structured output, misunderstanding cache billing, not separating input and output statistics, and changes in provider billing rules. It is recommended to export usage details by billing period and review them with a unified formula.