LLM Billing Truth: Token Consumption, Not Price per Token, Drives Real Costs

1. The Overlooked Truth: A Lower Price per Token Does Not Mean Lower Spending

When evaluating LLM service costs, many teams start by comparing unit prices: “How much does it cost per 1,000 tokens, and who is cheaper?” However, after running the service for some time, they often find that the actual bill is very different from the initial estimate. The root cause is that unit price is only a listed price. What really determines spending is the total number of tokens consumed.

This is similar to taking a taxi. Even if the base fare is low, the total fare can still be high once mileage and time increase. Likewise, a seemingly cheap model can become expensive if repeated context carrying, multi-turn accumulation, or verbose outputs consume a large number of tokens. In some cases, its total cost may exceed a more expensive option.

This article explains three things: what tokens are, how the two common billing models differ, and how to estimate and control costs in real applications.

2. Tokens: The Basic Billing Unit for LLM Costs

LLM inputs and outputs are not billed by characters or words, but by tokens. A token is the smallest unit of text processed by the model. Roughly, it corresponds to a word stem in English, or one to two Chinese characters, but the exact segmentation is determined by the model’s tokenizer. The same sentence may be split into different numbers of tokens by different tokenizers, which is one reason why comparing unit prices directly across models is often inaccurate.

Before evaluating costs, keep these facts in mind:

• Input tokens and output tokens are priced separately. The generated portion, or output, is usually more expensive than input.
• Billed usage per request = input tokens + output tokens, accumulated separately.
• A larger context window allows a single request to carry more information, but token usage also grows faster. This is a major cause of unexpectedly large bills.

A typical request’s token composition looks like this:

System prompt
  + conversation history (accumulated across turns)
  + current user input
  + reference documents / retrieved knowledge snippets (RAG)
  --------------------------------
  = input tokens
Model-generated response
  = output tokens
Total billed tokens = input tokens + output tokens

3. Two Main Billing Models: Subscription Plans vs. Pay-as-You-Go

Enterprises and individuals using LLM services usually encounter two types of billing. Their cost structures are completely different.

1. Pay-as-you-go

Pay-as-you-go billing is based on actual token consumption. You pay for what you use, with input and output priced separately. Its advantage is flexibility and no upfront threshold. It is suitable for scenarios with unstable traffic, early validation, or low call volume. Its disadvantage is that it has no cap. If request volume or per-request context grows, the bill may grow linearly or even faster, with little built-in constraint.

2. Subscription plans, monthly or annual

A subscription charges a fixed fee in exchange for usage quota over a period of time, usually expressed as total tokens, request count, or rate limits. Its advantage is predictable cost and often a lower unit cost. It is suitable for high-frequency scenarios with stable and predictable call volume. Its disadvantage is that unused quota may be wasted, and exceeding the quota may lead to throttling or additional charges.

The key decision is: first estimate your average monthly consumption, then choose the model.

• Low and volatile consumption → pay-as-you-go is often more cost-effective.
• High and stable consumption → a subscription can usually significantly reduce unit cost.
• Early validation → start with pay-as-you-go, collect real consumption data, then decide whether to switch to a subscription.

4. Why Actual Token Consumption Is the Main Cost Driver

Unit price only determines how many tokens you get per dollar. Consumption determines how many tokens you need to buy in total. What usually drives up the bill are several easily overlooked factors.

4.1 Repeated Context Carrying Is the Biggest Cost Driver

In multi-turn conversations, if every request sends the full system prompt and all history again, token usage grows approximately quadratically with the number of turns. Suppose the system prompt is 800 tokens and each new turn adds 200 tokens:

• Turn 1 input: 800
• Turn 2 input: 800 + 200 = 1,000
• Turn 3 input: 800 + 400 = 1,200
• …
• Turn 10 input: 800 + 1,800 = 2,600

After ten turns, cumulative input tokens approach 17,000, and this is only the input side. Much of the confusion where “the unit price did not change but the bill doubled” comes from here.

4.2 Output Tokens Are Usually More Expensive

Because output tokens generally cost more than input tokens, a verbose answer with hundreds or thousands of output tokens may cost more than an input request carrying a lot of context. Controlling output length is often the most direct way to save money.

4.3 Retrieval and Document Concatenation

In retrieval-augmented generation, or RAG, applications, retrieved knowledge snippets are added to the context for every request. The more snippets retrieved and the longer they are, the larger the input token usage becomes, and this consumption is repeated on every call.

4.4 Overly Large System Prompts

A system prompt containing thousands of tokens is billed repeatedly on every request. Streamlining the system prompt is one of the highest-return optimizations.

5. Estimating Costs Accurately: A Reusable Method

Before choosing a service or making a budget, estimate costs with real business samples. The following is vendor-neutral sample code that helps estimate tokens from character counts and calculate monthly cost for a given usage pattern:

def estimate_tokens(text: str) -> int:
    """Rough estimate: about 1.5 Chinese characters per token and 4 English characters per token.
    In production, use the corresponding tokenizer for exact values."""
    cjk = sum(1 for ch in text if '\u4e00' <= ch <= '\u9fff')
    other = len(text) - cjk
    return int(cjk / 1.5 + other / 4)
def monthly_cost(
    system_prompt: str,
    avg_input_extra: int,     # Extra input tokens per turn (history + user input + retrieval)
    avg_output: int,          # Average output tokens per turn
    rounds_per_session: int,  # Number of turns per session
    sessions_per_month: int,  # Number of sessions per month
    price_in: float,          # Input price per token
    price_out: float,         # Output price per token
) -> dict:
    sys_tokens = estimate_tokens(system_prompt)
    # Key point: input tokens accumulate as turns increase; this approximates the average increment over n turns.
    n = rounds_per_session
    input_per_session = sys_tokens * n + avg_input_extra * (n * (n + 1) / 2)
    output_per_session = avg_output * n
    total_in = input_per_session * sessions_per_month
    total_out = output_per_session * sessions_per_month
    return {
        "monthly_input_tokens": int(total_in),
        "monthly_output_tokens": int(total_out),
        "monthly_cost": round(total_in * price_in + total_out * price_out, 2),
    }

When you compare the result of this function with an intuitive estimate based only on unit price, the difference is often surprising. This quantifies the idea that actual consumption is the main cost driver. Note: the prices, quotas, and other parameters you enter should follow the official documentation of your service platform. Model prices can vary significantly across capability tiers.

6. Six Best Practices for Controlling LLM Costs

1. Simplify system prompts: Remove redundant instructions and split long fixed explanations into on-demand injections to avoid repeated billing for every request.

2. Manage context: For multi-turn conversations, use trimming or summarization strategies, such as keeping only the most recent N turns or compressing early conversation history, to prevent accumulation from the source.

3. Control output length: Constrain answer length through clear instructions, and use a maximum generation length as a fallback when necessary.

4. Use caching where available: Some industry solutions support caching stable long prefixes, such as system prompts or fixed knowledge bases, so that subsequent requests are billed only for changed parts. This can significantly reduce repeated input costs. Specific capabilities and billing rules should follow your chosen platform’s documentation.

5. Add monitoring and alerts: Record input and output tokens for each call through logging services, and set thresholds with monitoring and alerts. If abnormal growth occurs, intervene promptly instead of discovering the problem after costs have already run away.

6. Choose models by tier: Not every task needs a flagship model. Separate tasks by value and complexity: use high-performance models for high-value or complex reasoning, and lightweight models for batch or simple tasks. This can often reduce total cost without sacrificing quality.

7. Conclusion

When evaluating LLM costs, shifting attention from “how much per 1,000 tokens” to “how many tokens are consumed in total this month” is the first step to avoid budget overruns. First understand the token billing structure: input and output are priced separately and accumulated per request. Then choose between subscription plans and pay-as-you-go based on the stability of your call volume. Finally, estimate with real samples and control context, output, and prompts to reduce costs.

In one sentence: unit price determines whether something looks expensive or cheap, while consumption determines whether the total volume is large or small. Saving money depends on controlling consumption, not just comparing the listed price.