LLM Token Pricing Explained: Why a Low Unit Price Doesn’t Mean Low Cost

When developers evaluate the cost of calling LLM services, they often focus on a single metric: the price per 1K tokens. Based on that metric alone, they may conclude which provider is cheapest. However, the amount that ultimately appears on the bill is not determined by the unit price itself. It is determined by the total number of tokens actually consumed. With the same unit price, an application that repeatedly sends the full conversation history as context can spend several times—or even tens of times—more than an optimized implementation. This article explains the underlying logic of token-based billing and provides practical methods for estimating and optimizing LLM usage costs.

1. What token-based billing actually measures

LLM services generally do not bill per API call or per character. They bill by token. A token is the smallest semantic unit processed by the model. A Chinese character, an English word, or a sequence of code symbols may be split into one or more tokens.

A key point is that billing usually includes two components:

• Input tokens (prompt): The prompt, system instructions, conversation history, retrieved reference material, and other content sent to the model are counted as input.
• Output tokens (completion): The content generated by the model is counted as output.

In other words, the real cost of even a seemingly simple request is:

input tokens × input price + output tokens × output price.

Looking at only one of these prices is not enough to estimate the actual expense.

2. Why focusing only on unit price can be misleading

A low unit price does not necessarily mean a low total cost. Several factors explain this:

1. Input and output prices are often asymmetric. Many services charge a significantly higher unit price for output tokens than for input tokens. If an application has long inputs and long outputs—such as code generation or long-form writing—the cost on the output side can be rapidly amplified by the price multiplier.

2. Longer context creates repeated consumption. In multi-turn conversations, if the full history is included in every request, the input tokens for the N-th turn will include the content of all previous turns, creating a quadratic-like accumulation of consumption. In a 10-turn conversation, historical content may be billed close to 10 times.

3. System prompts are repeatedly counted. Role descriptions, tool instructions, and output-format requirements—sometimes several hundred characters long—will be re-billed in every request if they are included every time.

4. Retrieval-augmented generation has hidden overhead. In RAG applications, retrieved document snippets can heavily fill the context. A single request may contain thousands of input tokens, directly increasing total usage.

A simple example makes this clearer. Suppose a service charges CNY 0.01 per 1K input tokens, which looks very low. If a single request has 8,000 input tokens and 2,000 output tokens, the cost per request is approximately 8 × 0.01 + 2 × output price. If the output price is CNY 0.03 per 1K tokens, a single request costs CNY 0.14. If there are 10,000 calls per day, the daily cost is about CNY 1,400. In this case, the word “low” in “low unit price” provides little useful information.

3. Quantifying actual token consumption with code

Instead of guessing, it is better to put the cost model into code. The following is a simplified example of token-consumption and cost estimation logic:

def estimate_cost(input_tokens, output_tokens,
                  input_price_per_1k, output_price_per_1k,
                  cache_hit_ratio=0.0):
    """
    Estimate the cost of a single API call.
    cache_hit_ratio: the proportion of input tokens served from cache.
    Cached input is usually billed at a lower price.
    """
    # Cached input is billed at a discount. Here, 0.1x is used for illustration.
    cached_input = input_tokens * cache_hit_ratio
    fresh_input = input_tokens - cached_input
    input_cost = (fresh_input * input_price_per_1k
                  + cached_input * 0.1 * input_price_per_1k) / 1000
    output_cost = output_tokens * output_price_per_1k / 1000
    return input_cost + output_cost
# Estimate cumulative cost in multi-turn conversations
def multi_turn_cost(history_msgs, new_input, output_tokens,
                    input_price, output_price):
    total = 0
    context = []
    for turn in history_msgs:
        context.extend(turn)          # History keeps accumulating
        cur_input = count_tokens(context) + count_tokens(new_input)
        total += estimate_cost(cur_input, output_tokens,
                               input_price, output_price)
    return total

The key point of this code is that cost grows dynamically with context length. Any static formula such as “unit price × estimated number of calls” can significantly underestimate the real bill.

4. Subscription or credit packages vs. pay-as-you-go

In addition to pure pay-as-you-go billing, many platforms also offer subscription packages or prepaid credit bundles. These usually have lower unit prices but come with usage limits or time windows. The choice can be summarized as follows:

• Stable and predictable usage: Subscription packages or prepaid bundles may be suitable because they can lock in a lower marginal unit price.
• Highly variable usage with clear peaks: Pay-as-you-go is often more suitable because it avoids paying for idle credits.
• Check credit validity periods: Many credit packages expire if not used within the billing cycle. Their effective cost may be higher than the listed price, so they should be evaluated based on actual usage rate.
• Separate credit pools for different models: High-end models and lightweight models often have separate credit pools. When using them together, their usage should be calculated separately.

The right way to judge whether a plan is economical is not to compare listed prices. Instead, run real business samples to measure the average token consumption per call, multiply it by the estimated call volume, and compare the result with the package credits.

5. Engineering practices for cost optimization

The key to reducing costs is to reduce actual token consumption rather than to obsess over unit price. The following strategies are listed in order of return on investment:

1. Optimize context and avoid sending the entire history every time. In multi-turn conversations, keep only the most recent necessary turns, or summarize and compress earlier history. This is usually the optimization with the highest return on investment.

2. Simplify system prompts and instructions. Remove redundant role descriptions and repeated format constraints, and keep standing instructions to the minimum necessary set.

3. Use context caching when available. Many services provide caching for repeated prefix content, such as fixed system prompts or long documents. Cached input tokens are usually billed at a significant discount, making caching useful for long documents and fixed prompts.

4. Use model-tier routing. Send simple tasks, such as classification, extraction, and format conversion, to lightweight models, and reserve more capable models for complex reasoning. A routing layer that directs traffic by task difficulty can significantly reduce average cost per call.

5. Control output length. Set reasonable limits on output tokens to prevent the model from producing overly long responses. Because output-side unit prices are usually higher, limiting output length can have a clear cost-saving effect.

6. Use monitoring and alerting. Connect usage metrics such as call volume, token consumption, and cost to logging, monitoring, and alerting systems. Set daily budget thresholds to detect abnormal consumption, such as retry loops or runaway context growth.

6. The right way to choose and evaluate LLM services

Based on the analysis above, the cost of an LLM service or calling strategy should be evaluated in the following order:

• Step 1: Sample real business data. Measure the average distribution of input and output tokens instead of estimating from idealized short samples.
• Step 2: Calculate input and output costs separately. Pay attention to the impact of the output-side price multiplier.
• Step 3: Model multi-turn and long-context scenarios. Include cumulative consumption in the overall estimate.
• Step 4: Compare subscription credits with pay-as-you-go pricing. Convert package prices into effective unit prices based on real usage.
• Step 5: Monitor continuously after launch. Use actual bills to refine the estimation model.

Conclusion

The key idea is simple: do not focus only on the per-token price; the amount actually consumed is the major cost driver. Unit price is only a static parameter. The real bill is the dynamic result of input-output structure, context management, call frequency, caching strategy, and other factors. For developers, instead of repeatedly comparing differences of a fraction of a unit price, it is more productive to focus on engineering practices such as reducing context, routing requests to the right model tier, reusing cached prefixes, and monitoring usage. Once actual consumption is controlled, costs naturally come down.