A Bill That Surprises Many Developers

Many developers who use large language model APIs for the first time pause when they see their usage bill. After processing hundreds of millions or even billions of tokens, the charge may still be only a dozen or so RMB. To put that in perspective, it is like feeding an entire encyclopedia’s worth of text to the model several times, yet the bill is still less than the price of a cup of milk tea. The first reaction is often to wonder whether the billing system has made a mistake or whether the decimal point is misplaced.

In most cases, nothing is wrong. This is exactly what normal token-based billing looks like for current LLM services. Understanding it requires more than knowing how the calculation works. It also involves understanding why the price can be so low, whether this low price can continue, and how developers can make good use of these billing rules.

1. The Basic Rules of Token Billing

Large language models do not charge simply by character count, word count, or number of requests. They charge by token, the smallest unit of text processed by the model. A token can be an English word, a Chinese character, a punctuation mark, or a subword fragment. Because of this, two pieces of text that look similar in length may have very different token counts, especially when they contain mixed Chinese and English, code, JSON, or dense special symbols.

A request is usually split into two categories:

• Input tokens (prompt): the prompt, context, system instructions, and other content sent to the model.
• Output tokens (completion): the content generated and returned by the model.

Most platforms price by “cost per million tokens,” and output pricing is usually higher than input pricing. This is because generation, which decodes tokens one by one, generally involves more computation than input processing, which can often be parallelized during prefilling. When reviewing a bill, first separate input and output, then multiply each by its unit price to get an accurate total cost.

2. Doing the Math: Why Huge Token Volumes Are Still Cheap

The low cost is essentially a multiplication problem. Suppose the input price for a given tier is 0.05 RMB per million tokens:

• 100 million tokens × 0.05 RMB per million tokens = 5 RMB
• 300 million tokens × 0.05 RMB per million tokens = 15 RMB

“300 million” sounds astronomical, but it corresponds to 300 million-token units. When multiplied by a very small unit price, the result is only a dozen or so RMB. The magnitude of the unit price determines that even large token volumes translate into surprisingly small absolute amounts.

Two often-overlooked factors sit behind this:

1. Model scale and cost are not linearly related. As inference frameworks are optimized, hardware efficiency improves, and batching and quantization mature, the marginal cost of each inference is greatly reduced. As a result, the industry’s unit prices have been pushed down.
2. The billing unit itself is very large. One million tokens represents a huge amount of text. In everyday business scenarios, a single API call may contain only a few hundred or a few thousand tokens. The cost per call can be a fraction of a cent, and even accumulated usage may not look significant.

3. Where the Low Price Comes From: A Look at the Cost Structure

To judge whether low prices can last, it helps to understand the cost structure behind LLM inference services. Major costs include:

• Compute and hardware: procurement or rental of GPUs/accelerators, depreciation, and power consumption.
• Memory and bandwidth: long contexts and high concurrency can significantly increase memory usage and network overhead.
• Operations and redundancy: infrastructure investments such as high availability, elastic scaling, monitoring and alerting, and disaster recovery.
• Model maintenance: ongoing work on version iteration, evaluation, safety alignment, and other maintenance tasks.

During early customer-acquisition phases, vendors may use prices below long-term cost, or even subsidies, to expand their user base. Once scale effects are achieved, unit costs can be reduced through technological progress. This explains the two main sources of current low pricing: real cost reductions from engineering optimization and temporary price advantages from staged pricing strategies. Together, they create the experience of “processing hundreds of millions of tokens for only a dozen or so RMB.”

4. Why Low Prices May Be Temporary

Returning to the developer’s concern: the price is cheap now, but it may rise. This concern is grounded in reality. Price changes usually come from the following causes:

• Changes in supply and demand: when compute capacity is tight or demand surges, upward pressure on unit prices increases.
• End of promotional periods: subsidized low prices are difficult to maintain long term, and returning to cost-based pricing is inevitable.
• Model capability upgrades: larger context windows and stronger reasoning capabilities often come with higher unit costs.
• Changes in product forms: pricing may shift from simple pay-as-you-go to packages, committed volume, tiered pricing, or more complex models.

For developers, the conclusion is clear: do not treat the current unit price as a permanent constant. Leave flexibility in architecture and budget planning.

5. Estimating Costs Before You Build

Rather than being surprised by the bill later, it is better to quantify costs during solution design. A general estimation approach is to estimate input tokens and output tokens separately, multiply each by the corresponding unit price, and then factor in call frequency and expected growth.

def estimate_cost(input_tokens, output_tokens,
                  input_price_per_million, output_price_per_million,
                  calls_per_day):
    """Estimate cost based on daily call volume. Prices are illustrative; use official platform pricing."""
    per_call = (
        input_tokens / 1_000_000 * input_price_per_million
        + output_tokens / 1_000_000 * output_price_per_million
    )
    daily = per_call * calls_per_day
    return per_call, daily, daily * 30
# Illustrative example: 2,000 input tokens and 500 output tokens per call, 100,000 calls per day.
per_call, daily, monthly = estimate_cost(
    input_tokens=2000,
    output_tokens=500,
    input_price_per_million=0.5,   # Illustrative price, not official pricing.
    output_price_per_million=2.0,  # Illustrative price, not official pricing.
    calls_per_day=100_000,
)
print(f"Per-call cost: {per_call:.6f} RMB")
print(f"Daily cost:   {daily:.2f} RMB")
print(f"Monthly cost: {monthly:.2f} RMB")

The key point is that input tokens are often an underestimated cost driver. System prompts, long documents, and multi-turn conversation history can accumulate. If the full context is sent to the model every time, input costs can grow quickly.

6. Managing Token Costs: Optimization Practices

After estimation, the more important task is spending less. The following strategies are widely used in production environments:

1. Simplify prompts and context. Remove redundant examples and explanations, keep only necessary system instructions and effective context, and directly reduce input tokens.
2. Control output length. Use explicit length or format constraints, such as “summarize in three points” or “do not exceed 200 words,” to reduce output tokens. Since output pricing is usually higher, this can produce clear savings.
3. Route tasks across model tiers. Send simple tasks such as classification, extraction, and formatting to lighter models, and use larger models only for complex reasoning. Avoid using an unnecessarily powerful model for simple work.
4. Use context caching effectively. For stable system prompts or long documents, if the platform supports prompt caching or similar capabilities, reuse them instead of paying for repeated input every time.
5. Use batching and asynchronous processing. For tasks that can tolerate latency, batch API calls to improve resource utilization.
6. Use streaming responses. Returning results incrementally improves user experience and makes it easier to stop early when errors occur, avoiding wasted consumption.

7. Monitoring and Budget Guardrails: Keeping Costs Under Control

After estimation and optimization, continuous observation and constraints are still needed to keep costs under control. When building intelligent applications in the cloud, you can combine common capabilities:

• Usage collection: record input tokens, output tokens, and latency for each call, and send the data to a logging service.
• Cost visualization: aggregate costs by business unit, API endpoint, or user in a monitoring and alerting platform.
• Budget alerts: set daily or monthly cost thresholds and trigger alerts or rate limiting when approaching the limit.
• Quotas and rate limits: set token quotas for individual users or endpoints at the gateway layer to prevent abnormal calls or abusive usage.

A simplified example of runtime usage instrumentation:

def logged_completion(client, messages):
    resp = client.create(messages=messages)  # Illustrative call
    usage = resp.usage  # Includes prompt_tokens / completion_tokens
    record_usage(
        prompt_tokens=usage.prompt_tokens,
        completion_tokens=usage.completion_tokens,
        model=resp.model,
    )
    return resp

Once metering data is connected to a unified monitoring and alerting system, teams can detect abnormal cost increases early and identify the specific business line responsible, rather than regretting the situation only when the monthly bill arrives.

Conclusion

Spending only a dozen or so RMB on hundreds of millions of tokens is not a bug. It is the normal result of current token-based LLM billing: very low unit prices, the conversion of huge token volumes, and temporary promotional advantages together create this experience. However, for engineering decision-makers, the more important question is not “how cheap was this time” but how to move cost estimation earlier, control token usage, and design an architecture that remains resilient to price changes. When you can calculate the bill clearly and add monitoring and guardrails, you can respond calmly whether prices move up or down.

If you implement these practices on Baidu AI Cloud, you can combine the tiered model capabilities of its LLM service platform with general cloud products such as logging, monitoring, and alerting to build a complete cost-governance loop from metering and visualization to quota and rate limiting.