5x Price Hike on LLM APIs: How to Evaluate Inference Costs Scientifically

Recently, a prominent open-source large language model raised its official API prices by roughly fivefold compared to the previous generation. Input, output, and cache-hit rates landed in the tens, hundreds, and single-digit yuan range respectively (per million tokens), deliberately aligned with the standard pricing of mid-tier flagship models from top international providers. In a market that has long championed “low cost, good performance,” the number sent shockwaves. Debate followed immediately: is it expensive, or not?

But “expensive or not” is a conclusion, not an analysis. Break it apart and you get three sharper questions: How is the bill actually composed? Which part of the cost distribution does a reasoning model change? And what, exactly, is an open-weight project selling when it publishes an official price tag? The sections below unpack each layer and end with a cost-evaluation method you can apply before model selection.

1. Read the Bill: The Three-Tier Pricing Structure

Most mainstream LLM APIs split billing into three segments:

• Input: Everything you send to the model — system prompts, context, injected knowledge, the question itself — billed at the input rate.
• Output: Every token the model generates, billed at the output rate, typically 3–5× the input rate.
• Cache hit: Stable prefixes that repeat across requests, billed at a dramatically lower rate — in this case, roughly one-twentieth of the input price.

Using a flagship-tier price as an example (input 20 CNY, output 100 CNY, cache hit 1 CNY per million tokens), a single-call cost collapses to one formula:

def estimate_call_cost(input_tokens, output_tokens, cache_hit_tokens=0, price=None):
    """Estimate cost of a single API call. price unit: CNY per million tokens."""
    price = price or {"input": 20, "output": 100, "cache_hit": 1}
    M = 1_000_000
    return (input_tokens * price["input"]
            + output_tokens * price["output"]
            + cache_hit_tokens * price["cache_hit"]) / M
# Example: 8k input, 2k output, 72k cache-hit (long system prompt + reused tool-definition prefix)
print(estimate_call_cost(8_000, 2_000, 72_000))   # 0.432 CNY
# Without cache, the same 80k input would cost 1.8 CNY — a 4×+ gap

This example reveals the single most powerful lever in the entire pricing scheme: the cache-hit rate. A workload with 90 % prefix reuse can drive effective input cost from 20 CNY to under 3 CNY per million tokens. Agent scenarios are a natural fit — system prompts, tool schemas, and few-shot examples are carried across multiple turns, keeping the prefix highly stable. So the first question when judging whether a model “is expensive for you” is: how many of your tokens can actually hit the cache?

2. Where the Cost Shifts: Reasoning Models Push Bills Toward Output

With traditional chat models, the bulk of the cost usually sits on the input side — long contexts, RAG-injected knowledge, accumulated conversation history — while output is often a few hundred tokens and barely matters. Reasoning models flip this distribution for three reasons:

1. Thinking tokens count as output. The chain-of-thought tokens a model generates are billed at the output rate, which is already roughly 5× the input rate.
2. Thinking length is unbounded. Models are trained to “think as much as needed.” A single task can produce thinking tokens several times the length of the visible answer; on hard tasks, that ratio can reach tens of times.
3. Some models launch with only the highest thinking tier enabled. The full thinking budget is always billed, giving users no parameter to trim it.

Together these mean the 5× headline price increase is a floor, not a ceiling — the real bill is driven by a multiplicative relationship between higher per-token rates and more tokens spent thinking, not a simple addition.

This points to the real target market for high-priced reasoning models. Consumer chat products — short outputs, high frequency, extreme sensitivity to per-token price — can barely sustain hundred-yuan-level output rates. The customers who can are machines: automated agents, code-generation pipelines, batch-processing jobs. They consume tokens machine-to-machine and care about task quality and total cost, not the per-token sticker price.

3. Pricing as Narrative: Open Weights Meet Official List Prices

What makes this round of price hikes particularly interesting is that the model in question follows an open-source path. By convention in the open ecosystem, once weights are released, third-party inference providers list the same model at significantly lower prices within days, and the “window” where the official price holds sway is brief. So why set it so high?

Because at this point the price is not a sales figure — it is a value anchor.

• For users: When you see the same model on a third-party platform at one-third the official rate, your first reaction is “I got a bargain.” The official number has already etched “it’s worth a hundred” into your mind — anchoring in action.
• For the industry: Discounts are tactics; standard pricing is how a vendor values its own capability. Aligning with the standard rate of a top-tier flagship, rather than a promotional rate, says “our model is worth this much,” not “we are cheaper than someone.” It marks the first time a major segment of the domestic model ecosystem has openly rejected shadow positioning and price wars at the pricing level.

The “open weights + high official price tag” combination is nearly unprecedented in open-model history. It works only if one hard condition is met: the model’s public benchmark scores are strong enough to hold up the tag. Without scores to back it, the anchor crumbles; once weights are commoditized, demand flows to low-cost hosting channels and the official price becomes a hollow label.

4. From “Per Million Tokens” to “Per Task”

In the reasoning-model era, buyers’ math has changed. Nobody cares how much a million tokens costs; everyone cares how much a task costs. Token efficiency varies wildly across models — the same question might burn ten thousand thinking tokens in one model and two thousand in another. Across a batch of tasks, completion costs can differ by an order of magnitude even when per-token rates look comparable. Within the open ecosystem the spread is especially wide: flagship reasoning models may cost on the order of one dollar per task, while lightweight models need only a few cents; some peers have permanently cut output rates to single-digit yuan per million tokens, a ten-plus-fold gap versus hundred-yuan output pricing.

A simple model captures per-task cost:

def estimate_task_cost(stats, price=None):
    """stats: token counts and failure rate for a single attempt."""
    per_try = estimate_call_cost(
        stats["input_tokens"],
        stats["output_tokens"] + stats.get("thinking_tokens", 0),  # thinking counts as output
        stats.get("cache_hit_tokens", 0),
        price,
    )
    expected_tries = 1 / (1 - stats["failure_rate"])  # expected attempts until success
    return per_try * expected_tries

Two caveats matter. First, thinking_tokens must be folded into the output side; skipping it invalidates every estimate. Second, quality correction is non-negotiable — a model whose per-token price is one-sixteenth of a competitor’s can end up more expensive per task if it fails often, needs retries, or requires human rework. Cheap and cost-effective are not the same thing.

To evaluate in practice, follow four steps:

1. Build a benchmark task set. Pull 20–50 representative tasks from real workloads, spanning easy, medium, and hard difficulty. Avoid relying solely on public leaderboard problems — their token distributions rarely mirror yours.
2. Collect token distributions. For each task, record actual input, output, thinking token counts and cache-hit rates. Use distributions, not averages; long-tail tasks often drive the majority of cost.
3. Factor in quality-adjusted cost. Fold in failure retries, result rework, and human intervention into a per-task figure.
4. Compare by task cost, not token price. Use “cost per task” as the decision metric; the conclusion often contradicts gut feeling.

5. Reducing Your Bill: Pricing Is the Vendor’s Problem, Cost Is Yours

Whichever model you choose, these techniques will genuinely shrink your invoice:

1. Design for prefix caching. Put all stable content — system prompts, tool schemas, few-shot examples, long-running rules — at the very start of the prompt, and variable content — user input, the current question — at the end. Keep the prefix byte-identical across turns; any change kills the cache hit.

# Stable prefix (cache-friendly, place first)
- System prompt and role definition
- Tool / function schema definitions
- Few-shot examples and output format specs
- Persistent knowledge, safety, and formatting constraints
# Variable suffix (place last)
- Current user input
- Incremental conversation history
- Specific question and parameters

1. Control thinking budget. If the model exposes thinking tiers or budget parameters, set them by task difficulty. Never default to maximum on simple tasks — output is the most expensive rate, and every thinking token counts.
2. Route by model tier. Use rules or a lightweight classifier to send easy requests to a smaller model and reserve the flagship for hard ones. A hybrid routing strategy almost always beats calling the flagship for everything.
3. Constrain output. Structured output, field trimming, and max-length limits all shave off unnecessary tokens.
4. Batch where possible. For offline workloads, evaluate batch APIs; some inference services offer discounted rates for bulk requests.
5. Monitor and alert. Use logging and monitoring capabilities from cloud platforms to track token consumption by task type, establish a cost baseline, and set alerts for anomalies — you cannot optimize what you do not measure.

6. Conclusion: “Expensive or Not” Is an Engineering Question

Back to the original question: is a 5× price hike expensive? The answer depends on your frame of reference. For human-facing conversation, it almost certainly is. For automated, per-task workloads, it may sit in the same band as comparable flagship models — or it may be crushed by lightweight alternatives that are an order of magnitude cheaper. In the open ecosystem, the official price tag reads more like a declaration than a price list.

The only reliable methodology boils down to three rules: look at the bill, not the sticker price; look at cost per task, not per-token rate; and compute on a benchmark task set, not on intuition. Do those three things and “expensive or not” stops being a matter of opinion and becomes an engineering problem with a clear answer.