Why Large Model API Price Hikes Deserve a Closer Look: From Token Unit Price to Real Task Cost

Why the Latest Price Adjustments Matter

Recent generations of large language models from multiple vendors have come with noticeable price increases. In some cases, output token pricing has reached CNY 100 per million tokens, with input at CNY 20 and cached input at CNY 1—several times higher than previous versions. Community discussions often focus on whether the unit price feels “too expensive,” but that intuition misses the bigger picture.

Evaluating the cost of a large model is a systems engineering problem. The unit price is only the surface; the real expense depends on task structure, context length, number of calls, and cache strategy. This article breaks down how to think about it properly.

Understanding the Pricing Structure: Three Price Tiers and Their Logic

Most major model APIs charge per token across three tiers:

• Input tokens: The part the model “reads.” Usually the cheapest.
• Output tokens: The part the model “writes.” Typically priced 3–5× higher than input.
• Cached input tokens: Input that hits a prefix cache. Priced as low as one-tenth of the standard input rate, sometimes even lower.

Why Output Costs More

From an inference engineering perspective, the prefill phase processes an entire input sequence in parallel, achieving high compute utilization. The decode phase, by contrast, generates tokens autoregressively—one at a time. Each generated token requires reading and writing the full KV cache, making memory bandwidth the bottleneck. The result is fewer tokens produced per unit of compute. Higher output prices fundamentally reflect the lower hardware efficiency of the decode stage.

Why Cached Tokens Are Cheap

Prefix caching retains the computed results of repeated context—system prompts, tool definitions, long documents—so subsequent requests with the same prefix skip re-prefill entirely. This saves real compute for the provider, who can then pass the savings to users. For the customer, it means context organization is itself a cost variable. With the example pricing above (output at CNY 100, input at CNY 20, cache hit at CNY 1), moving reusable content into a cacheable prefix yields an order-of-magnitude benefit.

Changing the Metric: From Token Unit Price to Per-Task Cost

Cost Formula for a Single Call

Single call cost = (non-cached input tokens × input price) + (cached tokens × cache price) + (output tokens × output price).

The Cost Multiplier Effect in Agent Scenarios

In multi-turn agent tasks, each turn re-sends the full conversation history to the model. Input token counts grow roughly linearly—sometimes super-linearly—with the number of turns, while every turn also adds new output. Industry benchmark data shows that running a typical agent task on a flagship model can average close to USD 1 per task. Cost-efficient models can complete the same task for a few cents to a few dozen cents—a difference of an order of magnitude.

This raises an important question: before debating whether a model is “expensive,” clarify the unit of measurement. Token-based, call-based, and task-based pricing can lead to very different conclusions. A model that looks pricey per token may actually be cheaper per task if it completes work reliably with fewer retries.

A Reusable Cost Estimation Script

def call_cost(cached_in, fresh_in, out,
              p_cache=1, p_in=20, p_out=100):
    """Unit prices in CNY per million tokens; returns cost per call in CNY."""
    return (cached_in / 1e6 * p_cache
            + fresh_in / 1e6 * p_in
            + out / 1e6 * p_out)
# Example: 4,000 tokens of system prompt fully cached,
# plus 2,000 fresh input tokens and 1,000 output tokens
per_call = call_cost(cached_in=4000, fresh_in=2000, out=1000)
# ≈ CNY 0.144, with output contributing ~0.1 (nearly 70%)
task_cost = per_call * 20  # a 20-turn agent task ≈ CNY 2.9

This example highlights two often-overlooked facts. First, output tokens are likely the dominant cost driver—controlling output length is more effective than trimming input. Second, multi-turn costs accumulate with each turn, making context management a direct lever on your bill.

Building a Fair Comparison Framework

Compare Within the Same Tier, Separately Across Tiers

Directly comparing unit prices between models of different capability levels is not meaningful. A more rigorous approach: first fix the capability tier—whether the model reliably completes the task and how much human rework is needed—then compare per-task costs within that tier. When comparing across tiers, introduce the concept of “cost per correctly completed task.” A cheap model that requires three retries may end up more expensive than a pricier model that gets it right the first time.

Self-Hosting Open-Weight Models Is a Different Cost Model

Within the open-weight ecosystem, per-task costs also vary widely. Self-hosted costs do not show up as token prices. Instead, they are the sum of GPU depreciation or rental, inference throughput, cluster utilization, and engineering overhead. In practice, self-hosting only shows a clear advantage for sustained, high-volume, batch-friendly workloads with flexible timing. Spiky or long-tail traffic is usually cheaper through an API. For teams running comparisons, cloud platforms such as Baidu AI Cloud offer model serving alongside general-purpose services like logging and monitoring, making it possible to put measured costs from both deployment routes into the same report and break them down by application.

Engineering Responses to Price Increases

1. Tiered routing: Route simple tasks—classification, extraction, formatting—to lightweight models. Reserve flagship models for complex reasoning and planning. This alone often brings significant total cost reduction.
2. Prefix caching first: Keep stable, unchanged content at the front of the context to maximize cache hits. Place dynamic content later.
3. Output control: Use structured output and explicit length constraints to compress the number of generated tokens.
4. Batch and async: For non-real-time workloads, use batch channels to access lower price tiers.
5. Cost observability: Log token consumption by application and feature, configure cost alerts, and make the cost impact of every prompt change visible and attributable.

Conclusion

A price increase does not necessarily mean the model is “more expensive.” If the new generation completes tasks in fewer turns and with fewer retries, the per-task cost may even go down. What is needed is a measurement system built around tasks, not tokens: understand the inference economics behind the three price tiers, and use data rather than intuition to evaluate each model upgrade and each price change. In a period of rapid model iteration, the more important question than “which model is cheaper” is: what pricing granularity and model combination fit my task structure?