LLM API Prices Rose 5x: How to Calculate Real Token Costs and Control Your Bill

    A 5x LLM API price increase is easy to react to but hard to evaluate. Simply comparing per-token prices misses the point: different model generations sit in different price bands, and input, output, and cache-hit tokens are priced very differently. This article breaks token-based billing into its three components, explains why output tokens cost several times more than input, and shows how to build a cost model that measures real per-task spend. Through a worked formula, a sensitivity analysis, and cost profiles for long-document, code, and agentic workloads, it demonstrates that output length and cache hit rate are the two biggest cost levers. It also covers how to evaluate subscription plans versus pay-as-you-go, engineering practices for cost observability, context governance, model routing, and how to avoid vendor lock-in. By the end, you can rebuild your cost model, find practical optimization paths, and judge whether a model upgrade actually pays off.

    1. Introduction: After a 5x Price Increase, How Much Does the Bill Really Go Up?

    A new-generation model’s API pricing on a major LLM platform has recently drawn a lot of discussion. According to the public figures, the new model charges roughly 20 CNY per million input tokens, 100 CNY per million output tokens, and 1 CNY per million cache-hit tokens — putting the overall price level at about 5x the previous generation.

    Many practitioners’ first reaction is “that’s expensive,” while others argue “there’s a reason for it.” But drawing conclusions from unit price alone isn’t rigorous, because three things are easy to overlook:

    • Different baselines: models across generations, scales, and capability tiers simply don’t sit in the same price band;
    • Different structures: the gap between input, output, and cache-hit prices is large, and price sensitivity varies by scenario;
    • Different yardsticks: “how much per million tokens” and “how much per task” are separated by two conversions — token consumption and task success rate.

    A more useful question is: under the new pricing, how do you accurately measure cost, find practical optimization paths, and judge whether a model upgrade is worthwhile? This article follows a clear thread — break down the billing structure → measure real cost → compare plans against usage → evaluate upgrade ROI → apply engineering techniques.

    2. Breaking Down the Billing Structure: Three Token Types, Three Prices

    Most mainstream LLM APIs use a pay-per-token model. A token is the basic unit a model uses to process text; one token corresponds to roughly 0.5–1.5 characters of text (the exact ratio depends on language and tokenizer), and structured content such as code and JSON usually has a higher token density. The bill is made up of three parts:

    Billing item Meaning Illustrative price (CNY per 1M tokens) Multiple vs. input price
    Input tokens System prompt, context, and user request sent to the model 20 1x
    Cache-hit tokens Input portion identical to cached content and reusable 1 1/20
    Output tokens Content generated and returned by the model 100 5x

    Note: prices are illustrative values, used only to demonstrate the calculation method. Actual billing follows each platform’s documentation.

    There’s clear technical logic behind these numbers.

    First, output costs far more than input. The input phase (prefill) can process an entire prompt in parallel, with computation that resembles matrix multiplication. The generation phase (decode) must produce output token by token autoregressively, where each step depends on the previous one and can’t be fully parallelized. Decoding demands far more from memory bandwidth, KV Cache capacity, and batch scheduling, and serving the same throughput requires more compute — which is why output unit prices are commonly several times the input price.

    Second, cache-hit prices are far below the standard input price. Prompt caching keeps frequently reused context resident in memory on the inference side, eliminating repeated computation. Returning the caller 1/20 of the price is, in essence, an incentive to optimize how context is organized.

    Two important implications follow:

    • Output length has the biggest impact on the bill. With output priced at 5x input and cache hits at 1/20 of input, the same number of output tokens costs 5x uncached input and 100x cache-hit content.
    • Cache hit rate is one of the highest-leverage variables in cost governance. Every hit on the input side pushes the unit price for that portion down to 1/20.

    A 5x increase over the previous generation also means the absolute values across all three tiers — input, output, and cache — rise together. The “loose” calling patterns that were tolerable under the old pricing are now significantly amplified. If call volume stays the same, the bill theoretically grows by about 4x — but the room for optimization through caching and context management is amplified proportionally. So the first step in evaluating the new pricing isn’t comparing unit prices; it’s rebuilding your own cost model.

    3. Measuring Per-Task Cost: From Unit Price to Bill

    3.1 The Basic Formula

    Let the input tokens for a single call be I_in, of which I_cache hit the cache and the remainder is I_miss = I_in − I_cache, and let output tokens be I_out. The cost of one call is:

    1. C_task = I_miss / 1e6 × P_in
    2. + I_cache / 1e6 × P_cache
    3. + I_out / 1e6 × P_out

    With output tokens held fixed, define the input cache discount S:

    1. S = I_cache / I_in × (1 − P_cache / P_in)

    In this example P_cache / P_in = 1/20, so S = 0.95 × cache hit rate. In other words, each 10-percentage-point increase in cache hit rate reduces the input-side cost by about 9.5%.

    3.2 A Worked Example

    Take a typical coding-assistant task: system prompt and tool definitions of about 1,500 tokens, codebase retrieval context and conversation history of about 6,500 tokens, and model output of about 1,500 tokens. With common engineering practices — a uniformly maintained system prompt, a stable conversation prefix, and fixed tool definitions — a cache hit rate of about 80% is achievable.

    1. # Price per 1M tokens (CNY); illustrative values, actual per each platform's billing docs
    2. PRICE = {"input": 20.0, "cache": 1.0, "output": 100.0}
    3. M = 1_000_000 # Conversion base for per-1M-token pricing
    4. def estimate_cost(input_tokens, cached_tokens, output_tokens):
    5. """Estimate the cost of a single call (CNY)"""
    6. assert 0 <= cached_tokens <= input_tokens, "cached tokens cannot exceed input tokens"
    7. uncached = input_tokens - cached_tokens
    8. in_cost = uncached / M * PRICE["input"]
    9. cache_cost = cached_tokens / M * PRICE["cache"]
    10. out_cost = output_tokens / M * PRICE["output"]
    11. return {
    12. "input": round(in_cost, 4),
    13. "cache": round(cache_cost, 4),
    14. "output": round(out_cost, 4),
    15. "total": round(in_cost + cache_cost + out_cost, 4),
    16. }
    17. def month_bill(cost_per_task, tasks_per_day, days=30):
    18. """Estimate the monthly bill from average daily task volume (CNY)"""
    19. return cost_per_task * tasks_per_day * days
    20. if __name__ == "__main__":
    21. c = estimate_cost(input_tokens=8000, cached_tokens=6400, output_tokens=1500)
    22. print(c) # {'input': 0.032, 'cache': 0.0064, 'output': 0.15, 'total': 0.1884}
    23. print(month_bill(c["total"], tasks_per_day=1000)) # 5652.0

    The results:

    • A single task costs about 0.1884 CNY, of which output accounts for 79.6%, input for 17.0%, and cache hits for only 3.4%;
    • At 1,000 tasks per day, the monthly bill is about 5,652 CNY;
    • With no caching at all, the same input and output would cost 0.31 CNY per task, or about 9,300 CNY per month — meaning this one factor alone adds about 3,648 CNY in monthly spend.

    Running a sensitivity analysis with cache hit rate and output length as variables (1,000 tasks per day):

    Scenario Cache hit rate Output tokens Cost per task (CNY) Monthly cost (CNY)
    Baseline 80% 1500 0.1884 5652
    Cache miss 0% 1500 0.3100 9300
    Near-full cache hit 95% 1500 0.1632 4896
    Doubled output (no cache) 0% 3000 0.4600 13800
    Doubled output (with cache) 80% 3000 0.3384 10152

    You can see that when output doubles and caching is absent, the monthly cost nearly doubles to 13,800 CNY — about 2.5x the baseline. Even with caching working correctly, cost rises by roughly 80%. For long-output tasks, controlling output length is often more effective than optimizing the input.

    3.3 Cache Rules Determine Your Measurement Basis

    The measurement above is an “engineering basis.” Actual billing follows the platform’s cache rules. Four points deserve attention in practice:

    1. Activation: some platforms enable prompt caching by default and callers just need to structure requests well; others require explicitly marking cache points, so callers must declare the boundaries of cacheable content themselves.
    2. Matching granularity: token-level matching versus whole-request matching, the minimum cacheable token count, and whether prefix matching is distinguished from exact matching.
    3. Lifecycle: how long the cache is retained, invalidation conditions, and whether it’s shared across regions or availability zones.
    4. Extra fees: whether cache writes (storage) are billed separately and whether they count toward total input tokens.

    In addition, whether input and output include multimodal content (such as images) and whether token counts are based on characters or tokens should also follow the official documentation. It’s a good idea to turn these rules into test cases and recalibrate the cost model each time the pricing policy changes.

    4. Cost Profiles for Three Workload Types: Long Documents, Code, and Agents

    Different workloads have very different token compositions. In the table below, long-document Q&A and code generation assume 80% of input can hit the cache; agentic tasks involve many tool-call loops with context growing and changing every round, estimated conservatively at 30%.

    Workload type Input tokens (cached / total) Output tokens Cost per task (CNY) Cost structure Main optimization lever
    Long-document Q&A 45000 / 50000 800 0.225 Input 54.4%; cache offsets most input cost Compress retrieval results, chunk documents, raise cache hit rate
    Code generation & review 6400 / 8000 1500 0.1884 Output 79.6% Control output length, limit change scope, tiered routing
    Agentic tool calls 9600 / 12000 3000 0.366 Output 82.0% Reduce task steps, truncate tool returns, history summaries, budget circuit breaker

    The common thread across all three: even with caching working well, output accounts for about 80% of cost. So “cutting unnecessary generation” is the core lever for reducing cost.

    Specific approaches include: ask the model to return only a change diff rather than entire files; use structured-output constraints to reduce explanatory text; set a maximum number of tool-call rounds and a token budget per task to keep agents from looping; and truncate or summarize large logs and page content from tool returns before passing them back.

    5. Beyond Pricing: Subscription Plans Versus Pay-as-You-Go

    In addition to pay-as-you-go, many platforms offer monthly subscription plans. These plans typically include a base allowance or model-call quota — for example, an entry-level plan around 199 CNY/month and an advanced plan around 699 CNY/month.

    A widely circulated claim suggests dividing the monthly fee by “the monetary value of the in-plan quota converted at official usage prices” to get a rough discount ratio. For instance, if an entry plan maps to a 1,200–2,800 CNY allowance, the discount is roughly 1:6 to 1:14. The same argument is often used to claim that one product is “far more expensive” than overseas alternatives — reasoning that overseas mainstream services commonly offer, say, $400–800 of quota on a $20 tier and over $10,000 on a $200 tier, implying discount ratios above 20x.

    But this comparison has at least three methodological problems:

    1. Opaque conversion basis: it’s often unclear whether the quota is converted at input price, output price, or a blended price, whether cache discounts are deducted, and whether it covers only some models. A “20x starting” figure is likely an estimate rather than a verifiable billing rule.
    2. Available quota ≠ actual cost: how much quota you can call is not the same as how much a task actually costs. However large the discount, it only becomes meaningful when translated into “cost per task.”
    3. Cost-structure differences are flattened away: context length, token consumption, and task success rate aren’t part of the comparison. If a certain agentic task type inherently uses fewer tokens on average, the advantage implied by the conversion multiple would be significantly overstated.

    Returning to the core data of the original discussion: the average per-task consumption in one agentic-capability benchmark is roughly on par with the new model. This means that under the same task load, the bill gap between the two products is essentially just the price gap.

    Even where a price gap objectively exists, it doesn’t determine which product is better. A more sensible evaluation framework splits “capability” and “cost” into two independent axes:

    • Vertical axis: task success rate, first-pass rate, and manual rework rate on your own business evaluation set;
    • Horizontal axis: cost per task — the C_task computed above.

    Judge whether an upgrade is worthwhile by “quality improvement ÷ cost increase”: the larger the ratio, the more justified the upgrade; below 1 usually means the price increase can’t be offset by quality gains. Of course, for high-quality must-have scenarios like compliance auditing or complex code refactoring, a business may accept a lower cost-performance ratio — that’s a business tradeoff, not a technical judgment.

    Back to subscriptions: evaluating a plan comes down to three questions:

    1. Is the discount real: rely on the official settlement bill, not the converted quota value.
    2. Does the usage structure match: does the plan’s quota mix fit your input/output/cache ratio, and how is overage priced?
    3. Is unused quota wasted: does it reset monthly or roll over, does it require multiple-account sharing, and are there seat limits?

    A rough but practical rule: when pay-as-you-go monthly spend stays several times above a higher-tier plan’s monthly fee, and usage is stable and predictable, the plan’s cost advantage usually begins to show. The exact multiple depends on the overage pricing rules.

    6. Engineering Methods for Getting the Total Right

    6.1 Unified Measurement: Turn Cost into Observable Data

    No data, no optimization. Log the following fields for each call, write them uniformly to a logging service, and connect them to a monitoring and alerting platform:

    1. {
    2. "request_id": "req-20250612-000173",
    3. "timestamp": "2025-06-12T10:21:33+08:00",
    4. "model": "<logical model identifier>",
    5. "task_type": "code_review",
    6. "tenant_id": "team-payment",
    7. "input_tokens": 8000,
    8. "cached_tokens": 6400,
    9. "output_tokens": 1500,
    10. "latency_ms": 3860,
    11. "status": "success"
    12. }

    The key is to use a “logical model identifier” rather than a hardcoded model version number. When the platform upgrades a model version, you only need to adjust the mapping config to achieve a smooth switch and canary rollback, avoiding incidents caused by version numbers scattered across the codebase.

    6.2 Build a Cost Dashboard: Break Metrics Down to Task Granularity

    On top of the logs, set up at least the following metrics with alerting:

    • Task dimension: average cost per task, cost per thousand tasks, and cost share broken down by business line or tenant;
    • Efficiency dimension: cache hit rate, input/output token ratio, and average output length;
    • Quality dimension: task success rate, first-pass rate, and retry rate;
    • Risk dimension: daily budget attainment, day-over-day spikes, and timeout and rate-limit error rates.

    Alerting promptly when the cache hit rate drops noticeably or per-task cost spikes day over day often catches context-construction defects before the problem grows.

    6.3 Context Governance: The “Foundation” That Determines the Bill

    • Slim the system prompt: version it, remove redundant constraints and outdated examples, fix field order, and ensure the serialized byte sequence is stable;
    • Rolling summaries of history: compress earlier messages in long sessions into summaries, keeping only the most recent few rounds verbatim;
    • Truncate tool returns: cap the length of logs, search results, and web content, summarizing first and passing back later when necessary;
    • Retrieval reranking: retrieve with keyword search combined with vector search, then rerank, injecting only the Top-K and filtering out low-relevance content;
    • Converge tool definitions: reduce the number and descriptions of tools, and remove low-frequency ones.

    The code below demonstrates how to assemble context within a budget while prioritizing a stable cache prefix:

    1. def assemble_context(system_prompt, docs, history, user_msg, counter):
    2. """Assemble context within a budget, prioritizing a stable cache prefix (illustrative)"""
    3. budget = {"system": 2000, "history": 3000, "docs": 5000, "reserve": 500}
    4. used = {"system": 0, "history": 0, "docs": 0}
    5. messages = [{"role": "system", "content": system_prompt}] # prefix: keep stable
    6. used["system"] = counter(system_prompt)
    7. # Trim retrieved docs in relevance order: keep only docs that fit fully within budget
    8. kept = []
    9. for doc in docs:
    10. n = counter(doc)
    11. if used["docs"] + n > budget["docs"]:
    12. break
    13. kept.append(doc)
    14. used["docs"] += n
    15. # Keep history from the most recent turn backward; drop earliest messages when over budget
    16. kept_history = []
    17. for msg in reversed(history):
    18. n = counter(msg["content"])
    19. if used["history"] + n > budget["history"]:
    20. break
    21. kept_history.append(msg)
    22. used["history"] += n
    23. messages.extend(reversed(kept_history))
    24. for doc in kept: # dynamic content goes in the user message to avoid breaking the cache prefix
    25. messages.append({"role": "user", "content": f"<doc>{doc}</doc>"})
    26. messages.append({"role": "user", "content": user_msg})
    27. return messages

    6.4 Cache-Friendly Design: Treat “Hits” as First-Class

    • Prefix layout: stable content (system prompt, tool definitions, fixed examples) first, dynamic content (retrieval results, user input, timestamps, random IDs) last;
    • Stable ordering: sort list-type context by a fixed rule before concatenation, and append conversation rounds only at the tail;
    • Prompt reuse: proactively rebuild the cache after changing the system prompt, avoiding old caches that occupy memory but no longer hit;
    • Capacity monitoring: when nearing the context-window limit, compress the dynamic part first while keeping the prefix unchanged.

    Here’s a counter-example. Simply appending the current time to the end of the system prompt makes the prefix different on every request, invalidating the cache:

    1. # Bad example: dynamic content embedded in the prefix changes the prefix on every request
    2. bad_system = f"You are a coding assistant. Current time: {datetime.now().isoformat()}. Follow these rules: ..."
    3. # Good example: move dynamic info to the user message
    4. good_system = "You are a coding assistant. Follow these rules: ..."
    5. messages = [{"role": "system", "content": good_system},
    6. {"role": "user", "content": f"Current time: {datetime.now().isoformat()}\nUser question: ..."}]

    6.5 Tiered Model Routing: Let the Right Model Do the Right Job

    Split requests into simple, medium, and complex tiers, use a lightweight classifier or a small model to judge intent, and route each to a model of the appropriate capability tier. Illustrative code:

    1. def route_model(request, classifier, stats):
    2. """Route models by task difficulty (illustrative; falls back to the default model when no rule matches)"""
    3. if classifier(request) == "simple":
    4. return "lite"
    5. if request.estimated_tokens > stats["ctx_p95"]:
    6. return "flagship_long_context"
    7. return stats.get("default_model", "standard")

    Let the flagship model’s per-task cost be C_h, the lite model’s be C_l, and the share of simple tasks be p. The average cost after tiered routing is p × C_l + (1 − p) × C_h.

    For example, with C_h ≈ 0.19 CNY, C_l ≈ 0.02 CNY, and p = 60%, the average cost is about 0.088 CNY — roughly 54% lower than using the flagship model for everything. However, routing mistakes send a task back to the flagship model, causing duplicate consumption. So the lite model must first be calibrated for accuracy on real traffic, and an automatic escalation mechanism should be set up for failed requests.

    6.6 Output Constraints and Asynchronous Batching

    • Format constraints: constrain output structure with a JSON Schema and specify fields and length limits;
    • Output budgets: set an output-token cap per task, outputting conclusions and changes first, then explanations;
    • Asynchronous batching: run offline tasks like document summarization and data labeling asynchronously via a message queue, and prioritize the platform’s batch-inference capabilities to smooth peaks and lower unit cost;
    • Retry governance: set a retry limit for failed requests to avoid runaway cost from endless retries.

    7. Vendor Lock-In and Migration Risk

    Prices aren’t fixed. The case of a 5x increase for a new-generation model shows that vendors adjust pricing as capabilities improve. Businesses should design decoupling in advance:

    • Abstract model calls through a unified gateway, connecting to different vendors via logical model identifiers;
    • Build your own evaluation set and regression pipeline, and run effect comparisons before switching models;
    • Keep backup models for critical workloads to enable fast switching in the event of outages or price hikes.

    Beyond LLM services, Baidu AI Cloud offers general-purpose cloud products such as log services, monitoring and alerting, object storage, and message queues, which can be used to build the cost-observability and asynchronous-invocation systems described above. The reference architecture can be summarized as: business applications call models through a unified gateway → call logs are written to a log service → usage data feeds into monitoring and alerting → offline tasks are scheduled through a message queue for batch inference → evaluation and rollback flows run on a container platform. For the specific integration of each capability, refer to the corresponding product documentation.

    8. Conclusion

    A 5x increase over the previous generation looks, on the surface, like a question of “is it expensive or not.” In substance, it’s about whether cost structure can be managed effectively. The conclusions of this article come down to four points:

    1. Bill = uncached input × input price + cache hits × cache price + output × output price, where output length and cache hit rate are the two core variables;
    2. A 10-percentage-point increase in cache hit rate reduces the input-side cost by about 9.5% — one of the highest-ROI optimizations available;
    3. The value of a subscription plan depends on usage fit and the real discount; converted quota can’t be equated directly with actual cost;
    4. Whether a model upgrade is worthwhile should be answered by “quality improvement ÷ cost increase,” grounded in your own evaluation set and task-level cost data.

    When cache optimization, context governance, and tiered model routing form a closed loop, the bill under the new pricing is fully controllable — and developers can weigh quality against cost with greater confidence.