画板 1

    Why New LLMs Cut Token Prices but Increase Your Bill

    3 hr ago0 views

    When a new frontier LLM launches, vendors usually highlight a lower price per million tokens. This article explains why lower unit prices do not necessarily mean lower spending: reasoning tokens are billed, longer outputs dominate cost, and high reasoning tiers can multiply per-task expenses. It presents a practical cost model, explains how to compare models at equivalent effort levels, describes when to use standard versus extra-high reasoning tiers, and outlines engineering controls such as task routing, output compression, context caching, internal evaluation sets, cost observability, and isolated execution.

    1. A counterintuitive opening: the unit price falls, but the total bill rises

    Every flagship model release usually comes with three numbers: benchmark score, output speed, and price per million tokens. The first two usually rise, the third usually falls. It is easy to conclude that the new model is stronger, faster, and cheaper.

    But if you switch from “how much does it cost per million tokens?” to “how much does it cost to complete a task?”, the conclusion often reverses. The reason is not complicated. One of the main ways new-generation models improve capability is by allowing the model to perform longer internal reasoning before giving an answer, which means producing many reasoning tokens. If the unit price drops by 20%, but the output volume per task more than doubles, the final bill will rise.

    A common misunderstanding needs to be clarified: reasoning tokens are usually billed. They may not be shown to the end user as final answer text, but they are treated the same as regular output tokens for billing purposes. In other words, “invisible thinking” is still a real expense.

    Note: The scores, token counts, and prices in this article are illustrative examples used to explain cost accounting methods. They do not represent official parameters of any specific product. The method and engineering conclusions can be applied directly to real model selection.

    2. Build the right cost model

    Most teams lose control of model costs because they focus only on the unit price on the price list. A usable cost model should include at least four layers.

    Layer 1: Unit price

    Input and output are priced separately. Output is usually 4 to 6 times more expensive than input. This leads to an important fact: output length is the dominant cost variable, and reasoning levels amplify output length.

    Layer 2: Per-task cost

    1. Cost = (input tokens × input price + output tokens × output price) / 1,000,000

    Using illustrative data, compare three reasoning tiers. Assume the input price is $8 per million tokens and the output price is $40 per million tokens.

    1. def task_cost(in_tok: int, out_tok: int, p_in: float = 8.0, p_out: float = 40.0) -> float:
    2.     """Return the USD cost of a single task. Prices are in USD per million tokens."""
    3.     return (in_tok * p_in + out_tok * p_out) / 1_000_000
    5. tiers = {
    6.     "standard": dict(in_tok=18_000, out_tok=42_000),
    7.     "default": dict(in_tok=22_000, out_tok=96_000),
    8.     "extra-high": dict(in_tok=26_000, out_tok=274_000),
    9. }
    11. for name, t in tiers.items():
    12.     print(f"{name}: ${task_cost(**t):.2f}")
    13. # standard: $2.56
    14. # default: $4.02
    15. # extra-high: $11.17

    Moving from the default tier to the extra-high tier increases output volume by about 2.8 times, and the cost increases by about 2.8 times as well. At this scale, a lower unit price becomes almost negligible.

    Layer 3: Effective cost (the metric that really matters)

    In engineering work, a single call rarely produces a directly usable result. The code may fail to compile, tests may fail, or the review may be rejected. A more meaningful metric is:

    1. Effective cost = per-task cost / first-pass success rate

    A higher-tier model may cost twice as much per call as a lower-tier model, but if its first-pass success rate improves from 40% to 80%, its effective cost may actually be lower. This metric can only be calculated using a team-owned evaluation set. Public leaderboards cannot replace it.

    Layer 4: Opportunity cost

    Stronger models can shorten development cycles and reduce manual rework. For teams whose labor costs are much higher than token costs, pursuing the lowest possible token price may actually end up being more expensive.

    3. Diminishing returns from benchmark leaderboards

    The value of composite intelligence index leaderboards is that they are horizontally comparable, but they have two inherent limitations.

    The first is the nonlinear relationship between scores and task value. Going from 49 points to 65 points looks like a 33% relative improvement, but these scores often concentrate on automatically scorable capability dimensions such as mathematics, multi-step logic, and long-context retrieval. In a specific business context, users may only perceive that answers are “a bit more reliable,” rather than that efficiency has improved by one-third.

    The second is leaderboards do not reflect cost. The same leaderboard entry may have been produced at the default reasoning tier, or it may have been produced at the highest reasoning tier, with costs differing by several times. If you compare two models without fixing the reasoning tier, the conclusion will almost certainly be misleading.

    A more practical approach is to always compare models at the same reasoning tier and list cost as a parallel dimension. Example table:

    Comparison item New model (default tier) Previous flagship (highest tier) Conclusion
    Composite score 62 59 The default tier already surpasses it
    Per-task cost $9.08 $10.04 Lower
    Output speed Faster Baseline Better experience

    The conclusion is clear: the default tier is the cost-performance sweet spot for most scenarios. The highest tier is suitable for a small number of high-value tasks that must be correct on the first try, not for routine batch calls.

    4. Coding capability: real improvements, but calculate per-task costs

    Coding is the area where large models are easiest to quantify and closest to real paid use. New-generation models usually show several characteristics on coding benchmarks:

    • Complex refactoring and cross-file modification improve clearly, which is a direct benefit of long-context reasoning;
    • At the standard reasoning tier, they can approach the previous generation’s highest tier, at roughly half the cost;
    • Pushing them to the highest reasoning tier can still raise the upper limit, but marginal gains fall quickly while costs rise sharply.

    In engineering practice, coding tasks can be divided into three complexity levels and matched to different tiers:

    1. Boilerplate and single-file changes: type completion, unit test generation, formatting, simple bug fixes → low tier + batch processing, optimizing for throughput;
    2. Module-level implementation: adding endpoints, writing data access layers, supplementing test cases → default tier, optimizing for first-pass success rate;
    3. Architecture-level tasks: cross-service refactoring, difficult defect localization, performance analysis, code review → highest tier, accepting a high unit price in exchange for lower rework.

    The key point is that level 3 must have a manual or automated acceptance loop. Otherwise, expensive reasoning may only produce code that needs to be rewritten.

    5. Long-horizon autonomous tasks: the real signal of capability jumps

    More important than leaderboard scores is progress in long-horizon autonomous execution. Consider this validation scenario: starting from an empty directory, the model autonomously orchestrates thousands of subtask agents, runs continuously for multiple days, independently writes code, runs tests, performs self-review, and finally delivers a small operating system that can run on a personal computer—including the kernel, drivers, file system, graphical interface, login, and installer.

    The significance of this kind of demonstration is not how usable the operating system is, but that it validates four things:

    • Task decomposition and parallel orchestration: breaking a large goal into independently advancing subtasks and handling dependencies between them;
    • State persistence in extremely long contexts: keeping the goal and completed work intact over multiple days;
    • Self-verification loops: running tests, reading failure information, fixing issues, and rerunning, rather than generating once;
    • Tool use and artifact management: reading and writing files, executing commands, managing versions and build artifacts.

    These four points are exactly the core of real engineering delivery, and they were the biggest weakness of earlier code completion products.

    At the same time, the risks must be clear:

    • Unpredictable cost: continuous high-tier runs over multiple days can cost hundreds of times more than ordinary interactive use, so hard budget caps and circuit breakers are required;
    • Error accumulation: errors that are not corrected in time in long chains will amplify, and without validation gates the output may be unusable overall;
    • Permissions and isolation: agents that can execute commands and write files must run in isolated environments and must not hold production credentials;
    • Observability: every decision, command, and artifact must be fully recorded; otherwise, failures cannot be reviewed.

    6. Implementation checklist

    1) Tiered routing: make tier selection a runtime parameter

    Do not hardcode the reasoning tier in code. Instead, select it dynamically based on task type:

    1. def pick_tier(task) -> str:
    2.     if task.kind in {"doc", "format", "unit_test", "lint_fix"}:
    3.         return "low"
    4.     if task.files_touched > 5 or task.needs_reasoning_score:
    5.         return "xhigh"
    6.     return "default"
    8. # Key: set budget guardrails for expensive tiers
    9. BUDGET = {"low": 0.05, "default": 0.50, "xhigh": 3.00}  # Per-task cap, USD
    11. def run(task):
    12.     tier = pick_tier(task)
    13.     if task.estimated_cost > BUDGET[tier]:
    14.         tier = downgrade(tier)   # Downgrade or route to a human
    15.     return call_model(task, effort=tier)

    2) Suppress ineffective output

    Output length is the main cause of cost. Constraining the response format (structured output), requiring conclusions before evidence, and prohibiting repeated restatement of the input can all significantly reduce output tokens.

    3) Context and cache governance

    Put stable and unchanged prefixes—system prompts, coding standards, repository structure descriptions—at the front of the context so they can hit the cache. Inject retrieved context on demand rather than stuffing it all in. Summarize and compress long conversations promptly.

    4) Build an internal evaluation set

    Extract 100 to 300 real tasks from historical tickets, annotate expected results, and use them as the sole basis for model selection and tier tuning. The metrics are effective cost and first-pass success rate, not leaderboard rankings.

    5) Cost observability and quotas

    For every call, record the model tier, input and output tokens, latency, and cost. Connect them to logging services and monitoring alerts, and set daily budgets and alert thresholds by project or team. If you skip this step, the bill will inevitably spiral out of control.

    6) Isolated execution environments

    Run long-horizon agents in disposable sandboxes on a container platform. Store code and artifacts in object storage or an artifact repository. Issue credentials through a key management service with short-lived, least-privilege access, and revoke them when the task ends. If you use managed inference services with supporting capabilities such as object storage, container platforms, logging, and monitoring alerts, apply the same isolation and quota principles.

    7. Conclusion

    The value of new-generation flagship models is real: composite capability, coding level, and long-horizon autonomous execution have all made substantial progress, and higher output speed directly improves the interaction experience. But a lower unit price does not mean lower total spending.

    Three pieces of advice for developers:

    • Look at per-task cost and effective cost, not unit price; include first-pass success rate in model selection.
    • The default tier is the best cost-performance choice for most scenarios; reserve the highest tier for a small number of high-value, hard-to-verify tasks, and protect it with budget guardrails.
    • The stronger the capability, the more engineering constraints are needed: routing policies, context governance, internal evaluations, observability, and isolated sandboxes are the five prerequisites for turning new model capability into real delivery efficiency.
    09040904

    Top articles