From Benchmarks to Price Lists: The Cost Coordinate System for LLM Selection Is Being Redrawn

When a flagship model launches, the most telling section is often not the leaderboard. A more useful reading order is: price table first, capability table second, and that third table the vendor itself admitted two days later it got wrong—availability. Behind this order lies a fundamental shift in how developers and enterprise teams evaluate large language models.

1. Capability: Hard Metrics Have Reached the Top Tier

Look at the specs first. A recently released flagship model was derived from large-scale post-training of its predecessor, scaled to 300 billion parameters, extended to a 1-million-token context window, and natively supports visual understanding. These are the baseline specifications.

The benchmark numbers are compelling: 65.7% on an agentic operations benchmark, beating one leading overseas model’s 58.2% and trailing another overseas flagship’s 66.4% by just 0.7 percentage points. On the comprehensive evaluation leaderboard, it ranks fourth overall and first among domestic models, 11 points above its own predecessor.

The significance: the capability gap has narrowed to single-digit percentage points. When scores approach the ceiling, marginal improvements shrink, and decision weight naturally shifts to other dimensions. Capability still matters—it is just no longer the sole differentiator.

2. Pricing: The Strategy Behind the Unit-Price Drop

What really demands attention is the price. The listed rates per million tokens are: ¥1.3 for cached input, ¥13 for uncached input, and ¥68 for output. Compared to the previous generation (¥2 / ¥20 / ¥100), input dropped 35% and output dropped 32%.

Converted to USD list prices (roughly $2 for cached input, $10 for output), the model costs one-fifth of one leading overseas closed-source model ($10/$50) and half of another overseas flagship ($4/$20).

What does that mean in practice? Have the model read a 300-page bid document and then cross-check a round of data, and the bill is a fraction of what the top closed-source model would charge. When a domestic flagship pushes pricing to 20% of the global top tier, the evaluation framework must change: stop asking “how close are they to the leaders?” and start asking “how much does a real task cost, how many errors, how many retries?”

For developers, this shift is tangible. Previously, teams checked benchmarks first and treated price as a budget constraint. Now, price itself is part of capability—how many real tasks a unit of budget can complete is the comparable metric.

3. Availability: The Lesson the Vendor Itself Validates

Two days after launch, the vendor pinned an apology in the comment section of the announcement post: some subscribers hit rate limits while using the new model, tasks stalled before completing within the five-hour rolling window quota, and multi-agent tasks showed abnormal token consumption. The remedy was a full quota reset for all subscribed members plus a 25% monthly bonus.

This incident is more informative than any benchmark. The vendor’s stated fix direction reveals the true composition of per-task cost: reduce unnecessary sub-agent invocations and shorten unnecessary output.

In other words, token unit price is only one component of the bill. The other major component is the intermediate process the model burns through to finish a task on its own initiative. The price list says ¥13 per million input tokens, but a task that triggers twenty sub-agent calls carries a completely different real cost. This also explains why “highest benchmark score” and “lowest task cost” can both fail to hold true at the same time.

4. Cost Structure: The Hidden Layers Beyond Unit Price

Break a real task’s cost into three layers:

Layer 1 — Explicit token fees. Input and output unit prices multiplied by actual consumption. Transparent, budgetable, and directly reflected in the list price.

Layer 2 — Intermediate process consumption. Sub-agent orchestration, multi-round self-reflection, round-trip tool-call overhead. Not captured by unit price, yet capable of multiplying total cost several times over. A task requiring twenty sub-agent calls may end up more expensive than a single-turn model with a higher unit price, even after a 35% rate cut.

Layer 3 — Rework and failure costs. Incomplete tasks, disqualified outputs, manual re-runs—all count. Rate-limit-induced interruptions are a textbook example: quota is consumed, but no output is produced.

A simplified expression:

Total Cost = Σ(input_tokens × input_price)
           + Σ(output_tokens × output_price)
           + Σ(sub_agent_calls × per_call_overhead)
           + rework_rounds × cost_per_round

Looking at only the first two terms seriously underestimates real spend. The third and fourth terms are direct consequences of availability issues and process bloat.

5. Engineering Perspective: Keeping Cost Predictable

For enterprise users and developers, four practices help build cost controllability:

1. Establish a per-task cost baseline. Do not rely on token unit price alone. Measure end-to-end cost on representative tasks—document review, data verification, code generation, multi-turn QA—including retries and human intervention. Run a fixed test suite once and derive “average cost per task.” That number beats unit price for decision-making.

2. Constrain intermediate processes. Set caps on sub-agent calls, output length, and reflection rounds at the orchestration layer. Much intermediate consumption comes from the model’s own “extra” steps; engineering-side limits are more reliable than relying on model self-discipline.

3. Cache and reuse prompts. Cached-hit input is typically far cheaper than uncached input. For high-repetition scenarios—multi-turn QA over the same document, fixed system prompts—caching strategy directly affects the bill.

4. Treat availability as an SLA item. Factor rate limiting, quota exhaustion, and task interruption into model selection. A low-price model that frequently disrupts tasks can cost more in practice than a pricier but stable one. Compensation can offset the monetary hit but cannot undo business disruption.

6. Industry Context: The Benchmark-to-Price-List Evolution

This lesson is not unique to any one market. Over the past two years, the industry has undergone several coordinate shifts: one overseas vendor first made “inference-time compute” an explicit billing item, turning model launches from benchmark events into pricing events. Another used low-cost long-context windows to win developers, writing “good enough” into mainstream procurement logic. Yet another established a “mid-tier price, flagship output”路线 (route).

The common thread: pulling the competitive axis from “who is stronger” toward “who is more cost-effective.” This domestic flagship’s pricing move—and the availability hiccup that followed—pushed the axis further: it is no longer enough to compare unit prices; you must compare real per-task spend and whether the model delivers reliably under load.

7. Conclusion: Measure Models with a Different Ruler

For developers, the most valuable takeaway from this launch is not the fourth-place benchmark rank or the 30% price cut—it is the table the vendor itself admitted it got wrong. It reminds us:

• Benchmarks measure ceiling, price lists measure unit cost, and availability measures delivery rate. Only the sum of all three reflects usable capability.
• “Highest score” and “lowest task cost” can both be false, because intermediate consumption appears in neither benchmarks nor unit prices.
• The right question is not “how close are they to the leaders?” but “how much does a real task cost, how many errors, how many retries?”

When a domestic flagship prices at 20% of the global top tier, the evaluation framework must adapt. The new ruler measures not how powerful a model is, but how cost-effectively and reliably it completes tasks. That is the answer enterprise users actually need.