1. Price Cuts ≠ Lower Bills: Start With the Pricing Structure

A model vendor recently cut output-side prices from $50 to $40 per million tokens and input-side prices from $10 to $8. That is a 20% reduction, and media coverage calls it a “20% drop in API costs.”

The statement is not wrong, but it answers a narrow question: how much the unit price fell. It does not answer the one developers actually care about. When I do the same work, how much more will I pay this month?

The conclusion looks very different once you put all of the same vendor’s price tiers side by side. The table below uses illustrative prices (USD per million tokens) to show how the math works:

Tier	Input	Output	Multiple vs. flagship (after the cut)
Flagship	8	40	1×
Mainstream	4	20	0.5×
Light	2	10	0.25×
Ultra-fast	0.5	2	0.06×

After the cut, the flagship’s output price is still 2× the mainstream tier and 4× the light tier. Take a common task size, one call that reads 100K tokens and writes 10K tokens:

• Flagship: 0.1 × 8 + 0.01 × 40 = $1.20
• Mainstream: 0.1 × 4 + 0.01 × 20 = $0.60

For this single call, the flagship costs twice as much. That is not in dispute. The question worth asking next is what lies beneath. To get this job done correctly, how many calls does each tier need?

2. Why Unit Price Can Mislead You: The Overlooked Multipliers

Per-token pricing is a linear billing model, but real tasks do not consume tokens in a linear way. At least four variables can either amplify or erase the gap shown on a price sheet.

1. Hidden output token inflation. Reasoning models generate a chain of thought before they give an answer. The useful answer to a question may be only 200 tokens, while the intermediate reasoning can be tens of times that size. A model with a higher unit price can end up with a smaller total output if it gets the answer right in one pass.

2. Retry and self-correction rounds. Weaker models produce code that fails tests more often, which leads to a “generate → error → read a long stack trace → regenerate” loop. Each round re-reads almost the entire context, and the repeated input-side billing often hurts more than the output side.

3. Repeated context reads. Agentic tasks keep appending repository files and tool results to the context, so the input tokens on round 10 can be 8× those on round 1. Here the unit-price gap gets multiplied by the number of rounds.

4. Prefix cache hit rate. Reusable content such as system prompts, codebase summaries and tool definitions costs far less to bill when the prefix cache hits. Different tiers have different caching policies and hit rates, which can narrow or widen a “4× gap” on the price sheet.

Putting these together gives you a more honest metric:

Effective task cost = (cost per call × average rounds × (1 + retry rate)) / task success rate

PRICING = {  # Illustrative prices in USD per 1M tokens
    "flagship": {"in": 8.0, "out": 40.0},
    "main":     {"in": 4.0, "out": 20.0},
    "light":    {"in": 2.0, "out": 10.0},
}
def task_cost(tier, in_tok, out_tok, rounds=1, retries=0.0, success=1.0):
    p = PRICING[tier]
    per_call = in_tok / 1e6 * p["in"] + out_tok / 1e6 * p["out"]
    return per_call * rounds * (1 + retries) / max(success, 1e-6)
# The same job: 0.05 success rate with the light tier vs. 0.9 with the flagship
print(task_cost("light",    100_000, 10_000, rounds=4, retries=1.5, success=0.05))
print(task_cost("flagship",  60_000,  4_000, rounds=1, retries=0.2, success=0.90))

What matters in this sample code is not the numbers but the reminder that follows from them. The success rate in the denominator usually moves the result more than the unit-price difference in the numerator. If you do model selection by dividing numbers from a price sheet, you leave out the entire denominator.

3. Reasoning Tiers: The New Cost Lever

Newer models generally expose “thinking depth” as an adjustable tier (low/medium/high). This turns cost control, which used to mean switching to a different model, into a continuous dial within the same model. It also directly changes how you judge value for money.

Vendor-published task-level comparisons show one pattern again and again. A flagship-level model’s score can often be matched by a higher-tier model running at a lower reasoning tier. The higher-tier model may cost half as much to complete the task, and on some knowledge benchmarks less than one-tenth. That data comes from vendor self-assessments, so believe only half of it. The mechanism behind it is sound, though. A stronger model needs less search depth and fewer correction rounds, takes shorter detours and consumes fewer total tokens.

The engineering strategy is therefore to start with the mainstream tier at medium reasoning, and raise both the model tier and the reasoning tier together only when a task is judged “hard.” Do not run everything at maximum by default. Maximum settings are the easiest cost trap to fall into, because they treat the ceiling of capability as an everyday need.

4. The Capability Ceiling: Long-Horizon and Multi-Agent Workloads

Performance on long-horizon tasks is more revealing than benchmark scores. Two kinds of cases in the industry are worth watching:

• Building a system from an empty directory. An agent is given an empty folder. A controller coordinates over a thousand sub-agents that divide the work and run continuously for several days. In the end they produce a bootable prototype system that includes drivers, file storage and a login interface.
• Large-scale parallel formal proofs. One theorem formalization task used to require an internal research model to run continuously for 11 days. It was completed in about 30 hours using roughly a thousand agents in parallel.

These cases share one feature. The task is no longer carried by a single call but by an orchestration layer. The cost structure changes with it. The controller model’s unit price drives scheduling and decision overhead, while the sub-agents’ unit price drives total execution overhead. When you build a budget, estimate the two separately, or you will badly underestimate the total. Sub-agents typically consume one to two orders of magnitude more tokens than the controller makes decisions with.

The selection advice is clear. Only these long-running tasks justify the most expensive tier. Writing a weekly report, editing a document, looking things up or writing a few dozen lines of script needs the light tier or even the ultra-fast tier. Using the most expensive tier for them is a classic case of misallocated resources.

5. Don’t Use Public Leaderboards as a Procurement Basis

Public benchmark scores are increasingly poor at reflecting real delivery quality, for three reasons. Data contamination happens when training corpora overlap. Task distribution shift means benchmark questions are not the same kind as business questions. Single-metric distortion comes from pass rates, which do not show how much human rework is needed.

Delivery-oriented evaluations are more useful. They focus on whether a model can complete real remote outsourcing tasks on its own, and measure end-to-end delivery rate, how often a human has to step in and what share of work has to be redone.

For procurement decisions, though, the most reliable approach is still a small in-house eval set. Take 50–200 real tasks from your own business and fix the scoring rubric. Run different combinations of tiers and models, and record success rate, average rounds and actual billing. Building this eval set usually costs far less than making one wrong model choice.

6. Build Cost Control Into the Architecture

Tiered routing. Routing by task characteristics rather than habit is the single most valuable step:

def route(task):
    if task.est_input_tokens < 8_000 and not task.needs_deep_reasoning:
        return "light", "low"          # Summarization, rewriting, classification, extraction
    if task.horizon_minutes < 30 and task.max_tool_calls <= 5:
        return "main", "medium"        # Routine coding and Q&A
    if task.horizon_minutes >= 60 or task.subtask_count >= 20:
        return "flagship", "low"       # Long-horizon orchestration; start at the low reasoning tier
    return "main", "high"              # One-off hard problems: raise reasoning, not the model

Caching and batching. Route system prompts, tool definitions and codebase summaries through prefix caching. Send offline evaluations and bulk document processing through an asynchronous batch channel, use a message queue to smooth traffic spikes, and take advantage of non-real-time discounts to lower unit cost.

Observability and budget guardrails. Use a logging service to record the model tier, reasoning tier, input and output tokens and cache hit volume for every call. Aggregate them into two core metrics: cost per task and cost per successful task. On the monitoring and alerting side, set a per-task token cap, a per-user daily budget and alerts for abnormal retry rates. On Baidu AI Cloud, general-purpose capabilities such as model serving, object storage, a message queue, logging and monitoring are usually already available. The focus should be on making token consumption a first-class metric instead of finding problems only when the monthly bill arrives.

7. Conclusion and Decision Checklist

A price cut is worth watching, but it changes the unit price, not the bill. Before deciding to upgrade a model, ask these four questions in order:

1. Has the per-call failure rate for this task already reached the current tier’s limit? If not, don’t upgrade.
2. After upgrading, how much will average rounds and retry rate drop? Put the multipliers into the comparison before looking at unit prices.
3. Is it a short-horizon, single-step task or a long-horizon orchestration task? For the former, raise the reasoning tier first. Only the latter justifies raising the model tier.
4. Does a mainstream or light tier from the same vendor already meet an acceptable standard on your in-house eval set? If so, use it, because the premium buys you nothing.

In one sentence: pick the model by the job, and set the tier by the bill. A price sheet is for budget estimates. Task-level cost is for selection decisions. The two should never be mixed up.