Reasoning Effort Tiers and Cost: A Practical Guide to Large Language Model Selection

1. The symptom: lower unit prices, higher bills

After a new flagship large model was released, its composite intelligence index rose from 49 to 65 compared with the previous generation, and it topped a third-party independent benchmark leaderboard. Its price was also reduced by 20%, with input and output rates of $8 and $40 per million tokens. On those metrics alone, the news looks positive.

But the same evaluation contains an easily overlooked set of numbers: at the highest reasoning tier, the model’s average output tokens per task rose from about 118K in the previous generation to about 274K, an increase of roughly 132%. The average cost per task rose from $12.99 to $26.50, almost doubling.

Why does this happen? A simple formula helps:

Cost per task ≈ input tokens consumed × input unit price + output tokens consumed (including reasoning tokens) × output unit price

Unit price is only one multiplier. When the growth in reasoning-token usage far exceeds the decline in unit price, total cost must rise. This is not a billing “trap”; it is a normal characteristic of reasoning-oriented models. These models generate a longer internal chain of reasoning before producing an answer, and that reasoning content is billed as output tokens.

Average figures, however, only reflect an overall trend. Simple tasks and high-difficulty tasks should be measured separately to see the real cost distribution.

2. Reasoning tiers: trading a “thinking budget” for accuracy

Most mainstream reasoning models now offer multiple levels of “reasoning strength.” A higher tier allows the model to generate more internal reasoning before answering, but latency and cost increase accordingly. Common tiers, from low to high, include Low/Minimal, Medium, High, Xhigh, and Max.

Benchmark results show a clear pattern of diminishing marginal returns:

Configuration	Composite intelligence index	Average cost per task (example)
Previous-generation model, Max tier	49	$12.99
Previous-generation stronger variant, Max tier	59	$10.04
New-generation model, High tier (default)	62	$9.08
New-generation model, Max tier	65	$26.50

Two points stand out:

1. The default tier offers better value. The new-generation model uses High as its default tier. Its score already exceeds the score achieved by the previous generation’s top configuration at Max, while its cost per task is lower. For most tasks, the default tier can deliver the effect of the previous generation’s top configuration at a lower cost.
2. Each point at the highest tier is expensive. At Max, the extra 3 points cost an average of $17.42 more, or about $5.8 per point. This does not include longer waiting times or the capacity pressure created by a higher upper limit on per-task cost.

Of course, if each data point carries high manual review or rework costs, the highest tier may still be worthwhile. The key is knowing how much each point of capability is actually worth.

3. Tier-based routing: use the right tier for the right task

Therefore, “which model to use” should not be a global static configuration. It should be split into three dynamic decisions: which model to use, which reasoning-effort tier to enable, and when to escalate.

1. Separate tasks by difficulty

• Lightweight tier: structured tasks such as extraction, classification, format conversion, and summarization. Use a lightweight model or the lowest reasoning tier, and prefer prefix caching and batch requests.
• Standard tier: routine question answering, document understanding, and medium-complexity code completion. Use the default tier.
• Deep tier: complex logical reasoning, architecture design, difficult defect localization, and cross-module refactoring. First compare High and Max through A/B testing, and enable Max only when the improvement is significant.

2. Use failure-based escalation as a cascade

Start with a low-cost configuration, and escalate to higher tiers only after automated checks fail. Common checks include compilation and static analysis, unit tests, JSON Schema validation, rule matching, and result self-checks.

from dataclasses import dataclass
# Execution ladder from light to heavy: (model tier, reasoning effort)
LADDER = [("lite", "low"), ("standard", "high"), ("flagship", "xhigh")]
@dataclass
class CallStats:
    total_tokens: int = 0
    cost: float = 0.0
def solve(task, verifier, budget, stats: CallStats):
    """Solve the task from light to heavy, and stop when the budget is exhausted."""
    if stats.cost >= budget.hard_limit_usd:
        return {"result": None, "reason": "cost_limit_reached"}
    for model, effort in LADDER:
        resp = call_model(task, model=model, reasoning_effort=effort)
        stats.total_tokens += resp.usage.total_tokens
        stats.cost += resp.usage.input_tokens / 1e6 * resp.price_in \
                    + resp.usage.output_tokens / 1e6 * resp.price_out
        if verifier(resp.output):
            return {"result": resp.output, "used": f"{model}/{effort}"}
        # Pause further escalation when the warning threshold is reached and leave the decision to a human
        if stats.cost >= budget.warn_at_usd:
            return {"result": None, "reason": "need_human_review"}
    return {"result": None, "reason": "all_tiers_failed"}

When using a cascade, be careful: if a low-tier result is escalated automatically without effective validation, simple tasks may be pushed to higher tiers, increasing cost. The quality of the verifier determines the return on the cascade strategy.

3. Perform attribution analysis

Record each call’s tier, token count, latency, and cost in logs and monitoring systems, grouped by business line and functional module. An “average cost distribution” measured by task type is more useful for decision-making than the average values in any benchmark report.

4. Reduce repeated computation

• Prompt prefix caching: for fixed prefixes such as long system prompts, knowledge documents, and codebase indexes, use interfaces that support context caching to reuse intermediate computation results. Caching is usually billed by cache hits, subject to the pricing method of the platform used.
• Context compression: retrieve code snippets by relevance, compress logs and stack traces, and avoid repeatedly passing large blocks of context.
• Result reuse: cache final results for stable requests and version intermediate artifacts.

4. Advanced: cost grows multiplicatively for long-running agent tasks

Another trend in reasoning-oriented models is “long-horizon autonomy”: a model can work continuously for hours or even days, planning, executing, testing, and correcting its own work. For example, a model can start from an empty directory, coordinate thousands of subagents to work continuously for 8 days, autonomously complete coding, testing, and code review, and finally deliver a lightweight operating system containing a kernel, drivers, a file system, a graphical interface, login, and an installer. Such demonstrations show that models can support long-range collaboration and autonomous construction, but from an engineering perspective they also reveal the risk of rapidly expanding costs.

The cost of a single conversation depends only on one request. The total cost of a long-running agent task is roughly:

Total cost ≈ cost per call × number of subagents × average number of interaction rounds

Each factor can scale quickly with task complexity, and costs can easily get out of control when the task gets stuck and retries repeatedly. Key governance points are:

• Hard budget limits: set upper limits on total tokens, runtime, and cost at the same time, and support segmented quotas to prevent a single subtask from consuming the entire budget.
• Stage checkpoints: periodically write stage artifacts to object storage, and use a container platform to build reproducible execution environments. After an interruption, the task can resume from the most recent checkpoint instead of starting over.
• Parallelism and validation gates: the number of subagents should be set according to the complexity of the core task, not the total code volume, and should be paired with quality gates such as compilation and testing. Collaboration messages between subagents should be passed asynchronously through a message queue to avoid extra consumption caused by polling.
• Human review points: require human confirmation at critical nodes such as architecture decisions, external releases, and data deletion, rather than letting the model run to the end without oversight.

Budget guardrails can be configured as follows:

budget_guard:
  tier_default: high          # Default reasoning tier
  tier_escalate: xhigh        # Highest tier allowed after validation failure
  max_cost_per_task: 3.00     # Maximum cost per task (currency unit)
  warn_at: 0.8                # Warn when 80% of the limit is reached
  max_subagents: 32           # Maximum number of parallel subagents
  max_wall_clock_hours: 6     # Maximum wall-clock runtime

5. Implementation checklist

1. Build your own evaluation set: select 30 to 100 real tasks close to your business, with standard answers or automatic verifiers, and record pass rate, average tokens, and cost per task.
2. Start with the default tier: in most scenarios, the default tier can already match the top configuration of the previous generation. Only increase the tier after testing confirms a clear gain.
3. Make budget guardrails default configuration: per-task cost limits, runtime limits, and warning thresholds should be written into code, not enforced manually.
4. Monitor P95/P99 of per-task cost: averages hide long tails; a few out-of-control tasks may consume most of the budget.
5. Review periodically: after model version upgrades, rerun the evaluation set and redraw the applicable boundaries of each tier.

6. Conclusion

“Lower unit price” and “lower total spending” are different questions. Reasoning-oriented models trade more thinking tokens for stronger capability, and reasoning-token consumption is often underestimated. For developers, the real variable to manage is not unit price but the ratio of “task value ÷ reasoning cost.”

Concretely, treat reasoning tiers as an engineering variable alongside model selection, build tiered routing and failure-based escalation, set budget guardrails for long-running agent tasks, and use your own evaluation data to drive every configuration adjustment. On platforms such as Baidu AI Cloud, model services, container platforms, object storage, message queues, and log monitoring can be combined so that frontier model capabilities can deliver value while the cost of each call remains predictable and controllable.