Choosing LLM Reasoning Effort Tiers: Real Cost and Cost-Performance Evaluation in Practice

1. Small Version Update, Large Capability Jump: Read the Evaluation First, Then Choose the Model

After a new-generation large model completes a minor version iteration, the capability jump is often larger than the version number suggests. For example, a composite intelligence index may move from the 49 range to the 65 range, while a code-specific benchmark may reach a new best result. Such a jump is closer to a generational shift than to a routine parameter adjustment.

The reason is that capability growth in modern models increasingly comes from coordinated optimization across training-data mixtures, post-training alignment strategies, and inference-stage compute allocation. These improvements are not necessarily reflected in the version number.

For developers, the real risk is not “failing to understand leaderboards.” The bigger risk is using leaderboard scores to directly infer production value. When making decisions about model selection, reasoning effort tiers, and real cost, it is advisable to follow three principles when reading evaluation results:

• Separate composite scores from specialized capabilities. A composite intelligence index reflects breadth, while benchmarks for coding, math, and multi-step reasoning reflect depth. Production selection should prioritize the specialized scores closest to the target workload and use composite scores as a secondary reference.
• Watch for benchmark contamination. Problems and solutions from public benchmarks may have entered the training corpus, causing scores to be inflated. A reliable approach is to validate results with a private evaluation set built around real business scenarios.
• Pay attention to the marginal meaning of scores. Moving from a passing range to an excellent range can mean that tasks previously requiring manual work become automatable, which increases value nonlinearly. By contrast, gaining another 1 to 2 points within an already high range may provide limited benefit for many businesses.

2. Reasoning Effort Tiers: A Multi-Gear Transmission for the Same Model

Most mainstream models now provide tiered reasoning effort controls, such as low, medium, high, and maximum. In essence, the caller exchanges more reasoning tokens for stronger answer quality.

This leads to a key conclusion: the same model at different reasoning effort tiers can be treated as multiple products with different capabilities. Choosing a reasoning tier is therefore not a mystical decision; it can be engineered.

Tier selection and risk control

Task characteristics	Recommended tier	Main risk	Fallback / safety mechanism	Acceptance criteria
Classification, extraction, rewriting, formatting	Minimal tier or disable extended reasoning	Wasted reasoning overhead on tasks that can be decided in a single step	Disable extended reasoning	Exact match or field completeness meets target
Routine Q&A, single-file edits, SQL generation	Default tier	Quality may be insufficient on complex boundary cases	Retry failed samples at a higher tier	Test-case or unit-test pass rate meets target
Cross-file refactoring, complex bug localization, solution design	Higher tier	Errors accumulate across multiple assumptions	When the step limit is triggered, fall back and roll back to the last stable version	Build, unit tests, and static checks pass
Long-running autonomous tasks, difficult algorithm problems	Highest tier	Nonlinear cost growth and uncontrolled long-tail behavior	Circuit-break and terminate when a stage gate fails	All staged acceptance gates pass

How to choose: common mistakes, boundaries, and a pre-launch checklist

There are three common mistakes:

1. Setting the highest tier as the global default, which causes high-frequency simple tasks to be dragged down by excessive reasoning-token spending.
2. Treating the default tier as the minimum tier, or vice versa, resulting in an imbalance between quality and cost.
3. Choosing a tier based only on the highest leaderboard score while ignoring the difficulty distribution of your own tasks.

There is also an important boundary: reasoning tiers change answer quality and token consumption, but they do not change the model’s knowledge boundary or context window. Do not expect “raising the tier” to solve problems the model fundamentally cannot solve.

Pre-launch checklist:

• Do you already have a private evaluation set closely aligned with your business to verify tier benefits?
• Are tiers assigned by task difficulty rather than applied uniformly?
• Do you have token-based and spend-based circuit breakers?
• Are you using cost per successful task, rather than unit price, as the selection criterion?

Reasoning tiers can also be dynamically routed by task difficulty:

# Example: route the reasoning effort tier by task complexity.
# The tiers here are generic abstractions, not tied to a specific API.
def route_tier(task):
    # task should include:
    #   files_touched: number of files changed
    #   est_steps: estimated number of steps
    #   has_spec: whether a specification exists
    #   requires_cross_module_reasoning: whether cross-module reasoning is needed
    if task.files_touched <= 1 and task.est_steps <= 3:
        return "minimal"        # Low-risk, single-step task
    if task.has_spec and task.files_touched <= 5:
        return "default"        # Routine development with a specification
    if task.requires_cross_module_reasoning:
        return "high"           # Cross-module reasoning
    return "max"                # Long-running autonomous task; budget circuit breaker required

3. Real Cost Model: A Lower Unit Price Does Not Always Mean a Lower Total Cost

One of the most misleading statements is: “The API price has been reduced by 20%, so it is cheaper now.” In practice, the cost of a single task is determined by input tokens, output tokens, and reasoning tokens together:

Cost per task = (input tokens × input price + (output tokens + reasoning tokens) × output price) / 1,000,000

The key point is that raising the reasoning effort tier increases both output-side and reasoning-side token consumption. The following is an illustrative measurement used to explain the method. The full measurement conditions are noted below.

Measurement conditions:

• Models: a new-generation minor version, denoted M-new, and the previous generation, denoted M-old.
• Evaluation set: a private mixed set of coding and multi-step reasoning tasks built in-house, covering real online workloads and stratified into simple, medium, and difficult levels.
• Sample size: no fewer than several hundred items per level, for example N = 300, using the mean cost per task.
• Price version: settlement prices for M-new, with input and output unit prices reduced by about 20% compared with M-old.
• Measurement period: the end of a quarter, for illustration only.

Under the measurement scope described in this article, on the simple-to-medium subset, the new-generation model at the default or higher tier scored 62 with a per-task cost of about $9.08. On the full set including difficult tasks, the highest tier scored 65 with a per-task cost of about $26.50. In other words, the default tier can already achieve results close to the maximum tier at roughly one-third of the cost.

The following table compares token consumption and cost between the default tier and the maximum tier on the difficult/long-task subset, using the same new-version settlement price:

Tier	Input tokens	Output tokens	Reasoning tokens	Output-side total	Cost per task
Default tier	~270K	~90K	~28K	~118K	≈ $12.99
Highest tier	~277K	~180K	~94K	~274K	≈ $26.50

Moving from the default tier to the highest tier, output-side tokens increased from about 118K to 274K, an increase of about 132%. Reasoning tokens grew the most sharply, from 28K to 94K, while input tokens remained roughly flat, at about 270K to 277K.

This is where an apparent contradiction needs clarification. “The unit price has been reduced by 20%” refers to the change in per-token price from the old version to the new version, which is a price dimension. “The per-task cost rises from about $12.99 to about $26.50, nearly doubling” refers to the change in total tokens required to complete the same task from the default tier to the highest tier, which is a usage dimension. These are different dimensions and do not conflict.

The conclusion is that even if the new version has a lower unit price, once the reasoning tier is pushed to the maximum, the multiplied growth in tokens can outweigh the unit-price benefit. The table above is an illustrative measurement result; exact values should be based on the actual billing scope.

To put this into practice, it is useful to build estimation and accounting capabilities:

def estimate_task_cost(in_tok, out_tok, think_tok, price_in, price_out):
    """Estimate cost using a per-task scope.
    in_tok:     Number of input tokens
    out_tok:    Number of output tokens
    think_tok:  Number of reasoning/thinking tokens, usually billed at the output-side price
    price_in:   Input-side unit price per million tokens, based on the actual settlement price
    price_out:  Output-side unit price per million tokens, based on the actual settlement price
    Prerequisites:
    - Token counts must be the actual measured values for the task.
    - Unit prices must use the settlement scope rather than a nominal discounted price.
    """
    return (in_tok * price_in + (out_tok + think_tok) * price_out) / 1_000_000
def compare_tiers(observations):
    # observations: [
    #     (tier, in_tok, out_tok, think_tok, price_in, price_out, score, pass_rate)
    # ]
    # score: score for the tier
    # pass_rate: pass rate in batch evaluation, ranging from 0 to 1, as defined in Section 4
    rows = []
    for o in observations:
        cost = estimate_task_cost(*o[1:6])
        rows.append({
            "tier": o[0],
            "cost": round(cost, 4),
            "score": o[6],
            "cost_per_score": round(cost / o[6], 6) if o[6] else None,      # Cost per score = cost / score
            "cost_per_success": round(cost / o[7], 4) if o[7] else None,    # Cost per successful task = cost / pass rate
        })
    return rows

Recommended cost-governance practices:

1. Reduce context bloat. Every part of a long context is billed repeatedly in each reasoning turn. Use retrieval-based recall instead of stuffing everything into the context, trim conversation history on demand, and use prompt caching for stable prefixes. In scenarios with repeated long-context hits, prompt caching may significantly reduce duplicate billing, but the actual reduction depends on cache hit rate and billing rules. It should be evaluated through real measurements rather than assumed as a fixed multiplier.
2. Attribute failed tasks separately. Retries, loops, and repeatedly reading the same file are major sources of cost runaway in long-horizon tasks. Token consumption distribution should be recorded by task, not only by average.
3. Set budget circuit breakers. Define token and spend limits per session, per task, and per tenant. When limits are exceeded, downgrade the reasoning tier or terminate the task to prevent a few abnormal tasks from draining the overall budget.
4. Distinguish P50 cost from P99 cost. Averages can hide the long tail, and long-tail tasks often account for most of the spending.

4. Make Cost-Performance Quantifiable

To make selection conclusions reproducible, build an evaluation system around three metrics:

• Cost per score = cost per task ÷ score. This measures how much it costs to gain one point of capability and is useful for horizontally comparing different models and tiers. It matches the cost_per_score = cost / score calculation in the code above, and lower is better. Example: if the cost per task is $9.08 and the score is 62, then the cost per score is approximately $0.146 per point.
• Cost per successful task = cost per task ÷ pass rate. This measures the average cost of successfully completing a task and is the closest metric to real business value. Pass rate is defined as the number of tasks judged successful divided by the total number of tasks, with a value between 0 and 1. It refers to the pass rate in batch evaluation, not success or failure in a single run; the two should not be confused. Example: if the cost per task is $9.08 and the pass rate is 62%, then the cost per successful task is approximately $14.65 per successful task. A model with a low pass rate may still have a higher cost per successful task even if its unit price is cheap.
• Capability-cost frontier. Plot each tier as a point on a score-cost plane and take the Pareto frontier: the point with the highest score for a given cost, or the lowest cost for a given score. Configurations dominated by the frontier, meaning both more expensive and worse, should be eliminated.

A typical conclusion looks like this: the highest tier sits in the upper-right region of the frontier, representing the strongest capability but also the highest cost; the default tier sits near the elbow, representing the best cost-performance balance; mid-tier configurations may achieve code capabilities comparable to the previous generation’s highest tier while costing about half as much. This kind of “elbow tier” is usually suitable as the production default, while the highest tier is reserved as an upgrade path for difficult tasks.

Key points for building a private evaluation set:

• Sample real tasks from online logs and cover simple, medium, and difficult levels, with no fewer than several hundred items per level.
• Record human-acceptable answers as baselines.
• For coding tasks, prioritize test-driven methods that can automatically judge pass rates, avoiding noise from subjective scoring.

Only a private evaluation set with a consistent measurement scope can make the three metrics above truly reproducible.

5. Cost Characteristics of Coding and Long-Horizon Agent Scenarios

Coding is currently the area where capability improvement is most obvious and where cost tension is most visible. In the illustrative measurement, the higher tier’s code benchmark score was roughly on par with the previous generation’s highest tier, while the per-task cost was about half. Enabling the highest tier could further improve the best result, but cost rose sharply.

This shows that the premium for the highest tier in coding scenarios is optional, not mandatory. The most cost-effective strategy is to first run with the default tier and upgrade only the failed samples to a higher tier.

A more extreme case is long-horizon autonomous tasks. The industry has already seen demonstrations in which a model starts from an empty directory, coordinates thousands of subagents, runs continuously for multiple days, and autonomously completes design, coding, testing, and review before delivering a runnable system. Such tasks push the cost structure into a new dimension:

• Subagent fan-out multiplies token consumption. The main agent and each subagent maintain their own contexts.
• Shared state becomes a bottleneck. File systems, task queues, and review conclusions require consistent read/write protocols.
• Errors accumulate across days. Without staged acceptance, rework costs in the later stages can be much higher than review costs in the earlier stages.

Recommended engineering constraints:

• Set limits on the number and concurrency of subagents.
• Use a persistent task queue rather than passing context through memory.
• Define automatically executable acceptance gates for each stage, such as compilation, unit tests, static checks, and integration tests.
• Version artifacts and support rollback to any stage.

On the platform side, general cloud capabilities can support these workloads: a unified model service can manage models from different tiers and vendors; a container platform can provide elastic compute for long-running tasks; object storage can preserve stage artifacts and evaluation data; and log services and monitoring alerts can track token consumption, failure rates, and cost attribution. These are common capabilities of general cloud platforms. Specific products, integration methods, and billing scopes should follow the official documentation and settlement rules of the selected platform.

6. Conclusion

To evaluate the real capability and cost-performance of a new-generation model, teams should move away from the intuitive approach of “look at the score and look at the unit price” and adopt the engineering perspective of “look at cost per successful task.”

Four core conclusions:

1. Capability jumps can be much larger than the version number suggests, but they must be validated with a private evaluation set.
2. Reasoning effort tiers make the same model appear as multiple capability levels. The default tier is usually sufficient for most production tasks.
3. A lower unit price can be offset by multiplied growth in reasoning tokens. Total cost must be calculated at the level of a complete task.
4. Coding and long-horizon agent scenarios require additional fan-out control, staged acceptance, and budget circuit breakers.

For ordinary developers, the practical judgment is that running the highest tier by default is not suitable for daily high-frequency calls. However, by combining a default-tier-first approach, upgrading only difficult tasks, optimizing context and caching, and setting budget circuit breakers, it is possible to achieve results close to the highest tier within an acceptable cost range. Model selection should not be a one-time decision. It should become a sustainable evaluation and cost-observation loop that can be recalculated over time.