Reasoning Effort in Two Passes: Run Low Effort First, Retry Failures at High Effort

1. A Counterintuitive Conclusion: Maxing Out Effort Does Not Maximize Returns

When using LLMs that support multiple reasoning-effort levels—typically low / medium / high / xhigh—many teams instinctively set every task to the highest level. The reasoning seems straightforward: higher reasoning effort means longer chains of thought and more self-checking, so results should be more reliable.

Published industry evaluation data contradicts that intuition. On the same set of mainstream coding evaluation tasks, a two-pass strategy—run everything at low effort first, then retry failed tasks at high effort—achieved a final pass rate of about 97% at an average cost of about $0.17 per task. Running every task once at high effort achieved a pass rate of about 95.3% at about $0.29 per task.

The conclusion is twofold: the two-pass strategy was not only about 41% cheaper, but also about 1.7 percentage points more accurate.

This article breaks down the full logic behind these results and answers four questions:

1. Why can “low first, high later” outperform “high effort throughout”?
2. What preconditions must hold for this strategy?
3. What does the tradeoff curve between effort, score, and cost look like?
4. How can the strategy be implemented in coding agents or batch task systems?

2. Why “Low First, High Later” Is More Accurate: The Math of Attempt Counts

It seems strange that running everything at high effort loses to a two-pass strategy. High effort clearly has a higher single-attempt accuracy. How can the overall pass rate be lower?

The key is that each task receives a different number of attempts.

2.1 The Implicit Structure of the Two-Pass Strategy

Formalize the strategy. Let the task set be T, the low-effort pass rate be p_l, and the high-effort pass rate on tasks that failed at low effort be p_h.

• High effort throughout: each task runs once, so the overall pass rate is approximately p_h.
• Two-pass: each task first runs at low effort. If it succeeds, it stops with probability p_l. If it fails, it runs again at high effort. The overall pass rate is approximately p_l + (1 − p_l) × p_h.

Because the two attempts are independent—the second attempt gets a new context window and a new reasoning path—this expression is strictly greater than the single high-effort pass rate. That extra retry opportunity is the source of the 1.7 percentage points.

2.2 Low Effort Is Not “Low Quality”—It Is “Low Investment”

The evaluation data also shows that low effort alone scores only about 8 points lower than high effort, while costing about one-third as much. This means that in a batch of real tasks with a natural difficulty distribution, most tasks can be solved at low effort—they do not need deep reasoning in the first place. Running everything at high effort really means paying the premium price of hard tasks for all easy tasks as well.

2.3 Breaking Down the Cost of the Two-Pass Strategy

Total cost = all tasks × low-effort unit price + failed-task share × high-effort unit price. In the evaluation data, the low-effort unit price was about one-third of the high-effort price. Assuming a low-effort failure rate of about 20%, total cost is approximately 1/3 + 20% × 1 = 0.53× the high-effort per-task cost. That is in the same order of magnitude as the measured 0.17 / 0.29 ≈ 0.59×. The lower the failure rate, the greater the savings.

3. The Precondition: Outcomes Must Be Automatically Checkable

This strategy has an indispensable precondition. The official guidance states it plainly: “If you can check outcomes automatically.”

The reason is direct: the core action of the two-pass strategy is to “identify which tasks failed and pull them out for a high-effort retry.” If failed tasks cannot be identified automatically by a machine, you can only review them manually one by one—and the saved model cost will be offset by manual labor. The strategy no longer makes economic sense.

3.1 Common Automatic Validation Signals

In practice, common and reliable automatic validation signals include:

• Test pass: the gold standard for coding tasks. Run unit tests or integration tests; passing means success. Evaluation benchmarks themselves work this way.
• Build or compile success: weaker than tests, but fully automatic and suitable as a first filter.
• API call scripts: for API tasks, write a script that verifies response structure, status codes, and boundary conditions.
• Structured output validation: for extraction or transformation tasks, validate whether the output conforms to a predefined JSON Schema, regular expression, or data-integrity constraint.
• Self-consistency voting: run the same task multiple times at low effort and take the majority result. Use “multiple consistent results” as a confidence signal. This suits open-ended tasks that cannot be directly verified, but cost increases and must be traded off.

3.2 What If the Outcome Cannot Be Validated?

If task outcomes inherently depend on human judgment—such as copy style or solution tradeoffs—consider tiered routing instead of failure retry: run everything at low effort, sample and manually review results, then move high-failure categories as a whole to high effort. Humans then handle sampling rather than full volume, so cost remains controllable.

4. The Effort–Score–Cost Curve: Diminishing Returns Set In Quickly

Using high effort as the baseline, the evaluation data compares four effort levels for one high-end model on coding evaluations:

Effort	Score vs. high	Cost vs. high
low	about 8 points lower	about 1/3
medium	about 2.5 points lower	about 70%
high	baseline	1×
xhigh	about 1.4 points higher	2.5×

If you plot this curve, it flattens as you go up:

• From low to medium: about 2× the money for about 5.5 points. The value is reasonable.
• From medium to high: about 1.4× the money for about 2.5 points. It starts to get expensive.
• From high to xhigh: 2.5× the money for only 1.4 points—a classic marginal-return cliff.

Engineering implication: xhigh should be treated as a last resort, not a default option. There are only two reasonable times to use it: first, as the retry pass for failed tasks in the two-pass strategy; second, when explicitly specified for known high-difficulty, long-chain tasks such as cross-module refactoring or complex algorithm design.

Another notable phenomenon is that some stronger newer models can have a higher unit price but a lower per-task cost—because inference efficiency improves, the total number of tokens required to complete a task drops substantially, and the increase in unit price is offset by the decrease in usage. When selecting models, per-task cost is more meaningful than per-token price.

5. The Recommended Three-Step Order When Effort Is Not Enough

When task difficulty exceeds the capability ceiling of the current model and effort level, the guidance provides a progressive handling order that also applies to general scenarios:

1. Raise the effort level first: increase reasoning effort on the same model. This is the cheapest way to “get smarter.” The model’s knowledge, tool-calling habits, and context-handling style remain unchanged, and prompts do not need adjustment.
2. Then switch to a stronger model: if maxing out effort is still not enough, switch to a stronger model tier or series and recalibrate effort. Stronger models often reach the results of a smaller model’s high tier at medium or even low effort.
3. Finally consider task decomposition: after single-call capability is maxed out, split the large task into subtasks that can be verified separately, run each subtask back at low effort, and use an orchestration layer to ensure overall correctness.

The essence of this order is: first squeeze the value out of the current configuration, then upgrade the configuration, and finally change the problem itself. Doing it in reverse—starting with the strongest model plus the highest effort—is the most expensive path.

6. Implementing It in Coding Agents: Concrete Configuration Ideas

In common coding CLI and agent tools, implementing the two-pass strategy usually involves the following configuration points. Names vary slightly across tools, but the idea is the same.

6.1 Effort Configuration

Most coding tools allow the model and reasoning effort to be set in configuration files or environment variables, for example:

# Full batch phase: low effort + primary model
export MODEL="primary-model-ID"
export EFFORT="low"
# Retry phase: failed tasks only
export EFFORT="high"

Some tools also support specifying effort per task in request parameters. This is the foundational capability for allocating effort by task rather than by project.

6.2 Batch Processing and Failure Recovery Script Skeleton

results = []
for task in tasks:
    r = run_model(task, effort="low")
    if not auto_check(r):          # tests/build/script validation
        r_retry = run_model(task, effort="high")
        r = merge(r, r_retry)
    results.append(r)

For production use, also add:

• Failure classification statistics: record the category distribution of low-effort failures and continuously refine which categories are worth sending directly to high effort, skipping a wasted low-effort pass.
• Budget circuit breaker: set a budget cap for the retry phase to prevent cost overruns when the failure rate is abnormal.
• Result caching: persist successful task results and skip them directly during the retry phase.

6.3 One-Line Principle

Effort should be allocated by task, not by project. Treat reasoning effort as a compute resource that can be allocated on demand, and use automatic validation signals to drive that allocation. This is one of the most overlooked—and most effective—ways to reduce cost in current LLM engineering. In this evaluation, it simultaneously delivered 41% lower cost and higher accuracy—a win-win that is rare in engineering optimization.