Tiered Retry for Code Repair Tasks: Run Low Effort First, Escalate Only Failures to High Effort

3 hr ago0 views

This article explains why running every coding task at a high reasoning tier is usually the wrong default, and shows how a tiered strategy — execute everything at low effort first, verify automatically, then re-run only the failures at a higher effort with structured failure context — can improve pass rate while cutting per-task cost. It covers the conditions the strategy depends on, a cost model for estimating expected pass rate and spend, tier design and risk classification, failure-detection criteria, a reusable scheduling implementation, policy configuration, observability metrics, the cases where the strategy does not fit, and a phased rollout path from pilot to production.

1. The Starting Point: Why “Always High Tier” Is Not Always Optimal

In agentic coding tasks, automated repair, code generation, and engineering Q&A, many teams set a high reasoning effort level by default, hoping to get a stable result in a single pass. The motivation is direct: high-tier models generally have stronger reasoning, so routing every task to the high tier seems like the best way to minimize rework.

In real engineering evaluations, this approach tends to expose three problems:

Cost is significantly amplified: a large share of tasks are simple and do not require heavy reasoning, yet they are billed as expensive work.
Marginal returns diminish: for simple defects, formatting completions, and single-file edits, moving up a reasoning tier buys very little additional quality.
It does not necessarily raise the pass rate: over-reasoning can lead the model into long deliberation, repeated self-confirmation, and scope creep in its changes, and on simple tasks it can even produce results that drift away from the goal.

A more sensible strategy, therefore, is not “always high tier” but tiered scheduling: cover the bulk of the work at the low tier first, then escalate only the tasks that fail verification to a higher tier and re-run them.

One public code repair evaluation contains a representative data set. The same batch of tasks was first run entirely at the low tier, with only failing items re-run at the high tier. The pass rate was about 97%, at roughly $0.17 per task. When all tasks were run directly at the high tier, the pass rate was about 95.3% at roughly $0.29 per task. Running the low tier alone typically scores only about 8 points below the high tier, at roughly one third of the cost.

This points to a key conclusion: the low tier is not merely a quality downgrade — it works as a task filter; and the high tier is not a universal insurance policy — it should be used for targeted re-runs.

2. Core Strategy: Cover with the Low Tier, Then Re-Run Failures at the High Tier

The whole process can be abstracted into a tiered execution chain:

Run all tasks at the low reasoning tier first.
Use an automated acceptance mechanism to decide whether each task passes.
Finish tasks that pass immediately.
Escalate failed tasks to a higher tier, carrying the failure context with them.
If a task still fails, decide — based on budget and risk — whether to escalate further, hand it to a human, or mark it as failed.

What makes this strategy work is not the assumption that “the low tier is always enough,” but rather:

Most tasks can be solved reliably at the low tier.
Low-tier failures can be identified accurately.
High-tier re-runs can use the failure information to improve the success rate.
Re-run cost is lower than the cost of running everything at the high tier.
The task system provides a repeatable, rollback-capable, auditable execution environment.

3. Prerequisites for This Data to Hold

“Low tier first, high-tier re-run on failure” is not inherently effective; it depends on several engineering prerequisites. Without them, the strategy degrades into a flow with higher latency and more complicated cost accounting.

3.1 Objective acceptance criteria are required

Code tasks cannot rely on the model’s self-assessment of success. At least one or more of the following automated checks are needed:

Unit tests pass.
Integration tests pass.
The code compiles.
Static checks pass.
Format checks pass.
Dependency resolution succeeds.
Regression tests on key paths pass.
Security scanning raises no high-risk findings.

Without objective acceptance criteria, low-tier failures cannot be identified and high-tier re-runs have no clear target, so the strategy loses its meaning.

3.2 The low tier must cover most tasks

If the task set is inherently highly complex and the low-tier pass rate is very low, “low tier first, high tier later” turns into “almost every task needs a re-run.” Total latency then rises noticeably and the cost savings shrink.

The ideal situation is that the low tier handles most simple, repetitive, localized tasks, and only a small number of complex tasks need escalation.

3.3 High-tier re-runs must be able to use the failure context

A re-run should not be a fresh random generation; it should be an informed repair. A re-run should take as input:

The original task description.
The code or patch from the failed first attempt.
Failure logs.
Test failure messages.
Compile errors.
Directions that have already been tried.
Parts that must not be modified.
Additional constraints.

If a re-run simply resubmits the original task without using the failure information, the improvement will be very limited.

3.4 Tasks must be repeatable

For code repair, bulk migration, automated test repair, and similar tasks, the execution environment should be as far as possible:

Repeatable.
Rollback-capable.
Isolated.
Traceable.
Non-destructive to production.
Free of uncontrolled side effects.

If a task has strong side effects — for example, modifying a database directly, sending messages, or triggering a production deployment — then approval gates, sandboxes, canary releases, and rollback mechanisms must be added.

3.5 Cost must be quantifiable

The benefit of a tiered strategy has to rest on an observable cost model. At minimum, a team should track:

Model calls per task.
Average cost per task.
Cost share by tier.
Re-run rate.
Re-run success rate.
End-to-end latency.
Human takeover rate.
Distribution of failure causes.

Only when cost is quantifiable can a strategy move from “looks good” to “sustainable in operation.”

4. Cost-Benefit Analysis: Why Low Tier plus Re-Run Can Beat Always High Tier

Assume a task set of ( N ) tasks, a low-tier pass rate of ( p_l ), a high-tier pass rate of ( p_h ), and a success rate of ( r ) when failed tasks are re-run at the high tier. Let ( c_l ) be the cost of one low-tier run and ( c_h ) the cost of one high-tier run.

The expected pass rate of “low tier first, high-tier re-run on failure” is:

[
P = p_l + (1 - p_l) \times r
]

The expected cost is:

[
C = c_l + (1 - p_l) \times c_h
]

If ( c_h ) is roughly three times ( c_l ) and the low-tier pass rate is high, the average cost of the re-run strategy is clearly below the cost of always running at the high tier.

For example, assume a low-tier pass rate of 85% and a high-tier re-run success rate of 80% among failures. The final pass rate is:

[
0.85 + 0.15 \times 0.8 = 0.97
]

That is 97%.

If the high tier costs three times the low tier, the average cost is:

[
1 + 0.15 \times 3 = 1.45
]

That is, 1.45 times the low-tier cost, versus 3 times for always running high. In this case the re-run strategy both raises the pass rate and lowers cost.

This kind of data feels counterintuitive because many people assume “high tier = higher pass rate.” But an engineering system is not a single point of model capability; the outcome is determined jointly by model capability, verification mechanisms, retry strategy, and task routing.

5. How to Design Reasoning Tiers

In a real system, reasoning tiers can be abstracted into several levels. Different levels suit different work, and the top tier should not be applied to every task.

5.1 Tier design principles

Tier	Suitable scenarios	Typical goal	Control focus
Lightweight	Formatting, naming, comments, simple completion, localized fixes	Fast completion, low-cost coverage	Strict acceptance to avoid misjudgment
Standard	Single-file logic repair, handling common errors	Balance cost and quality	Limit the scope of changes
Enhanced	Cross-file changes, dependency analysis, test-driven repair	Handle complex context	Add regression checks
Expert	Complex refactoring, architectural changes, deep defect repair	Raise the pass rate on hard cases	Strong constraints, strong auditing
Deep	High-risk changes, critical-path tasks, final arbitration	Highest reliability	Limit budget and timeouts

5.2 Do not look at difficulty alone — also look at risk

Task routing cannot consider technical difficulty in isolation; it must also consider the cost of failure:

For internal script repair, cost optimization can take priority.
For base library changes, stronger regression coverage is required.
For production environment changes, approval and canary release are required.
For security-related code, dedicated review is required.
For data-related tasks, permission and privacy controls are required.

Tier selection is therefore not a single parameter. It should combine task complexity, business risk, budget ceiling, and acceptance strictness.

6. Failure Determination: Automated Verification Matters More Than “The Model Says It Succeeded”

In a tiered retry mechanism, failure determination is the most critical link. If the criteria are too loose, wrong results are accepted as successes. If they are too strict, simple tasks are re-run needlessly and effort is wasted.

6.1 Recommended failure determination dimensions

Syntax-level failure
- Compile errors.
- Type errors.
- Import errors.
- Syntax errors.
Functional-level failure
- Unit test failures.
- Integration test failures.
- Unexpected return values from key interfaces.
Convention-level failure
- Formatting does not conform to the standard.
- Lint rules are triggered.
- Naming violates constraints.
Scope-level failure
- Files that should not have been modified were modified.
- The patch scope is too large.
- Extra dependencies are introduced.
- Unrelated code is changed.
Runtime-level failure
- Timeouts.
- Resource exhaustion.
- Environment initialization failure.
- Unavailable dependencies.

6.2 Model self-assessment can only be auxiliary

A model can give a preliminary judgment on its own result, but it should not be the final acceptance basis. A more reliable arrangement is:

Automated tests first.
Static checks as a fallback.
Human review for critical tasks.
Multi-person approval for high-risk tasks.

For code tasks, “compiles, passes tests, passes regression” usually matters more than “the model sounds plausible.”

7. Re-Run Is Not Simple Repetition: Carry Structured Failure Context

Many re-runs underperform because not enough effective failure information is provided at re-run time. A high-quality re-run should include structured context.

A typical re-run input can look like this:

Task goal:
Fix the intermittent timeout defect in the user login endpoint under high concurrency.
First attempt result:
Modified the cache read/write logic, but unit tests still fail.
Failure reason:
Test LoginServiceTest.timeout did not complete within 30 seconds.
Constraints:
1. Do not change public interface signatures.
2. Do not introduce new external dependencies.
3. Modify no more than 3 files.
4. Keep the existing log format.
Suggested directions:
Check lock scope, async thread pool configuration, and connection timeout settings;
prefer a localized fix.

This input is more valuable than simply repeating the original task, because it tells the model:

Where the last attempt went wrong.
Which directions have already been ruled out.
Which constraints must be respected.
What the verification standard is.

8. Engineering Implementation: A Reusable Tiered Scheduling Flow

Below is an illustrative flow that expresses the core logic of tiered scheduling. In a real system it can be replaced by an internal model gateway, a task orchestrator, or an agent execution framework.

def solve_task(task):
    # Run at the low tier by default
    result = call_model(
        task=task,
        effort="low",
        timeout=task.timeout,
        budget=task.budget,
    )
    if verify(result, task):
        return result
    # Record the failed attempt for the later re-run
    history = [result]
    # Decide the escalation path based on the task budget
    escalation_levels = get_escalation_levels(task)
    for level in escalation_levels:
        if task.budget_exceeded():
            break
        retry_prompt = build_retry_prompt(
            original_task=task,
            previous_attempts=history,
            constraints=task.constraints,
            failure_reason=extract_failure_reason(history[-1]),
        )
        result = call_model(
            task=retry_prompt,
            effort=level,
            timeout=task.timeout,
            budget=task.budget,
        )
        if verify(result, task):
            return result
        history.append(result)
    return mark_as_failed(task)

The key design points here are:

Low tier by default: complete most tasks at low cost first.
Record failures: save the failure context so the same mistakes are not repeated.
Escalate level by level: decide the escalation path from task risk and budget.
Automatic acceptance: only a result that passes verification ends the flow.
Budget control: prevent endless re-run loops.
Failure closure: tasks that still fail should be handed to a human or a higher-level process.

9. Policy Configuration Example: Making the Rules Explicit

To avoid internal disagreement about “when to escalate and when to stop,” the policy can be expressed as configuration.

task_policy:
  default_effort: low
  escalation:
    - standard
    - enhanced
    - expert
  max_retries: 3
  verification:
    unit_test: true
    integration_test: false
    lint: true
    compile: true
    security_scan: false
    timeout_seconds: 600
  budget:
    max_cost_per_task: 0.20
    max_duration_seconds: 900
  safety:
    require_approval_for:
      - production_config
      - database_migration
      - security_code
      - public_api
  observability:
    log_effort_level: true
    log_retry_reason: true
    log_cost_estimate: true

This kind of configuration has several benefits:

The policy is auditable.
Cost is predictable.
Risk is controllable.
Different task classes can be configured differently.
Later optimization has a data basis to build on.

10. Why Low Tier plus Re-Run Sometimes Achieves a Higher Pass Rate

A common misconception is that because the low tier is weaker, the final pass rate must be worse than running everything at the high tier. But in an engineering system the result depends not only on single-pass inference capability, but also on error discovery, strategy switching, and the re-run mechanism.

10.1 The low tier tries fast and reduces wasted computation

For simple tasks, the low tier is often sufficient. It can quickly generate a candidate fix and move straight into verification. This avoids very long reasoning on simple problems and reduces unnecessary token consumption.

10.2 The high tier is reserved for hard cases, concentrating resources

When the low tier fails, the task is marked as a hard case. Using a high-tier model at that point concentrates stronger reasoning where genuinely complex analysis is needed, for example:

Cross-file dependencies.
Concurrency issues.
Hidden state pollution.
Complex type errors.
Deep call chains.
Environment-dependent defects.

10.3 A re-run creates a second chance

A failure re-run is not repeated computation; it is an informed second attempt. It effectively adds a “diagnose then repair” layer on top of the original model. Compared with a single high-tier call, the re-run has more explicit context and is therefore more likely to succeed.

10.4 Always-high-tier can cause over-reasoning

On some tasks, a high-tier model may generate excessive explanation, modify too many files, introduce unnecessary abstractions, or even deviate from the minimal-fix principle in pursuit of being “more thorough.” Engineering code repair generally demands:

Minimal change.
Verifiability.
Low regression risk.
Easy review.
No breakage of existing behavior.

Over-reasoning can therefore sometimes lower the pass rate instead of raising it.

11. Observability Metrics: How to Tell Whether the Strategy Actually Works

For this strategy to run over the long term, a metrics feedback loop must be in place. At minimum, track the following.

11.1 Quality metrics

Low-tier first-pass rate.
Final pass rate after re-runs.
Pass rate by tier.
Pass rate by task type.
Regression error rate.
Human review pass rate.

11.2 Cost metrics

Average cost per task.
P95 cost per task.
Call share by tier.
Cost share of re-runs.
Share of tasks hitting the budget ceiling.
Share of ineffective re-runs.

11.3 Latency metrics

Average end-to-end duration.
Model inference time.
Verification time.
Re-run time.
Queue time.
Timeout rate.

11.4 Stability metrics

Re-run failure rate.
Re-run conflict rate.
Environment error rate.
Tool invocation failure rate.
Patch conflict rate.
Human takeover rate.

If these metrics are wired into general cloud capabilities such as log services, monitoring and alerting, and object storage, a fairly complete operational loop can be formed. For example, in a cloud environment such as Baidu AI Cloud, task logs, metric data, model invocation records, and result artifacts can be archived together for later analysis, auditing, and cost governance.

12. Risks and Boundaries: When “Low Tier First, High Tier Later” Does Not Fit

Tiered retry works for most code repair tasks, but it is not suitable for every scenario.

12.1 When acceptance criteria are unclear

If a task has no clear acceptance criterion — an open-ended goal such as “improve the code structure” — it is hard to judge quality differences between tiers, and re-runs may oscillate back and forth.

12.2 When tasks are highly complex and the low-tier pass rate is very low

If the low tier can barely complete any task, the re-run rate becomes too high and the strategy’s benefit shrinks. The right move is to improve task decomposition, context preparation, and verification first, rather than simply raising the model tier.

12.3 When tasks have strong side effects

If model output directly affects live systems, the following must be added:

Sandboxing.
Approval.
Canary release.
Rollback.
Auditing.
Permission isolation.

12.4 When latency is extremely sensitive

Re-runs add end-to-end latency. If the system requires second-level responses, options include:

Fast return from the low tier.
Asynchronous repair at the high tier.
Follow-up human confirmation.
Caching historical repair results.
Escalating high-risk tasks earlier.

13. Team Rollout Path: From Pilot to Scale

To adopt this strategy, it is advisable to proceed step by step.

Step 1: Build an internal evaluation set

Collect historical code defects, task tickets, repair records, and test cases to build a repeatable evaluation set. The task set should cover:

Simple syntax repair.
Single-file logic repair.
Cross-file repair.
Test completion.
Dependency repair.
Complex defect localization.

Step 2: Build acceptance scripts for each task class

Without acceptance scripts, there is no way to determine whether the low tier failed. Prioritize connecting:

Unit tests.
Compile checks.
Format checks.
Static scans.
Regression tests.

Step 3: Measure the low-tier baseline

Start without escalation: run only the low tier and observe:

Low-tier pass rate.
Low-tier cost.
Low-tier latency.
Distribution of failure causes.

This step determines whether the strategy has a real cost advantage.

Step 4: Design the re-run rules

Define:

Which failures need a re-run.
Which failures need a human takeover.
Which failures need an environment fix.
Maximum number of re-runs.
Per-attempt budget.
Total budget.
Timeout threshold.

Step 5: Roll out to a small traffic slice

Enable the tiered strategy first on low-risk tasks, such as internal scripts, test repair, and documentation code snippet completion. Observe changes in pass rate, cost, and latency.

Step 6: Tune continuously

Adjust based on metric changes:

Whether the low tier is too weak.
Whether the high tier is too strong.
Whether the re-run context is sufficient.
Whether the acceptance rules are too strict or too loose.
Whether budgets need to be tiered.

14. Key Conclusions

“Always high tier” looks safe, but in an engineering system it is not necessarily the most efficient choice. The better approach is to promote model reasoning effort from a fixed configuration to a scheduling strategy:

Cover with the low tier: solve most tasks at low cost.
Filter with automatic acceptance: use tests, compilation, and checks to decide success or failure.
Escalate only failures: use the high tier only for tasks that are genuinely hard.
Re-run with failure context: make the re-run a post-diagnosis repair rather than a simple repeat.
Close the metrics loop: continuously monitor cost, pass rate, latency, and risk.
Classify by risk: give different task classes different escalation paths instead of applying one rule to all.

In practice, the value of this strategy is not just saving money — it is matching inference resources to task difficulty. The low tier provides breadth and the high tier provides depth; automated verification filters, and re-runs recover; budget control defines the safety boundary. The result is not a single-model-call scheme but a continuously optimizable, observable, and governable scheduling system for intelligent code tasks.