How to Choose Reasoning Effort: An Engineering Approach to Avoid Token Waste

Many teams encounter the same problem after integrating large language models: why does the model respond quickly and cost-effectively to the same request in some cases, but produce lengthy internal reasoning, take much longer, and still fail to deliver a better result in others?

The root cause is often not a lack of model capability, but an insufficiently engineered approach to reasoning effort.

Reasoning effort, thinking depth, or reasoning level essentially represents the additional analysis budget allocated to the model by the caller. Higher levels generally allow the model to spend more computation and tokens on decomposing the problem, checking constraints, comparing candidate answers, and performing additional internal verification. This is useful for tasks with high uncertainty, high risk, or many constraints. However, it should not be treated as the default setting for every request.

Setting reasoning effort to the maximum for every task is like assigning a senior engineer to handle every log-format conversion, field explanation, and copy-editing request. It can work, but the resource allocation is inefficient. Mature practice is not about making the model “think as much as possible” on every call. It is about making the model think just enough for each call.

1. Why Tokens Get Consumed

Model usage costs are generally related to input tokens, output tokens, and the additional computational budget used during reasoning. From a user’s perspective, three types of consumption are especially noticeable.

1.1 Overly Large Input Contexts

Many applications send complete chat histories, entire business documents, raw logs, or large codebase fragments to the model in a single request. As a result, the model consumes a significant amount of context budget before it even begins addressing the actual question.

For example, a user may ask, “What caused this API error?” while the system attaches dozens of pages of previous conversations, every configuration file, and unrelated logs. Even with low reasoning effort, the overall cost will still be high.

Reasoning-level optimization must therefore be combined with context governance. Do not focus only on whether the model is “thinking too much.” Also check whether the system is supplying too much information.

1.2 Analysis Expansion at Higher Reasoning Levels

For simple tasks, high reasoning effort can lead to over-analysis. Examples include:

• Rewriting a sentence in a more formal style
• Extracting fields from a fixed format
• Converting structured data into a table
• Checking whether content contains a specific keyword
• Generating a notification from a predefined template

These tasks have clear inputs, objectives, and acceptance criteria. With high reasoning effort, the model may spend additional resources restating the task, discussing edge cases, comparing alternatives, or performing redundant checks. This increases latency and token usage while producing only a limited improvement in accuracy.

1.3 Missing Stop Conditions and Quality Boundaries

Another common problem is overly broad prompting, such as:

• “Analyze this comprehensively.”
• “Be as detailed as possible.”
• “Think deeply from every possible angle.”

Such instructions tend to expand the response and encourage longer reasoning paths.

A more controllable prompt should specify:

• The output format
• The maximum length
• The dimensions that must be covered
• The content that does not need to be covered
• The acceptance criteria
• What to do when information is insufficient

Instead of saying, “Analyze the possible causes of the incident,” use a more constrained instruction:

Based on the following logs, provide a troubleshooting conclusion:
1. List the three most likely causes in probability order.
2. Provide one verification method for each cause.
3. If the logs do not support a conclusion, explicitly identify the missing fields.
4. Keep the response under 500 words.
5. Do not reproduce the full logs.

The clearer the task boundaries, the less the model needs to infer the user’s intent through lengthy reasoning.

2. Reasoning Effort Is a Resource Strategy, Not an Intelligence Level

It is easy to interpret “high reasoning” as “more intelligent” and “low reasoning” as “less capable.” That interpretation is inaccurate.

The model’s underlying knowledge, language ability, and tool-use capabilities do not disappear when the reasoning level is reduced. The primary difference is usually how much additional analysis the model is allowed to allocate to the current problem.

This can be understood as different execution strategies in an engineering system:

• Low reasoning: Fast execution for tasks with clear rules and easily verifiable results
• Medium reasoning: A balance between quality and efficiency for most routine knowledge-work and development tasks
• High reasoning: Deeper analysis for complex constraints, long-range planning, and high-risk verification
• Adaptive reasoning: Start with a lower budget and increase it when uncertainty or quality problems are detected

The practical principle is therefore not “use the highest level by default,” but “start at a moderate level and upgrade based on the task.”

3. Task Classification: When to Use Low, Medium, or High Reasoning

In production systems, teams can classify tasks based on complexity and the cost of errors.

3.1 Low Reasoning: Deterministic or Easily Rule-Based Tasks

Low reasoning is suitable for:

• Information extraction and field structuring
• Classification, tagging, and topic recognition
• Format conversion, such as text to tables or tables to fixed-structure data
• Short translation, editing, and summarization
• Fixed-template generation
• Simple code explanations
• Organizing retrieved information when the answer is already known
• Initial screening for content moderation

These tasks typically have clear objectives, stable constraints, and a limited answer space. They can also be validated quickly through rules, format checks, or manual sampling.

For example, when extracting an order number, timestamp, issue type, and urgency from a support ticket, use low reasoning and require a structured output format. There is no need for a long problem analysis.

{
  "ticket_id": "string",
  "event_time": "string",
  "issue_type": "string",
  "priority": "low|medium|high",
  "confidence": 0.0
}

3.2 Medium Reasoning: The Default for Routine Development and Business Analysis

Medium reasoning is generally appropriate as the default for most interactive applications, including:

• Code review suggestions
• Routine defect diagnosis
• API design discussions
• Summarizing data-analysis findings
• Initial technical proposals
• Documentation writing and requirements clarification
• Multi-condition filtering and business-rule explanations
• Question answering with a limited amount of context

These tasks require the model to understand context, identify multiple constraints, and provide a structured response. However, they usually do not require an extremely long reasoning process.

For example, a developer may provide an exception log, related configuration, and an API call chain, then ask for a troubleshooting sequence. Medium reasoning can usually produce reliable diagnostic hypotheses, verification steps, and remediation suggestions while maintaining good response speed.

For internal enterprise assistants, customer-support assistance, developer Q&A, and retrieval-augmented knowledge-base applications, medium reasoning is also a sensible starting point.

3.3 High Reasoning: Complex, Ambiguous, and High-Cost-of-Error Tasks

High reasoning should be reserved for tasks that genuinely require it, such as:

• System architecture involving multiple modules
• Complex code refactoring and dependency analysis
• Multi-stage execution planning
• Resource scheduling with multiple constraints
• Detecting contradictions in long documents
• Complex mathematical, logical, or algorithmic problems that require step-by-step verification
• Argumentation and counterexample checking for high-risk decision support
• Security-risk analysis, permission-boundary review, and incident retrospectives

These tasks typically have several characteristics:

1. There is no single obvious answer.
2. Multiple objectives must be balanced, such as cost, performance, stability, and maintainability.
3. Relevant information is distributed across different sections or sources.
4. An incorrect conclusion can create significant downstream costs.
5. The user needs not only an answer but also traceable decision reasoning.

High reasoning does not guarantee high reliability. For factual tasks, incorrect source material, outdated knowledge bases, or missing external data cannot automatically be corrected by increasing the reasoning budget. High-risk scenarios should therefore combine high reasoning with retrieval, rule validation, human approval, and audit records.

4. Recommended Strategy: Start Fast, Then Go Deeper

Instead of sending every request directly to the highest reasoning level, a more efficient approach is to design a layered routing mechanism.

The basic process can be represented as follows:

Request received
  ↓
Task classification and risk assessment
  ↓
Initial processing with low or medium reasoning
  ↓
Quality checks
  ├─ Pass: return the result
  └─ Fail: add context or increase reasoning
                   ↓
              High-reasoning review
                   ↓
              Return the final result

This strategy allows most simple tasks to complete through a lower-cost path. Only genuinely complex requests enter the more expensive path.

Signals for Increasing Reasoning Effort

A system can use the following signals to decide whether to escalate:

• The model explicitly indicates insufficient information or low confidence.
• The output does not match the required structure.
• Required fields are missing.
• Rule-based validation fails.
• Multiple generations contradict one another.
• The user asks the model to recheck the conclusion.
• The task contains high-risk keywords.
• The input contains multiple conflicting constraints.
• The task requires reasoning across documents, modules, or time periods.

For example, a system could first use medium reasoning to generate a code-fix proposal, then run static checks, unit tests, or structural validation. If the checks pass, return the result directly. If they fail, submit the error information together with the original context to a higher reasoning level for a second attempt.

This is generally more efficient than starting every request at the maximum level.

5. Token Optimization Requires Context Governance

Reasoning effort is only one part of cost control. The quality of context design also has a major effect on token consumption.

5.1 Trim the Context

Do not send every available document to the model without filtering. First select, segment, and rank the information so that only material directly relevant to the current question remains.

For troubleshooting tasks, prioritize:

• Error logs from the most recent relevant time window
• Key nodes in the current call chain
• Related configuration changes
• Recent release records
• Knowledge-base entries matching the error code

Historical logs unrelated to the current issue, ineffective debugging output, and duplicate configuration should be removed whenever possible.

5.2 Replace Complete History with Summaries

In long-running conversations, compress earlier exchanges into a structured summary. For example:

Confirmed facts:
- The user is investigating an API timeout.
- The issue began within 30 minutes after a release.
- Database connectivity is normal.
- Network reachability has been ruled out.
Questions to verify:
- Has the downstream dependency become slower?
- Has the cache hit rate decreased?
- Is the connection pool exhausted?

This type of summary uses fewer tokens than the complete conversation history and helps the model focus on the key facts.

5.3 Reduce Repeated Instructions

System prompts, role definitions, formatting requirements, and safety rules should not be repeated across multiple sections. Repetition increases input costs and may introduce instruction conflicts.

Stable rules should be consolidated into a shared template. Business-specific variables can then be passed as concise fields, such as the task objective, user question, retrieved material, and output mode.

6. Use Evaluation Data to Select Reasoning Levels

The choice of reasoning level should not become a subjective debate between people who prefer speed and people who prefer depth. It should be treated as a measurable engineering decision.

Teams should build evaluation sets for representative tasks and track metrics such as:

Metric	Description
Task success rate	Whether the output meets the business objective
Format compliance rate	Whether the output follows the required structure or template
Factual accuracy	Whether key conclusions are correct
Average latency	Time from request submission to response
Average token consumption	Input, output, and reasoning-budget usage per task
Cost per successful task	Overall cost of completing one valid task
Manual rework rate	Percentage of outputs requiring additional user or reviewer edits

At a minimum, compare low, medium, and high reasoning across the evaluation set. In practice, medium reasoning often performs nearly as well as high reasoning for routine tasks while offering much better control over cost and latency. High reasoning tends to show a clear advantage only on a smaller set of complex examples.

Routing rules should ultimately be based on these measurements rather than on the assumption that a higher setting is always safer.

7. A Practical Decision Template

When integrating model capabilities, add a small set of decision fields to each request:

Task type: extraction / classification / Q&A / coding / planning / review
Error cost: low / medium / high
Context size: small / medium / large
Number of constraints: few / medium / many
Cross-document reasoning required: yes / no
External validation required: yes / no

Then choose the reasoning level according to rules such as:

If the task is extraction, classification, or format conversion:
    Use low reasoning
If the task is routine Q&A, document generation, code explanation, or general troubleshooting:
    Use medium reasoning
If the cost of error is high and the task has many constraints,
complex context, or cross-document dependencies:
    Use high reasoning
If complexity cannot be determined:
    Start with medium reasoning and escalate after failure

These rules are simple, but they can prevent large amounts of indiscriminate high-reasoning usage.

8. Conclusion: Reserve Reasoning Budget for Truly Difficult Problems

The reasoning ability of a large language model is a valuable resource. The more powerful the resource, the more carefully it should be managed. Applying excessive reasoning to simple tasks does not make a system more professional. It only increases cost and latency and makes the service harder to scale.

A better operating model is to:

• Use medium reasoning as the default.
• Lower the reasoning level for deterministic tasks.
• Increase it when tasks are complex or high risk.
• Reduce irrelevant input through context trimming.
• Use format checks, rule validation, and test results to trigger second-stage escalation.
• Continuously adjust routing policies based on real evaluation data.

The maturity of model usage is not demonstrated by allocating the maximum budget to every request. It is demonstrated by accurately determining how deeply the model needs to reason for each specific task.