How Thinking Levels Affect LLM Coding Cost and Quality: A Hands-On Test

1. Background: How Thinking Levels Work and the Trade-Off Behind Them

Most mainstream large models now expose a “thinking level” mechanism: at call time, you can choose how much internal reasoning the model performs before producing its final answer. Common tiers are Low / High / Max (or Low / Medium / High). The higher the tier, the longer the chain of thought generated before the actual output, and the more rounds of solution design, self-questioning, detail extrapolation, and backtracking the model goes through.

For developers, the choice of tier directly affects three things:

• Token consumption and cost: the reasoning process itself is billable output;
• Response latency: the longer the model thinks, the later the first usable deliverable appears;
• Output quality: especially solution correctness and robustness on complex tasks.

A common misconception is that “more thinking is always better.” Thinking has a cost, and excessively long reasoning can lead to over-engineering, oscillating direction, and turning simple problems into complex ones. This article uses a concrete coding task to measure the real differences between the three tiers.

2. Test Design: A Task That Balances Engineering and Creativity

This test was run in the command-line environment of an AI coding tool. The task was chosen to require both engineering implementation and creative expression. The full prompt was:

Build a minimalist emoji system HTML page based on SVG animation:
1. Support smooth switching between the following expressions at any time: angry, disgusted, fearful, sad, surprised, happy, calm,
   extensible, 12 in total;
2. Each expression state should include some dynamic behavior, appearing lively rather than stiff;
3. Include an automatic switching mechanism that drifts between expressions over time at a seemingly random rhythm.

Reasons for choosing this task:

• Clear engineering checkpoints: SVG animation implementation, an expression state machine, scheduled switching, and a random drift algorithm;
• Subjective room on the creative side: the visual design of expressions and the “liveliness” of animation timing cannot be fully specified;
• Quantifiable process: reasoning length, execution time, number of bugs, fix overhead, and final output can all be recorded.

3. Measured Performance of the Three Tiers

3.1 Low: Draft-Style Thinking, Rushed Execution

The Low tier’s reasoning was very short—about 8 lines in this test—showing a clear “draft-style” character: it listed a few quick points and started writing immediately, with a tendency to rush to completion.

To be clear, “draft-style” is not inherently a flaw—experienced developers also draft first and iterate. The problem is that the Low tier’s draft is too short, short enough that it fails to surface solution-level risks. In this case, the Low tier’s initial approach failed outright, forcing a large amount of re-editing and verification work afterward. The final time and cost actually went up.

3.2 High: The Adaptive Sweet Spot

The most striking feature of the High tier is that it is adaptive: simple problems are reasoned about quickly, but not extremely briefly; complex problems get longer reasoning with serious engineering analysis. In this case, the reasoning was about 600 lines. It did not waste budget on simple sub-problems, nor did it cut corners on complex ones. This “allocate on demand” behavior is the fundamental reason it has become the sweet spot for everyday development.

3.3 Max: Deep Reasoning, but a Grind

The Max tier’s reasoning was about 900 lines. Its typical pattern: first a long round of agonizing over solution design, then another round over implementation details, possibly overturning part of the approach midway, and only then settling on a plan and starting to code. In theory, deep reasoning should yield a higher first-pass correctness rate. In practice, however, the Max tier still failed on one bug—though it was a minor fix, so not a big deal. In other words, the extra reasoning cost did not fully convert into “zero rework.”

3.4 Time Comparison: High and Max Are Nearly Tied

An interesting observation: the end-to-end execution time of High and Max was not very different. The reason is that both tiers’ total overhead consists of “first write + verification and fixes.” Max spent longer thinking before the first write, but it also went through the fix phase, and the two effects cancel out.

4. Cost Breakdown: Don’t Just Look at Reasoning Tokens—Calculate the “Rework Tax”

Breaking the task’s total overhead into parts, it mainly consists of two stages:

1. First write: understanding requirements, designing the approach, and producing the initial code;
2. Verification and fixes: running, finding bugs, localizing them, fixing them, and re-verifying.

This leads to a counterintuitive but important conclusion: the thinking tier’s effect on “first write” overhead is roughly linear, but its effect on “verification and fixes” overhead is nonlinear. Because the Low tier’s initial approach failed, its rework cost far exceeded the cost it saved during the reasoning phase—the tokens saved come back doubled in the form of a “rework tax.”

The takeaway for developers: to evaluate a tier’s true cost, don’t just look at the reasoning token count—look at the total bill of “thinking + writing + fixing.” This is especially true in coding scenarios that require running and verifying against environment feedback.

5. A Counterintuitive Finding on Creativity: Longer Thinking Isn’t Necessarily More Interesting

Judged by the aesthetics of the final output, the Low tier’s work actually had more potential—Max was not necessarily the best.

This phenomenon deserves analysis: the evaluation criteria for creative tasks (visual design, interaction mechanics, animation timing) are divergent and subjective, whereas long reasoning tends to converge repeatedly toward “engineering-safe” solutions. Extensive reasoning smooths out those “imperfect but interesting” attempts. For such tasks, a more effective pattern is:

• Use a lower tier to quickly produce multiple differentiated drafts;
• Let a human pick the direction and participate in experimental iteration;
• Trade the “think it through” budget for “try it several more times.”

This suggests a layered approach: engineering tasks (correctness-oriented) use mid-to-high tiers; creative tasks (diversity-oriented) use a low tier plus human selection; complex architecture tasks (reasoning-oriented) are where the highest tier earns its keep.

6. Let the Model Write Its Own Retrospective

During the test, the model was also asked to produce its own retrospective report. This is a low-cost, high-return practice: by structuring a review of its own execution, the model can clearly point out which step failed and which design was overturned midway, helping a human decide whether the next round should adjust the prompt wording or the tier—rather than guessing.

7. An Easy-to-Misjudge Detail: Displayed Reasoning Length ≠ Reasoning Depth

There is another pitfall when comparing across tools: different platforms display “reasoning processes” differently. Some truncate reasoning by default and show only the first line or a summary, making it look like “very little thinking” when the model may have reasoned for a long time in the background—some models even claim a default high reasoning effort while showing only one line of thinking in the UI. So using “displayed reasoning length” to assess reasoning depth across platforms is unreliable; judge by actual task performance and end-to-end time instead. It is also worth noting that even at the Low tier, coding performance was not inferior to the typical level of the previous generation of models, which suggests that lowering the tier mainly affects reasoning depth and process overhead, not the model’s base capability.

8. Best Practices for Choosing a Tier

Based on the tests above, here are the recommendations:

1. Everyday coding and most tasks: use High (or the default mid tier). It’s not that Max is bad—it’s just too much of a grind; and on simple tasks High won’t consume excessive tokens either, so its adaptive behavior makes it the best value.
2. Clearly simple tasks (config changes, small scripts, local tweaks): the Low tier is usable, but stay alert to the first-pass approach and budget time for verification and fixes.
3. Architecture design, complex algorithms, high-risk changes: the Max tier is worth trying, but pair it with human checkpoints—intervene at key decision points to avoid amplifying directional errors.
4. Creative exploration tasks: run a low tier several times plus human experimentation, which is often more effective than a single Max run.
5. Never skip verification: regardless of tier, include “run-time verification” in the expected cost and design the post-failure fix path into the workflow.

In addition, mainstream model service platforms (such as model services on Baidu AI Cloud) typically provide configuration options related to reasoning effort and thinking depth. Developers can implement dynamic routing at the application layer based on task type: first make a lightweight assessment of task complexity, then decide which tier to send the request to, optimizing the overall cost curve. For specific parameter names and values, refer to the official documentation of the platform you use.

9. Conclusion

The essence of thinking tiers is a trade-off between “thinking cost” and “rework risk”: the Low tier bets on getting it right the first time, the High tier pursues allocation on demand, and the Max tier trades budget for certainty. No tier is absolutely better—the key is matching the task. For most scenarios and most developers, the mid-to-high tiers are the sweet spot; for creative work, trading the saved reasoning budget for “multiple experiments + human involvement” often pays off more. Before choosing a tier, be clear about whether your task is really testing engineering or creativity.