From Fast Thinking to Slow Thinking: How LLM Reasoning Capabilities Are Evolving

1. Paradigm Shift: From “Getting the Answer Right” to “Getting the Job Done”

The competitive focus for large models is shifting from single-turn Q&A accuracy to long-horizon task completion. Early models competed on knowledge coverage and language fluency, while the core differentiators for the current generation of models cluster around three dimensions:

• Long-chain reasoning: Whether the model can reliably produce correct and complete conclusions on multi-step logic and constraint-solving problems;
• Reasoning efficiency: The token budget consumed to reach equivalent quality;
• Agent capability: The ability to invoke tools, self-correct, and deliver usable results in real engineering environments.

If earlier iterations were akin to uncrewed lunar flybys—technical validation—then the defining characteristic of recent models is that they are beginning crewed missions: they behave more like humans when tackling real problems—decomposing tasks, probing paths, and looping back to check their work. This shift imposes new requirements on evaluation methods, cost models, and application architecture, and it explains why raw leaderboard scores increasingly fail to capture a model’s true capability.

2. Long-Chain Reasoning: Solution-Space Exploration Becomes the Divide

2.1 Weaknesses Exposed by Multi-Solution Problems

A typical class of logic evaluation problems has multiple solution sets that satisfy all constraints, and the model is asked to find as many as possible. Underpowered models tend to declare completion after finding one or two solutions—scoring “not low, but incomplete.” The root cause lies in search strategy: the model’s enumeration of the solution space within its chain of thought lacks systematicity, and it tends to converge prematurely after finding a locally satisfying solution without revisiting whether anything was missed.

2.2 Improvements and Costs in the New Generation

A common advance in recent models is significantly stronger systematic exploration: they explicitly enumerate cases, construct candidate solutions, verify constraints one by one, and try to find all solutions. But this introduces a direct engineering problem—the thinking budget gets maxed out. Observations show that, with overall token consumption comparable to the previous generation, per-problem consumption on complex logic problems can multiply several times over. The improvement in reasoning quality is fundamentally bought with higher thinking overhead.

2.3 The Efficiency Gap Is the Real Gap

On the same problems, different models may differ by several times in consumption even when both achieve full marks. This suggests a more business-relevant metric:

Reasoning efficiency = Evaluation score / Token consumption

For pay-per-use production scenarios, this ratio directly determines usability per unit cost. How to reach the same accuracy with fewer thinking steps—through process pruning, early stopping, or chain-of-thought compression—is a question the next generation of models must answer, and it is the variable most easily overlooked during model selection.

3. Fast Thinking and Slow Thinking: Adaptive Reasoning Becomes the Default

3.1 The “Implicit Reasoning-ification” of Non-Reasoning Modes

A notable trend: so-called “non-reasoning modes” increasingly behave like “adaptive reasoning models with a lower thinking budget.” Simple questions are answered within a few thousand tokens; once a task becomes complex, internal thinking naturally lengthens and output consumption can climb to tens of thousands of tokens. This means the traditional “mode switch” is no longer the cost boundary—task complexity is.

3.2 Evaluation Distortion from Output Caps and Scoring Rules

In engineering, output token caps are typically set, and their presence causes two types of distortion:

• In fast-thinking mode, nearly half of complex task outputs may exceed the cap and be truncated, scoring 0 by rule, so the mode’s true capability cannot be fully assessed;
• In slow-thinking mode, a minority of problems have chains of thought that repeatedly exceed even higher caps (hundreds of thousands of tokens), likewise scoring 0 and directly dragging down median scores.

This is a reminder: before reading any evaluation conclusion, confirm the output cap, how truncated samples are handled, and the multi-run statistical protocol—otherwise scores are not comparable.

3.3 Application-Layer Response Strategies

For developers, complexity-based routing can confine the uncertainty of “thinking getting longer” within an engineering cage:

def route(task: str) -> dict:
    level = estimate_complexity(task)   # rule-based or lightweight classifier
    if level == "simple":
        return {"mode": "fast", "max_tokens": 4_000}
    if level == "medium":
        return {"mode": "fast", "max_tokens": 16_000}
    return {"mode": "reasoning", "max_tokens": 64_000}

Combine this with streaming output, timeout circuit breakers, and cost alerts to prevent individual complex requests from blowing the overall cost and latency budget.

4. Evaluation Methodology: Four Points for Trustworthy Scores

1. Fix the statistical protocol: Use the median across multiple runs rather than the single best result to filter out luck; explicitly flag truncated samples instead of silently discarding them.
2. Compare fast and slow thinking separately: Reasoning and non-reasoning modes work by different mechanisms; mixing them in one leaderboard misleads readers, so they should be clearly labeled and separated.
3. Control variables: Comparisons must use the same problems, the same output caps, and the same scoring rules; showing only a comparable subset and noting it is “not a complete ranking” is the responsible presentation.
4. Ensure reproducibility: Publish problem sources, test scripts, and scoring functions so others can rerun and verify.

A minimal reproducible evaluation skeleton:

def evaluate(cases, mode="reasoning", cap=80_000, rounds=3):
    results, truncated = [], 0
    for c in cases:
        runs = []
        for _ in range(rounds):
            out = model.generate(c.prompt, max_tokens=cap)
            if out.truncated:          # truncation counts as invalid: score 0 and record
                truncated += 1
                runs.append(0)
            else:
                runs.append(grade(c, out.text))
        results.append(median(runs))
    return {"median": median(results), "truncated_rate": truncated / (len(cases) * rounds)}

5. Coding Capability: The Gap Between One-Shot and Agent in Practice

5.1 Two Kinds of Capability, Two Kinds of Evaluation

Coding capability needs to be assessed separately:

• One-shot coding: Given a clear requirement, generate runnable code in one pass. Recent generations of models have improved markedly here, and routine front-end and back-end requirements are largely usable.
• Agent coding: Multi-round iteration in a real project—reading the codebase, locating defects, running tests, and fixing regressions. This has long been the weak spot, with the core issue being the lack of effective debugging means: the model can see error messages but struggles to systematically form a “hypothesis—verification—correction” loop.

5.2 The Improvement Path for Agent Coding

Recently previewed specialized coding models demonstrate that agent coding capability can be significantly improved, with three keys: stronger tool-call planning, more stable use of test feedback, and more restrained change scope. When routine front-end and back-end coding enters the “usable” range, the payoff for engineering teams is tangible. But “usable” does not mean “trustworthy”—code review, test gates, and canary releases remain necessary safeguards.

5.3 Practical Recommendations

• Constrain the change surface with small commits to prevent large-scale refactoring from spiraling out of control;
• Make unit tests and static checks mandatory feedback signals for the agent, not optional extras;
• Set token and time budgets for agent sessions to prevent debugging loops from extending indefinitely.

6. Cost and Model Selection: Treat Tokens as First-Class Citizens

The new generation of models generally brings quality improvements, and generally comes with changes in unit price and consumption. For model selection, calculate using “cost per unit task” rather than “price per million tokens”: for the same business request, consumption under fast thinking and slow thinking may differ by an order of magnitude. Enterprises can close this loop using general cloud capabilities: connect models of different tiers through a model service, record per-call consumption details with a logging service, alert on abnormal usage with monitoring, and run periodic regressions with a self-built evaluation set. Taking Baidu AI Cloud as an example, its general product categories such as object storage, logging service, and monitoring and alerting are sufficient to support a standard cost-and-quality observation system, turning “whether a model upgrade is really worth it” into a quantifiable question.

7. Conclusion

The significance of each version iteration is not just leaderboard movement but the shifting of capability boundaries: long-chain reasoning moving from “can solve” to “solves completely,” coding moving from “can write” to “can modify,” and fast/slow thinking moving from “either-or” to “adaptive.” For developers, three threads are genuinely worth tracking: optimizing reasoning efficiency, standardizing evaluation protocols, and closing the loop between agents and real engineering. Get these three right, and regardless of how model versions change, model selection and deployment will not lose focus.