Multi-Step LLM Agents: Core Capabilities, Evaluation, and Engineering Practices

1. Why Multi-Step Execution Is a Key Divider for Large Model Capability

Over the past year, the industry’s focus when evaluating large models has gradually shifted from whether a model can answer a single question accurately to whether it can independently complete a complex task that involves multiple steps and spans a longer time period without constant human intervention. This type of capability is often described as “multi-step execution” or “agentic capability.”

The question is not whether the model can generate a good response once, but whether it can break down a high-level goal into executable steps, choose appropriate tools, adjust its strategy based on intermediate results, and ultimately deliver a usable outcome.

Compared with single-turn generation, multi-step execution requires the model to maintain goal consistency over a longer time span. This depends on three foundational conditions:

• Sufficient context capacity: The model needs enough context window space to hold task history, tool outputs, and intermediate reasoning.
• Cross-context reasoning: The model must be able to connect information from earlier parts of the conversation with later steps, rather than merely remembering it without using it.
• Robust planning and error correction: The model must be able to backtrack, retry, and revise its strategy when tools fail or results do not meet expectations.

Understanding these conditions also explains why “larger parameter count” alone does not automatically mean “better at completing work.” Multi-step execution is a composite capability that depends on both architecture and reasoning mechanisms.

2. Four Core Dimensions of Multi-Step Capability

1. Long-Context Understanding and Cross-Context Reasoning

A long context window addresses how much information can be stored, while cross-context reasoning addresses whether that information can actually be used. If a model has a context window measured in hundreds of thousands or millions of tokens, but can only make local associations within recent content, it may still lose critical clues when facing a large codebase, a long document, or a multi-turn interaction history.

The real value lies in the model’s ability to locate key facts within a long context, connect dependencies across different parts of the input, and correctly reference them in later steps.

To evaluate this capability, one practical approach is to combine “needle-in-a-haystack retrieval” with “multi-hop reasoning” tests. For example, several distributed clues can be embedded in a long text, and the model can be asked to synthesize them into a single conclusion. This helps determine whether the model genuinely understands the full context or is only performing surface-level matching.

2. Task Decomposition and Planning

Planning is the “brain” of multi-step execution. Several general paradigms already exist. One is the reasoning-action alternating pattern, often described as “think while acting.” Another is the two-stage pattern of “plan first, then execute.”

The first pattern is flexible and can adjust based on environmental feedback. The second provides a clearer global view and is easier to review. In practice, a common approach is to combine the two: a planner first generates a high-level step list, then an executor advances step by step. After each step is completed, the process returns to the planner to re-evaluate whether the remaining plan needs revision.

A key feature of a high-quality planner is its ability to transform vague goals into verifiable intermediate states. For example, “refactor this module” should be decomposed into a concrete sequence such as: list dependencies, extract shared logic, modify interfaces, run tests, and fix failing cases.

3. Tool Calling and Computer-Use Operations

Multi-step execution must ultimately lead to action. Tool calling allows a model to perform retrieval, run code, read and write files, call external services, and more. A further step is “computer use,” where the model must understand interface states, generate operation sequences, and determine whether operations succeed based on feedback.

In engineering, the quality of the tool layer directly determines the upper limit of what an agent can do:

• Tool descriptions should be clear, so the model can accurately judge when and how to use them.
• Tool outputs should be structured and information-dense, avoiding overly verbose responses that overwhelm the context.
• Every operation should provide a clear success or failure signal, allowing the model to decide whether to continue or roll back.

4. Reflection, Error Correction, and Strategy Adjustment

A true agent does not blindly continue along a failing path. When a step fails, it needs to read the error message, identify the cause, and adjust its approach. This “reflection” capability is a key boundary between multi-step systems that can only demonstrate behavior and those that can work reliably.

In practice, explicit self-check steps can be introduced. After completing each subtask, the model can compare the output against the original goal and correct deviations immediately, rather than discovering at the end that the entire task has failed.

3. How to Evaluate Multi-Step Capability Scientifically

Since external evaluation sets are often not directly reusable, it is useful to define general evaluation dimensions that can be adapted to specific business scenarios. The following categories can help quantify multi-step capabilities:

• Code generation and modification tasks: Given a real repository and a requirement description, evaluate whether the model can locate relevant files, generate correct changes, and pass tests.
• Terminal or command execution tasks: Evaluate whether the model can automatically combine commands, interpret outputs, and handle errors.
• Long-horizon planning and simulation tasks: In environments that require multiple rounds of decision-making, such as resource allocation or business simulation, evaluate strategy consistency and final outcomes.
• Human preference evaluation: Have real users compare different systems in blind tests, focusing on task completion, instruction following, and fluency. This can complement automated metrics.

During evaluation, two points are especially important: task realism and result verifiability. Overly toy-like tasks do not reflect real-world capability, while tasks without automatic judgment criteria are difficult to scale. The ideal approach is to keep evaluation tasks close to real workflows and provide verification scripts that can be executed automatically.

4. Engineering Practices for Building Multi-Step Agents

A practical multi-step agent usually consists of four components: planner, executor, memory module, and toolset. The following example loop skeleton illustrates the control flow. It is intended for explanation rather than as a specific product interface.

def agent_loop(goal, tools, memory):
    plan = planner.decompose(goal, context=memory.snapshot())
    while not plan.is_empty():
        step = plan.next()
        # Execute a single step and obtain structured feedback
        result = executor.run(step, tools)
        memory.append(step, result)
        if result.failed:
            # Trigger reflection and replanning
            revised = planner.revise(plan, error=result.error, memory=memory)
            plan = revised
        if evaluator.check(goal, memory):
            break
    return memory.summary()

Around this skeleton, several engineering points are particularly important:

1. Context management: Long-running tasks continuously generate intermediate results. A strategy of “summarize, compress, and archive” is needed to condense historical information into fragments useful for current decisions, preventing unbounded context growth.
2. Idempotency and retries: Tool calls should be designed to be safely retryable, preventing duplicate writes and dirty data after failures.
3. Observability: Recording the input, output, and latency of each step is essential for identifying where a task gets stuck.
4. Cost guardrails: Multi-step tasks naturally consume more tokens. Step limits, timeouts, and budget thresholds are needed to prevent infinite loops.

5. Cloud Deployment: Balancing Cost and Stability

Multi-step applications place new demands on underlying infrastructure. When building on the cloud, teams can use general capability categories to control complexity and cost:

• Use object storage to save intermediate artifacts and task snapshots, enabling recovery and resumption.
• Use message queues to decouple planning, execution, and callbacks, allowing long-running tasks to progress asynchronously.
• Use container platforms to host executors and tool sandboxes, supporting elastic scaling and environment isolation.
• Use logging, monitoring, and alerting services to track the full task lifecycle and detect abnormal steps quickly.

For model selection, a practical strategy is layered routing: assign simple, high-frequency substeps to lower-cost lightweight models, and use more capable models only at critical points that require complex reasoning and planning. This approach helps control overall cost while still approaching higher-level intelligence and covering more task scenarios. On the cloud, Baidu AI Cloud provides model services and supporting tools that can help teams move from prototype to production.

6. Conclusion

The core of multi-step capability is not leadership in a single metric, but the coordination of a complete mechanism: long-context reasoning, task decomposition and planning, tool calling, and self-correction. To evaluate it, automated evaluation close to real workflows should be combined with human preference evaluation. To build it, clear planner-executor architecture, robust context management, and error management are required. To deploy it, cloud-based systems need layered routing and observability to control cost and stability. Only by making these fundamentals solid can large models move from “answering questions” to “getting work done.”