From Answering Questions to Getting Work Done: The Evolution of Agentic AI Capabilities

Why “Answering Questions” Is Not the Same as “Getting Work Done”

For years, the primary way to evaluate LLMs was through static benchmarks: multiple-choice questions, math problems, reading comprehension. These tasks share three traits—single-turn interaction, closed inputs, and one correct answer. Real-world work tasks are almost the opposite in every dimension:

• Multi-step, long-horizon: Delivering a usable data analysis report requires understanding requirements, pulling data, cleaning it, modeling, visualizing, and writing conclusions—any step may require revisiting earlier ones.
• Dependent on external systems: Querying databases, calling APIs, reading files, and operating browsers—the answers are not inside the model’s weights but in the environment.
• Open-ended outcomes with side effects: There is no single correct answer, only “did the goal get achieved.” Every database write or notification sent is an irreversible real-world action.

This creates three structural gaps:

1. Misaligned training objectives. The base objective of a language model is next-token prediction; alignment optimizes for “generating responses humans like,” which is still fundamentally about answering questions. Getting work done, by contrast, requires making a series of correct decisions in a dynamic environment. These are different task distributions.
2. Lack of grounding in the environment. Models never truly interact with tool return values, API errors, or the real state of systems. They tend to continue generating text without knowing what actually happened after execution—which is why hallucinations are amplified in execution chains.
3. Error accumulation. If a single step has a 95% success rate, a 20-step task chain succeeds end-to-end at only about 0.95^20 ≈ 36%. Long-horizon tasks demand near-perfect reliability at every step. This is the core difficulty in agent deployment.

So going “from answering to doing” is not a quantitative increase in the same capability—it is a qualitative shift from static generation to dynamic decision-making.

The Five-Layer Capability Stack of an Agent

A capable agent is built from five layers. Missing any one of them will expose weaknesses in real tasks.

1. Task Understanding and Planning

Translating a vague human instruction into an executable sequence of steps. Planning capability shows up in three ways: task decomposition (breaking a goal into dependent subtasks), dynamic re-planning (adjusting the path when a step fails or the environment changes), and resource awareness (knowing which steps are worth doing and which can be skipped). A common engineering pattern is “plan globally first, then execute step by step”: the planner outputs a task graph, and the executor follows the dependency order.

2. Tool Calling

This is the most direct marker of “getting work done.” The model must convert natural language intent into structured function calls: given a set of tool names, descriptions, and parameter schemas, it outputs arguments that conform to the schema. Description quality directly determines call accuracy—the clearer the description and the more explicit the parameter constraints, the lower the probability of choosing the wrong tool or passing the wrong arguments. A well-defined tool schema looks like this:

{
  "name": "query_order_status",
  "description": "Queries the current status of an order by order ID. Only call this when the user provides an order ID.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": { "type": "string", "description": "The order ID" }
    },
    "required": ["order_id"]
  }
}

Note the negative constraint: “Only call this when…” As the number of registered tools grows, the decision space widens, and clear boundary descriptions reduce false invocations.

3. Memory and Long Context

Long tasks generate a large amount of intermediate artifacts: tool return values, web content, logs. No matter how large the context window, it will eventually hit a limit. This calls for a tiered memory strategy—recent steps retain full detail, earlier information is compressed into summaries, and regenerable data (like file contents) is stored as references rather than full text, re-read only when needed. Good context management keeps the model from “losing memory” or being buried in irrelevant information in the latter half of a long task.

4. Reflection and Error Correction

The “act–observe–reflect” loop is what separates an agent from a chatbot with tools. After each tool call, the model must judge: Does the return value match expectations? Should I retry differently? Should I terminate early and ask a human? Feeding failure information back into the model for re-decision creates a negative feedback loop. Without this layer, a single failure kills the entire chain.

5. Code Execution and Sandboxing

For many tasks, the best tool is “write a quick script on the spot”: data transformation, format parsing, batch computation. The model generates code, executes it in an isolated sandbox, reads the result, and continues reasoning. This gives the model a safe workbench for trial and error. Sandbox isolation is both a security requirement and an engineering prerequisite for letting the model experiment boldly.

What Drives the Capability Leap

The visible improvement in agent capabilities over the past two years rests on four technical pillars:

1. Reinforcement learning with verifiable rewards. Tasks like math, code, and tool calling can be automatically graded. This allows “task completion” to serve directly as a reward signal. Compared to fitting human preferences, verifiable rewards align the model with “getting things done,” teaching it to explore, backtrack, and persist across multiple steps.
2. Agent trajectory data and environment simulation. By building task environments with realistic tool returns, models experience complete success and failure trajectories in synthetic settings, learning which tools to call under what circumstances and how to recover from failures.
3. Long-context and long-horizon consistency training. While context windows extend to hundreds of thousands of tokens, training also ensures the model can still accurately locate and use information in the later portions of long texts—avoiding the “saw it but didn’t use it” problem.
4. Standardization of tool integration. The industry is converging on unified tool description and invocation conventions. Tool developers register capabilities in the same format, and models invoke them in the same format, which dramatically reduces the engineering cost of connecting agents to external systems. This has accelerated the growth of the tool ecosystem.

Engineering: Turning Model Capabilities into Reliable Systems

Model capability sets the ceiling; engineering sets the floor. For production-grade agent applications, the following design points matter most.

Architecture layering: Decouple planning, execution, validation, memory, and observability into five modules. The model handles decisions; deterministic logic belongs in traditional code. Do not make the model do things that a validator or retry mechanism should do. A minimal pseudocode of the main loop:

MAX_STEPS = 25
memory, step = Memory(), 0
while not task.done() and step < MAX_STEPS:
    action = llm.decide(task, memory, tool_schemas)   # Plan next step
    try:
        obs = executor.run(action.tool, action.args)  # Execute tool call
        validator.check(action, obs)                  # Validate structure and business rules
    except ToolError as e:
        obs = f"Execution failed: {e}"                # Feed failure info back
    memory.append(action, obs)                        # Write to memory
    step += 1

Key engineering practices:

• Idempotency and retry: Write operations must carry idempotency keys. A retry triggered by network jitter must not cause duplicate orders or duplicate submissions.
• Least-privilege permissions: Each tool is granted only the minimum permissions required for the task. High-risk operations (fund transfers, data deletion, external communications) require human approval gates.
• Step-level observability: Record every decision rationale, tool inputs and outputs, latency, and cost. Full replay must be possible; otherwise, production issues cannot be diagnosed.
• Incremental delivery: Start with human-in-the-loop (model suggests, human executes), then expand automation gradually as reliability data accumulates—rather than going fully autonomous in one step.

On the model service side, mainstream cloud platforms currently provide general-purpose components for model inference, function calling, sandboxed execution, and workflow orchestration. Developers can build the architecture described above on platforms such as Baidu AI Cloud’s general model services and agent development components, focusing their effort on business-specific tools and evaluation systems.

Evaluation: From Answer Accuracy to Task Completion Rate

Evaluation determines the direction of iteration. In the agent era, evaluation must answer four questions:

1. End-to-end completion rate: The proportion of tasks that ultimately achieve their goals. This is the north-star metric.
2. Step-level success rate: Tool selection accuracy and parameter validity rate, used to pinpoint capability bottlenecks.
3. Cost and latency: Average token consumption, number of tool calls, and wall-clock time per task. Capability gains must not come at the cost of uncontrolled spending.
4. Stability: The variance across multiple runs of the same task. An agent that succeeds 3 out of 10 times is far less deliverable than one that succeeds 8 out of 10.

Evaluation sets should come from real business tasks as much as possible, and deliberately include dirty data, broken interfaces, and other interference to test the model’s error-recovery ability rather than its “test-taking” ability.

Challenges and Outlook

Three hurdles remain for current agents. First, the long tail of reliability: common tasks now achieve high completion rates, but edge cases remain fragile. Second, security: prompt injection can induce models to misuse tools, and longer tool chains mean larger attack surfaces. This requires sustained investment in input filtering, permission isolation, and behavior auditing. Third, cost: long-context multi-turn interactions are significantly more expensive than single-turn Q&A. Caching and model-level routing (small models for simple steps, large models for critical decisions) help amortize the expense.

Looking ahead, the direction is clear: longer autonomous task horizons, finer-grained UI and system operations, multi-agent collaboration, and a shift from “executing instructions” to “proactively understanding goals.” For developers, rather than obsessing over benchmark leaderboards, the sooner you build your own task evaluation set and observability infrastructure, the better—because the ability to “get work done” can only grow through repeated collision with real tasks.