From Answering Questions to Getting Things Done: How LLM Agent Capabilities Are Evolving

1. The Shift in Capability Boundaries: Why “Answering Well” Is No Longer Enough

Early large language models earned their reputation as “more powerful question-answering machines”: given a prompt, they returned a piece of text, and the interaction ended with one question and one answer. This form did create value in education, customer service, and copywriting, but once plugged into real business workflows, two structural weaknesses emerged:

1. Output is the endpoint. The model produces “suggestions,” not “results.” Turning a suggestion into a business action—querying a database, calling an API, writing a file, sending a message—still requires a human.
2. Lack of process controllability. Businesses don’t want “a plausible-sounding paragraph”; they want verifiable intermediate states: what each step did, what data it depended on, where it failed, and how to retry.

As a result, the industry’s yardstick is shifting from “was the answer correct” to “was the task completed.” This is the fundamental driver behind the leap from conversational AI (Chat) to agents: the model no longer just produces text, but must—given a goal—autonomously plan, call tools, and close the loop on the task.

2. The Four Pillars of the Agent Paradigm

A task execution that actually “gets work done” typically involves four capability modules working together.

2.1 Planning

Faced with a goal like “compile last week’s sales data into a weekly report and send it to the team,” the model first needs to decompose it:

• Define deliverables: a report document plus a send action;
• Identify dependencies: where the data comes from (database, spreadsheet, or API), who receives it, and what format is required;
• Break down the steps: fetch data → clean and aggregate → generate charts and text → validate → send.

Two strategies are common: “plan first, execute later,” which produces a complete plan up front and then implements it step by step; and dynamic planning, which adjusts the remaining plan after each step based on observed results. Real systems usually blend the two: a coarse-grained plan comes first, and fine-grained actions are generated on demand.

2.2 Tool Use

Tool calling is the bridge from “talking” to “doing.” At its core, it translates natural-language intent into structured calls: the model decides “which tool to call and what parameters to pass” based on tool names, descriptions, and parameter constraints, while the runtime performs the actual execution and returns the results.

The quality of tool definitions directly determines call accuracy. Field-proven practices include:

• Tool descriptions should state clearly “when to use the tool and when not to,” not just the parameter specs;
• Parameter types, enum values, and required fields should be strictly constrained, leaving the model no room to improvise;
• Tool granularity should be moderate: too fine and the call chain becomes long; too coarse and each tool carries too much internal logic to reuse.

2.3 Execution Environment

Some tasks require more than API calls—they demand real computation and verification, such as statistical analysis, format conversion, and batch processing. This is where a code execution environment comes in: the model generates code, runs it in a controlled sandbox, and feeds the execution output (results, errors, logs) back to the model for further reasoning. Error messages are especially valuable—they turn “whether the model guessed right” into “whether the system verified it.”

2.4 Memory and Reflection

Long tasks span many rounds of calls, and context management becomes the bottleneck. The common approach is layering:

• Working memory: the current task’s conversation and observations, summarized and compressed when they grow too long;
• Long-term memory: user preferences and historical conclusions, persisted and retrieved on demand;
• Reflection: after a step fails, the model attributes “why it failed,” adjusts its strategy, and retries, rather than looping through the same failure.

3. The Core Loop: Reason–Act–Observe

Stringing the pillars together yields an execution loop widely adopted across the industry:

1. The model receives the goal and the current state;
2. It reasons and decides: answer directly, call a tool, or request human confirmation;
3. It executes the action and obtains an observation;
4. It merges the observation into its context and returns to step 1 until the final deliverable is produced.

Illustrative pseudocode:

messages = [system_prompt, user_goal]
while True:
    resp = llm.chat(messages, tools=tool_schemas)
    if resp.tool_calls:                      # The model decides to call tools
        for call in resp.tool_calls:
            args = validate(call.arguments)  # Validate against constraints; block fabricated params
            result = dispatch(call.name, args)
            messages.append(tool_message(call.id, result))
    else:                                    # No tool call: treat as the final answer
        final = resp.content
        break

Two engineering details deserve emphasis:

• Validate parameters up front. Never unconditionally trust model-generated parameters; validate them against the tool definitions before business execution to significantly reduce incidents caused by “hallucinated parameters.”
• Make termination conditions explicit. Maximum step counts, budget caps, and human-approval checkpoints should all be written into the orchestration logic to prevent the model from falling into ineffective loops.

4. A Typical Production Architecture

An agent system that delivers production value usually spans five layers:

• Model layer: route tasks of different difficulty to different model tiers—simple intent recognition goes to lightweight models, complex planning to reasoning-enhanced models—balancing quality and cost;
• Orchestration layer: implements the loop above, along with state machines, retry, and rollback strategies;
• Tool layer: wraps internal APIs, database queries, retrieval services, messaging, and other atomic capabilities;
• Runtime layer: code sandboxing and permission isolation to ensure execution safety;
• Observability layer: end-to-end logging and tracing that records the inputs and outputs of every reasoning step and call, supporting replay and root-cause analysis.

Whether you build from scratch or on a cloud platform, this layered structure is broadly applicable. Model service platforms such as Baidu AI Cloud commonly provide categories of general capabilities—model inference APIs, function calling protocols, knowledge retrieval, and agent orchestration—on top of which developers can compose execution systems tailored to their own businesses. The critical work still lies in tool design and workflow governance.

5. Upgrading the Evaluation System in Sync

When model capabilities change, evaluation must change with them. Traditional QA benchmarks measure “score per question”; agent evaluation must measure “end-to-end delivery”:

• Task success rate: whether the final deliverable meets the bar on real or highly simulated task sets;
• Process efficiency: steps taken, tool calls made, token consumption, and latency;
• Robustness: whether the system degrades gracefully under injected dirty data, tool failures, or restricted permissions;
• Security and compliance: whether unauthorized operations, sensitive data leaks, or unapproved irreversible actions occur.

Teams are advised to build internal evaluation sets covering their core business workflows early, turning “can this go live” from a gut feeling into quantifiable, regression-testable metrics.

6. Common Challenges and Engineering Countermeasures

1. Error accumulation. In multi-step tasks, even high per-step accuracy compounds into a declining overall success rate. Countermeasures: introduce checkpoint mechanisms that validate intermediate results at key nodes; parallelize steps where possible to shorten the call chain.
2. Hallucination and privilege escalation. Countermeasures: tool whitelists, strict parameter validation, and the principle of least privilege; mandatory human approval for write operations and money-related actions.
3. Cost and latency. Countermeasures: tiered model routing, result caching, and long-context compression—avoid routing every request to the largest model.
4. Insufficient explainability. Countermeasures: require the model to output a brief rationale before key actions, paired with observability-layer records for after-the-fact auditing and continuous improvement.

7. Conclusion

From “answering questions” to “getting things done,” the surface change is a capability upgrade driven by model iteration; the deeper change is a migration of interaction paradigms: the model shifts from information producer to task executor, and applications evolve from chat windows into digital collaborators with planning, tools, memory, and boundaries. For developers, what ultimately determines delivery success is often not the model itself, but the granularity of tool design, the rigor of orchestration logic, and the completeness of the evaluation system. Get these three right, and the model’s capability dividend converts into deterministic business value.