Agent Foundation Models: Core Capabilities and Implementation Practices for Moving from Chat to Work
This article examines why large language models that perform well in conversation often struggle with multi-step agent tasks. It explains the difference between single-turn dialogue and long-horizon agent workflows that require tool invocation, feedback handling, error recovery, and state retention. It then outlines three core capabilities for agent foundation models—stable tool calling, complex task planning and decomposition, and sustained execution over long tasks—and offers engineering practices such as clear tool design, controlled task-chain length, observability, and human checkpoints. The goal is to help teams build more reliable agents by combining suitable foundation models with robust engineering.
I. A Common Phenomenon: Why Chat-Capable LLMs Are Not Necessarily Work-Capable
Over the past year, the concept of agents has continued to gain momentum. Many individual developers and enterprise teams have eagerly built their own agent systems: connecting a large model, configuring a set of tools, and writing prompts in the hope that it will act like a “digital employee” and automatically complete long-chain tasks such as data organization, code development, and report generation.
But in practice, the gap is often large:
- For simple conversation, article writing, and question answering, the model performs quite well.
- Once it is asked to execute a complex task spanning multiple steps—for example, “read a batch of files, analyze the content, call an external API to write the results, and finally summarize the report”—tool calls fail, parameters are passed incorrectly, execution stalls halfway through, or the model even “forgets what it is doing.”
Many users have the intuitive feeling that an agent built with considerable compute and engineering cost can be less useful in daily experience than an ordinary chat assistant. This gap is not mysticism; there are clear technical root causes behind it.
II. Technical Root Cause: Dialogue Tasks and Agent Tasks Require Different Capability Models
1. Dialogue Tasks Are Single-Turn and Short-Chain
Ordinary question-answering and writing tasks are essentially a short-chain “input—generation—output” process. The model only needs to understand semantics and organize language. It does not involve interaction with external systems, and the requirements for output-format precision are relatively low. Current mainstream large models are already quite mature in this type of task.
2. Agent Tasks Are Multi-Turn, Long-Chain, and Externally Interactive
Agent tasks are completely different. They require the model to repeatedly complete the following actions within a task chain that may last for hours:
- Understand the goal: transform a broad, vague goal into an executable plan;
- Call tools: initiate tool calls in a strictly structured format such as JSON, with parameters that must precisely match the interface definition;
- Observe feedback: read the tool return result, determine success or failure, and decide the next action;
- Correct dynamically: after a step fails, be able to backtrack and adjust the strategy rather than getting stuck or giving up;
- Maintain state: preserve memory of the task context, completed steps, and remaining goals throughout the long chain.
If any of these five stages is unstable, the entire task chain breaks. General large models are trained mainly for language generation in the form of “next-token prediction,” and they are not specifically optimized for “format precision in multi-turn tool calling” or “planning persistence over very long tasks.” This is the fundamental reason many models are “strong at chatting but weak at working.”
A widely accepted industry formulation is:
A usable agent = strong foundation model × good engineering
Model capability and engineering framework are both indispensable. If a team only stacks prompt techniques and tool plugins while the foundation model itself lacks structured invocation and long-horizon planning capabilities for agent tasks, the system is bound to be fragile.
III. Three Core Capabilities of Agent Foundation Models
To judge whether a model is suitable as the “brain” of an agent, the industry generally focuses on the following three dimensions:
1. Stability of Tool Calling
This is the first threshold for agent reliability. Typical failure modes include calling a nonexistent interface, passing parameters with the wrong type or format, and reinterpreting the result of a function call as user input. A foundation model trained for agent scenarios should be specifically strengthened in format adherence for structured output—for example, generating invocation parameters strictly according to the schema and maintaining consistent semantic understanding of invocation results. In practice, risks can be reduced as follows:
// Example tool definition: a clear parameter schema can significantly improve call success rate{"name": "query_database","description": "Query the business database by conditions","parameters": {"type": "object","properties": {"table": { "type": "string" },"filters": { "type": "object" },"limit": { "type": "integer", "maximum": 100 }},"required": ["table"]}}
It is also recommended to add parameter validation and failure retry mechanisms at the engineering layer, forming a double safeguard of “model capability + engineering fallback.”
2. Planning and Decomposition of Complex Tasks
A qualified agent should act like both a project manager and an architect: after receiving a broad goal such as “build a data dashboard,” it should first decompose the work into stages such as requirements analysis, technology selection, module implementation, and integration verification, then refine them layer by layer into executable steps. When necessary, it should also be able to assign independent subtasks to multiple sub-agents for parallel processing. This requires the model to have strong task decomposition and reflection capabilities rather than mechanically “doing whatever step comes to mind.”
3. Sustained Execution of Long Tasks
Real productivity scenarios often mean tasks that “run unattended for hours”: batch code refactoring, cross-system data migration, and overnight automated testing. This places extremely high demands on the model’s context management—it must not “lose memory” midway, deviate from the goal, or spin repeatedly on a certain error. Only by combining engineering methods such as breakpoint recovery, task-state persistence, and checkpoints at key nodes can such long-running tasks be supported reliably.
IV. Engineering Best Practices for Making Agents Usable
Even if a foundation model optimized for agent scenarios is selected, several engineering best practices have been repeatedly validated:
- Tool design should be “small and clear”: each tool should have a single responsibility, an accurate description, and explicit parameter constraints. This has a much higher call success rate than providing one “universal large tool.”
- Control the length of a single task chain: artificially divide very long tasks into multiple stages. At the end of each stage, persist the state and generate a checkpoint so that failures can recover from the breakpoint rather than starting from zero.
- Establish observability: fully record the input, output, and latency of every tool call, connect to logging services and monitoring alerts, and quickly locate “which step is stuck.”
- Visual orchestration and human intervention points: manage multiple agents like a virtual team, set human confirmation at key decision nodes, and balance automation efficiency with controllability of results. Cloud platforms that integrate model invocation, tool orchestration, and monitoring alerts can serve as reference infrastructure for building such systems.
V. Conclusion
“Chatting” and “working” are two different things, and this is one of the most easily underestimated lessons in current agent implementation. Rather than complaining that agents fall short of expectations after the novelty fades, it is better to return to the technical essence: choose foundation models with stable tool calling, task planning, and sustained long-task execution, and then use rigorous engineering to close reliability gaps. When model capability and engineering framework work together, agents can truly evolve from “demo toys” into productivity tools that can be entrusted with long tasks.
