AI Agent Selection Guide: Four Capability Patterns Explained

1. Why Agent Selection Should Not Rely Only on Demos

Many AI agent products complete impressive tasks in demos. They search for information, generate code, operate files, and run multi-step workflows in sequence. However, a large gap separates a demo from production. Production agents must handle permission boundaries, context length limits, tool reliability, cost budgets, audit and compliance requirements, failure rollback, and human takeover. Asking which agent is “smarter” does not help much. The more useful questions are which task pattern an agent fits, which capability layers it has, whether its boundaries are clear, and whether its governance can be implemented in practice.

Common agents fall into four broad patterns: memory-augmented personal assistant agents, autonomous research agents, pair programming agents, and terminal-native coding agents. These patterns are not mutually exclusive. They differ in focus. Understanding the patterns offers more long-term value than memorizing specific product names.

2. A Unified Model: Layers of an Agent

Before making a selection, establish a common model. A practical agent typically consists of the following layers:

1. Model layer: Understands intent, reasons, and generates content.
2. Context layer: Manages inputs such as the current session, task instructions, file fragments, and historical results.
3. Memory layer: Stores user preferences, project state, past decisions, and audit information.
4. Planning layer: Breaks goals into steps, decides when to call tools, and decides when to replan.
5. Tool layer: Connects to search, files, databases, business systems, and code execution environments through tool registration and invocation protocols.
6. Execution layer: Runs commands, reads and writes files, and calls interfaces inside a sandbox, container, or controlled terminal.
7. Governance layer: Handles permission control, approvals, data masking, logging, monitoring and alerting, and rollback.

When evaluating an agent, do not stop at “Can it write code?” Check each layer. Is memory controllable? Is planning explainable? Do tool calls have permission boundaries? Is execution isolated? Are failures observable? The following sections cover the four patterns.

3. Four Capability Patterns and Their Boundaries

1. Memory-Augmented Personal Assistant Agents

The core value of this pattern is long-term context and personalization, not the ability to complete a single task. These agents typically offer preference memory, project state memory, task follow-up, and tool bus capabilities. Users can set an interaction style and have the agent remember long-term constraints, such as “no meetings during certain time periods” or “a project depends on a third-party interface that is not yet connected.” This pattern suits personal knowledge management, cross-application workflow orchestration, long-term project tracking, and scenarios that require stable working habits.

Key technical points include:

• Layered memory: Store short-term session memory, user preferences, project state, and audit logs separately.
• Retrieval and forgetting: Prevent outdated information from affecting decisions.
• Minimal tool permissions: A “butler-like” experience should not grant access to every system permission.
• Controlled memory writes: Require human confirmation or policy checks before writing to memory.

The main risks are privacy leakage, memory contamination, and excessive permissions. When selecting this type of agent, check whether memory can be viewed, edited, and deleted, and whether tool calls go through an approval chain.

2. Autonomous Research Agents

These agents are driven by goals and plan independently. The user provides a goal. The agent then creates a plan, searches for sources, extracts information, compares and verifies findings, and retries when it detects anomalies. It runs an “observe, think, act” loop until it produces a result. These agents usually do not need long-term memory and may plan from scratch for each task, but they have a strong closed loop for a single task. They suit industry research, information synthesis, competitor analysis, test exploration, and open-ended research questions.

Key technical points include:

• Replanning: The planner must support task decomposition and replanning.
• Research tools: The tool layer should provide search, web fetching, parsing, and verification.
• Loop limits: Set budget caps, maximum step counts, and termination conditions to prevent runaway loops.
• Verification: Results need source citations and fact checking.

The main risks are high token consumption, uncontrolled loop costs, and insufficient source credibility. When selecting this type of agent, evaluate its budget controls, termination strategy, citation capability, and human review mechanisms.

3. Pair Programming Agents

These agents understand code context in depth. They read and write files, generate patches, run tests, perform version control operations, and iterate based on test feedback. They suit feature development, refactoring, bug fixing, test generation, and code explanation. Their strength is that they can take over a task after reviewing the current context. Their capabilities are usually limited to code repositories and development workflows, and they drop off noticeably outside development scenarios.

Key technical points include:

• Codebase indexing: Index the repository and select relevant context.
• Patch generation: Produce code changes as patches.
• Test feedback loop: Run tests and iterate based on the results.
• Change scope control: Limit which files and areas the agent can modify.
• Code review integration: Connect output to the existing review process.

The main risks are generating incorrect code, introducing dependency risks, and modifying sensitive files without authorization. When selecting this type of agent, check whether it supports sandboxed execution, whether it requires human approval, whether it produces auditable change descriptions, and whether it works with your existing review process.

4. Terminal-Native Coding Agents

These agents run in a terminal or development environment. Their core functions are command execution, file operations, multi-step debugging, and environment troubleshooting. They keep iterating based on error output, which makes them suitable for local engineering tasks, environment configuration, script automation, and fault localization. Compared with pair programming agents, they place more emphasis on operating in a real execution environment, so their permission and isolation requirements are higher.

Key technical points include:

• Command allowlists: Permit only approved commands.
• Execution sandboxes: Run commands in isolated environments.
• Approval for dangerous operations: Require human approval before high-risk actions.
• Network egress control: Restrict outbound network access.
• Operation logs and rollback: Record actions and support reverting them.

The main risks are accidental file deletion, execution of dangerous commands, leakage of secrets, and unauthorized access. When selecting this type of agent, first confirm that it defaults to least privilege, supports dry-run mode, records complete commands and output, and can run in an isolated container.

4. Architecture Breakdown and Examples

A governable agent loop can be represented with the following pseudocode:

loop:
  goal = parse_user_intent(input)
  plan = planner(goal, memory, context)
  for step in plan:
    tool = select_tool(step)
    if not policy.allow(tool, step):
      require_approval_or_abort()
    result = sandbox.execute(tool, step.params)
    observation = validate(result)
    memory.update(step, observation)
    if need_replan(observation):
      plan = replan(goal, memory, observation)
    if budget_exceeded() or goal_done():
      break

Tool registrations should explicitly state the risk level and approval policy:

{
  "name": "read_file",
  "description": "Read files within an authorized directory",
  "risk": "low",
  "params": {"path": "string"},
  "policy": "allow_in_sandbox"
}

Memory should also be organized into layers:

memory:
  profile: User preferences and long-term constraints
  project_state: Task status and key decisions
  episodic: Historical events and result summaries
  audit: Operation logs and approval records

5. Selection Criteria and Decision Method

Score candidate agents on the following dimensions: whether the task is long-running, whether it must remember preferences, whether it needs autonomous planning, whether it centers on code, whether it requires terminal execution, whether the tool ecosystem fits, context size, unit task cost, security and compliance requirements, observability, and deployment model.

Use these guidelines to narrow the choice:

• For long-running tasks that emphasize personalization, prioritize memory-augmented agents.
• For open-ended tasks that require multi-step research, prioritize autonomous research agents.
• For work centered on a code repository, prioritize pair programming agents.
• For tasks that require command execution and troubleshooting, prioritize terminal-native coding agents.
• For strict compliance scenarios, prioritize solutions that support private deployment, sandbox isolation, approvals, and audit logs.

During a proof of concept, build an evaluation set instead of testing a single demo task. Useful metrics include task success rate, average number of steps, tool failure rate, human takeover rate, unit task cost, unauthorized action interception rate, and rollback success rate.

When deploying on the cloud, general-purpose cloud product categories can support the agent. Object storage can hold generated artifacts, log services can record audit trails, monitoring and alerting can detect anomalies, and container platforms can isolate execution environments. Baidu AI Cloud also offers general-purpose cloud services in these categories that can be used to build surrounding governance capabilities.

6. Rollout Recommendations

Start with a narrow scope. Choose a task with clear boundaries, failures that are easy to roll back, and easy human takeover, such as document summarization, test generation, or log troubleshooting. Then gradually expand tool permissions and autonomy.

Apply these default rules from the start:

• High-risk tools require approval by default.
• All execution runs in a sandbox by default.
• All memory writes can be edited and deleted.
• All tasks keep complete logs.

Drive iteration with metrics rather than demo performance when making purchasing decisions. Agent selection is about choosing a governable way to execute tasks, not the chat interface that looks the smartest.