Agent Selection Guide: From Scenarios to Architecture

Agent selection is first an architecture problem, not an experience evaluation. Many teams collapse “can chat, can write code, can complete a task unattended” into a single score. After launch, the agent either forgets context, spins in loops when no one is watching, or fails completely outside the repository. A more reliable approach is to decompose capabilities into composable modules: perceiving the current state, calling external tools, planning the next step, retaining necessary memory, and delivering results within permission boundaries. Selection should compare the strength and composition of these modules, not a product name.

Understand Three Capability Profiles in Task Language

For readers who are not familiar with function calling or tool calling, it helps to ignore the terminology at first and look only at how work gets done.

Long-running companion executor

This profile fits work that spans days, crosses systems, and requires strong personalization. The system maintains an updatable user and project profile: preferences, prohibitions, fixed rhythms, unfinished items, and key dependencies. The interaction layer can have a stable style, but that is policy configuration, not the capability itself. What determines value is whether the agent can retrieve relevant memory before you speak, turn vague intent into executable steps, and still align with progress after you have been away for a while. It may not be the strongest code generator, but it offers the highest return on “understands context and reduces repeated explanation.” Suitable examples include scheduling and document organization, cross-tool daily operations, and project follow-up that requires continuity of context.

Goal-driven autonomous researcher

You give it an acceptable goal—for example, summarizing public technical progress in a field over the past three years and producing a comparison table. It decomposes the work into retrieval, extraction, verification, backtracking, and writing, and loops through “observe—judge—act” until it meets the stopping condition or exhausts the budget. It usually does not carry personal preferences, and the next task starts planning from zero by default. Its advantage is persistence and coverage on a single task; the cost is high token and time consumption, and it can over-search on open-ended goals. It fits literature reviews, competitor information organization, first-draft proposals, and analysis that requires multiple rounds of self-checking. It does not fit decisions that require taste, accountability, or immediate interruption.

Coding collaborator embedded in the work environment

Its advantage is not broad knowledge, but the ability to read your current code, diagnostic output, and change scope. You describe intent in one sentence—for example, add timeout retries to a function and log failure reasons—and it should modify the code within the existing style, run checks, revise based on failures, and limit changes to relevant files. Because it is deeply coupled with the editing environment and command execution, it is efficient inside the repository. Once it leaves code, configuration, and tests, it should not be expected to handle ticket booking, approvals, or cross-department coordination. Clear boundaries make it safer.

Break Them Down with the Same Architecture Vocabulary

The three profiles can be compared within the same reference architecture. The main differences are where state lives, where the loop stops, and how wide the tools are opened.

Context and state

Short-term context is the current conversation, current files, and recent tool results. Long-term state is retrievable profiles, task boards, and artifacts. The companion profile treats long-term state as a first-class citizen. The researcher treats plans and intermediate evidence as state. The coding collaborator treats the workspace and diffs as state. During selection, ask: after a disconnect or a session change, what information must still be there, and what must be discarded to prevent contamination?

Planning and stopping

The researcher’s core is an explicit plan, reflection, and rollback. The coding collaborator’s planning is usually shorter and centered on “change—verify—fix.” The companion’s planning spans sessions and emphasizes decomposition and reminders. Any autonomous loop must have stopping conditions: acceptance criteria met, step or cost cap reached, consecutive steps with no new information, or a high-risk action that requires human confirmation. “Keep working” without a stopping condition merely transfers uncertain cost to the caller.

Tools and permissions

Tool calls turn natural language into controlled actions: reading files, retrieving knowledge, writing drafts, running tests, and sending notifications. Engineering practice should use least privilege: read-only by default; writes and external side effects approved in tiers; each tool with a clear input/output schema, timeout, and idempotency notes. High-risk actions—deletion, external sending, production configuration changes, moving funds, or personal data—must have a human in the loop. Do not give a reverse-engineering analyzer, a production console, and a personal account to the same agent at once in the name of “universality.” Automated reverse engineering, bulk probing, or bypassing access controls are not part of legitimate selection scope and should not be sold as capability.

Memory and personality

Memory should be auditable, correctable, and expirable. Preferences and project facts should be stored separately. Inferences should be marked as inferences, to avoid turning a one-time chat remark into a permanent rule. Personality is essentially a response strategy and refusal strategy. It can be constrained by system instructions, but it must not override safety boundaries. Users being able to view and delete long-term memory at any time is a prerequisite for trust.

Cost and quality

The cost of an autonomous loop is approximately “steps × per-step context length × retries.” The researcher is the most likely to overspend because it keeps putting retrieval results back into context. The coding collaborator also slows down if it repeatedly runs the full test suite. The companion’s cost is steadier, but bad memory causes continuous rework. Evaluate task completion rate, number of human handoffs, first-pass correctness, and cost per task—not just whether the demo feels fluent.

Observability

Each step should leave behind an input summary, selected tool, key parameters, result status, cost, and whether human confirmation was required. When something goes wrong, you need replay, not just a plausible-looking final answer. Logs themselves must be redacted so that keys and personal data are not written into traces.

A schematic controlled loop looks like this. It is not a product interface; it only shows where control points should sit:

state = load_scoped_memory(task)
for step in range(max_steps):
    if budget_exceeded(state) or goal_met(state):
        break
    action = plan(state)          # read, write, retrieve, or request human
    if action.risk == "high":
        action = wait_for_approval(action)
    result = run_with_timeout(action)
    state = reduce(state, result) # trim context, keep only evidence and diffs
persist_audit(state)

Decide by Scenario, Not by Ranking

Four questions can narrow the scope.

Is the task a one-off closed loop or a long-term relationship?

For one-off reviews and data verification, prioritize autonomous research capability and set strict budgets. For cross-week projects, repeated preferences, and multi-system follow-up, prioritize long-term memory and task boards. For modifications, tests, and commit messages in the current repository, prioritize coupling with the coding environment. Many roles actually need a combination: a coding agent changes code, a research agent prepares evidence, and a companion agent handles only progress and preferences—rather than letting one agent do everything.

What is the cost of a mistake?

Drafts and internal notes can allow greater autonomy. External publishing, production changes, privilege escalation, and private data access should shorten autonomous steps and force diff review and approval. Agents can prepare materials and run checks, but responsibility remains with humans.

Where does the context mainly live?

If key information is in conversations and profiles, strengthen memory retrieval. If key information is in repositories and workspaces, strengthen the editor, terminal, and diff view. If key information is in public materials, strengthen retrieval, citation, and deduplication. Connecting an agent to the wrong context source only amplifies hallucination.

Can you accept the result?

Do not delegate without acceptance criteria. A review needs sources and comparison dimensions. Code needs tests or static checks. Operational actions need a definition of done and a rollback path. Acceptance criteria are also stopping conditions.

A practical rollout order is: choose a low-risk, measurable scenario; expose only read-only or rollback-capable tools; record steps, success rate, and human edit volume; confirm that memory does not mix users or projects; then gradually add write permissions. For cloud deployment, compute, containers, object storage, message queues, logs, and monitoring/alerting are enough to support operation, artifacts, and audit. There is no need to bind to one vendor’s proprietary API narrative. If using a general-purpose cloud platform such as Baidu AI Cloud, design only by these capability categories, and treat keys, network isolation, and data retention periods as part of the architecture rather than a post-launch patch.

Common Pitfalls

Treating role-play as memory. A stable tone cannot replace a retrievable, correctable fact store.

Treating longer loops as smarter. More steps often just repeat retrieval. Use “stop when there is no new evidence” to cut the loop.

Taking the coding agent outside the repository. It lacks the permission model and responsibility boundaries of business systems, which makes misoperation likely.

Using demo videos instead of evaluation sets. Prepare fixed tasks, distractors, and failure examples. Check whether the agent admits uncertainty, oversteps authority, and stops within budget.

Finally, do not mistake a tool list for capability. The more tools there are, the larger the surface for prompt injection, misinvocation, and data leakage. Denying unknown tools by default and treating external content as untrusted is a common baseline for all agent types.

The selection map can be summarized in one sentence: use memory for continuous service, planning for open-ended research work, and local context for code-editing efficiency; then use permissions, budgets, audit, and human confirmation to bring these three capabilities into a deliverable scope. Teams can combine modules according to their own task duration, error cost, and context location, and make independent decisions instead of chasing constantly changing product names.