Search Capabilities for AI Agents: From Information Retrieval to Reliable Decisions

1. Background: Why Search Is the Bottleneck for Agents

A typical agent task follows this flow: understand the goal, break it into steps, call tools to gather information, and continue reasoning based on what it observes. Search is the main way an agent obtains external information, so the quality of search results sets the ceiling for everything that follows.

In practice, three failure scenarios appear often:

1. Corporate due diligence: The agent is asked for a company’s shareholding structure, but the results are usually just the company homepage or promotional profiles. Primary data, such as shareholder changes and capital contribution records, comes from registration authorities and regulatory filings. Ordinary web search rarely reaches these sources.
2. Engineering implementation: The agent is asked for a production-grade code solution, but the results are mostly beginner tutorials or code snippets. These often lack error handling, concurrency control, and performance limits.
3. Security analysis: The agent is asked to assess threat intelligence for a suspicious IP address, but the results are mostly general educational articles. They lack structured indicators, attack sample characteristics, and timeliness information.

The root cause of these problems is not the model’s reasoning ability. It is the quality of the input information. No matter how capable a model is, it cannot draw reliable conclusions from primary materials it never read. Search should therefore be treated as core infrastructure for an agent, not as an optional add-on.

2. Five Dimensions for Evaluating Search Quality

When you design or select a search capability, evaluate it on these five dimensions:

• Coverage: Can the search reach the types of information the task requires, such as registration records, source code, technical documentation, announcements, and datasets?
• Source authority: Do the results come from primary sources or secondary restatements? Official filings, standard documents, and original data are generally more authoritative than content that has been reposted through several layers.
• Structure: Do the results include parseable fields such as title, publishing organization, date, URL, and content summary? Structured fields make programmatic processing easier.
• Timeliness: Are publication or update dates visible? Can the system distinguish historical records from the current state?
• Traceability: Can each conclusion be linked to a specific source? This makes manual review and auditing possible.

These dimensions constrain each other. For example, broader coverage usually brings more noise. This is why filtering and ranking steps are needed after retrieval.

3. Layered Design of the Retrieval Pipeline

A stable search pipeline for agents typically uses multiple layers rather than sending the user’s question directly to a single search interface. A typical structure looks like this:

User task
  → Intent parsing (identify task type, entities, and time range)
  → Query planning (generate multiple complementary queries)
  → Multi-source retrieval (select data sources based on task type)
  → Deduplication and ranking (merge duplicate results, score by relevance and authority)
  → Content extraction (extract body text, publication date, author or organization)
  → Evidence organization (build a set of evidence with sources)
  → Reasoning and answer (generate conclusions from the evidence and cite sources)

The example below shows the basic structure of an evidence object and query planning. All functions are illustrative implementations.

from dataclasses import dataclass, field
@dataclass
class Evidence:
    url: str                 # Source URL, used for traceability
    title: str               # Title of the page or document
    snippet: str             # Body text excerpt relevant to the task
    source_type: str         # Source type, such as official document, code repository, news, or forum
    published_at: str | None # Publication or update time; None if it cannot be determined
    score: float = 0.0       # Overall score, filled in by the ranking module
@dataclass
class SearchTask:
    intent: str                          # Task type, such as company_research, code_impl, or threat_intel
    entities: list[str] = field(default_factory=list)
    time_range: str | None = None
def plan_queries(task: SearchTask) -> list[str]:
    """Generate complementary queries based on task type, rather than relying on a single keyword."""
    queries = []
    for entity in task.entities:
        if task.intent == "company_research":
            queries.append(f"{entity} 股东 变更 公开披露")
            queries.append(f"{entity} 年度报告 官方")
        elif task.intent == "code_impl":
            queries.append(f"{entity} 官方文档 最佳实践")
            queries.append(f"{entity} 生产环境 错误处理 示例")
        elif task.intent == "threat_intel":
            queries.append(f"{entity} 威胁情报 报告 时间线")
    return queries

The main benefit of this layered design is that each layer can be replaced and tested independently. For example, if you change the retrieval data source, you do not need to modify the ranking or evidence organization logic.

4. Tailoring Search Strategies to Task Types

The same search interface should use different retrieval strategies for different tasks.

4.1 Corporate Information Tasks

Prioritize primary materials such as public records from registration authorities, regulatory filings, and annual company reports. Content from a company’s website can help describe its business, but it should not be the only basis for its shareholding structure. For information that changes over time, record the disclosure date of each item. This prevents an outdated shareholding structure from being treated as the current state.

4.2 Engineering Implementation Tasks

Prioritize official documentation, release notes, standard specifications, and actively maintained open-source projects. To judge whether a piece of code is ready for production, check these signals: whether it includes exception handling, whether it describes concurrency and resource limits, whether it has test cases, and how long ago it was last updated. Beginner tutorials help with understanding concepts, but they should not be the basis for a production solution.

4.3 Security Analysis Tasks

Prioritize structured threat intelligence formats, vulnerability identifier databases, vendor security advisories, and incident response reports. Pay close attention to timeliness. The ownership and risk level of an IP address can change over time. For conclusions with unclear sources, mark them as pending verification rather than writing them directly into the analysis report.

5. Evaluating and Cross-Checking Results

Retrieved results need to be scored and verified before they enter the reasoning stage. One practical approach is to weight several dimensions:

def score_evidence(ev: Evidence, weights: dict[str, float]) -> float:
    """Compute the evidence score from relevance, authority, and freshness."""
    authority = {
        "official": 1.0,
        "academic": 0.9,
        "code_repo": 0.7,
        "news": 0.5,
        "forum": 0.3,
    }.get(ev.source_type, 0.2)
    freshness = 0.5 if ev.published_at is None else 1.0  # Example rule; in practice, decay the score based on the age of the publication
    relevance = keyword_overlap(ev.snippet, ev.title)     # Assumed relevance function, implemented elsewhere
    return (
        weights["relevance"] * relevance
        + weights["authority"] * authority
        + weights["freshness"] * freshness
    )

Adjust the weights for each task. Corporate due diligence should give more weight to authority. Code implementation tasks can give more weight to relevance and recent activity.

Beyond scoring individual results, cross-checking matters more. Key conclusions should be supported by at least two independent sources, and those sources should not cite each other. If several sources are reposts of the same material, treat them as a single source. When sources conflict, keep the conflicting information and identify where each claim comes from. Do not pick one side at random.

6. Key Engineering Considerations

6.1 Caching and Rate Control

Repeated queries within the same task should use a cache. Cache keys should include the query string, the data source, and the time range, and they should have a reasonable expiration time. Time-sensitive categories, such as security intelligence, should use shorter expiration times. Calls to external services need rate limiting and exponential backoff to avoid being throttled under high concurrency.

6.2 Timeouts and Graceful Degradation

When a single data source times out or fails, the pipeline should continue with the other sources and note which sources are missing in the output. A single failed source should not stop the entire task. Failures should also not be ignored silently, because users would then assume the information is complete.

6.3 Security Boundaries

Web page text, document content, and search snippets are all untrusted data. They may contain instructions meant to change the agent’s behavior, such as orders to ignore existing rules or to reveal context. Treat external content strictly as data. Extract facts from it, and never execute instructions it contains.

Search queries can also leak information. For tasks involving internal project names, customer names, or unpublished data, remove sensitive details before sending a query. Include only the minimum information needed to perform the search.

6.4 Traceability and Auditing

Every piece of information that enters a final conclusion should be stored with its source URL, retrieval time, publication date, and original excerpt. These records make manual review easier and help locate the cause when a conclusion turns out to be wrong. Link retrieval logs to reasoning logs to form a complete chain of evidence.

7. Common Pitfalls

• Treating the number of results as quality: Returning dozens of results does not mean the information is sufficient. What matters is whether primary sources are included.
• Trusting snippets directly: Search snippets are truncated and rewritten, and they may drop important conditions. For figures, dates, and legal status, check the original text.
• Ignoring the time dimension: Content without a visible publication date should not be treated as current fact.
• Expecting a single search to be complete: Complex tasks usually need several rounds of retrieval. Start with an overview, then add queries around the points of uncertainty. This usually works better than building one very long query.
• Missing failure handling: If the system does not handle empty results, timeouts, and malformed responses, the agent may fabricate answers when data is insufficient.

Search capability determines the range of facts an agent can access. When designing it, prioritize reliable sources, clear structure, and traceable processes. Recall speed and breadth of coverage come after those.