Vector Search Is No Longer the Default: How Code Agents Are Rethinking Retrieval

1. The Often-Misread SWE-bench Baseline

Many people cite early SWE-bench numbers as if they were results from vector RAG. That needs clarification first. According to the original 2023 SWE-bench paper, its reference baseline used lexical retrieval such as BM25: first find files related to the issue, then pass snippets to the model to generate a patch. The paper reported a resolution rate of about 1.96% under a specific model and setup, and citations should be checked against the original paper.

Later, SWE-agent, which centers on tool calling, reported a result of about 12.47%. Its difference lies in the retrieval approach: the model no longer receives a batch of snippets all at once. Instead, it opens files, searches symbols, and scrolls through content itself. Public leaderboard results have since become noticeably higher, but specific values vary by leaderboard version and model, so the leaderboard page should be treated as the source of truth.

This comparison shows that letting the model actively acquire information is usually more effective than passively receiving a batch of snippets. It does not by itself prove that vector retrieval is necessarily worse than keyword retrieval, because the models, retrieval settings, and evaluation environments differ across these experiments.

2. Semantic Similarity Is Not the Same as Code Relevance

The core assumption of vector retrieval is that text snippets with similar semantics are also similarly relevant to the task. This assumption largely holds in open-domain question answering and general document retrieval. In code repositories, it often breaks down.

Code has explicit structural relationships, including import dependencies, type definitions, and function call chains. When changing a function’s return type, what really needs to be found is all callers, not other functions with similarly written comments. Embeddings flatten these structural relationships. Semantically similar code may sit close together in vector space, but there is no reliable correspondence between “semantically similar” and “will cause a compilation error if changed.”

Take “Where is processPayment defined?” as an example. The user needs the exact definition location. Vector search results may include handlePayment or processPaymentHistory, while the actual definition may be pushed outside the top-k because its comment style differs. False positives waste triage time, while false negatives directly lead to missed changes.

It should be noted that this section discusses precise symbol localization and cross-file impact analysis. In open-domain semantic question answering, vector recall remains valuable.

3. Identifiers Are High-Precision Retrieval Keys

A key difference between code repositories and ordinary document collections is that identifiers are unique. Function names, class names, variable names, and file paths form a natural index.

Finding symbol definitions, listing all references to a variable, and confirming the callers of an interface are all exact-match tasks. grep and rg have long been used by developers precisely because their results are clear: a hit is a hit, and a miss is a miss.

By contrast, similarity-ranked results depend on the embedding model, distance metric, and cutoff threshold. The same query may produce a different candidate set after the model is changed. For code operations that need to be reproducible and verifiable, this uncertainty increases triage cost.

4. Index Maintenance Is an Ongoing Cost

Code repositories change every day. Every commit and merge invalidates part of the vector index. Functions are added and renamed, files are moved, and the drift between the index and the code continues to accumulate.

Teams usually rely on incremental updates or periodic full rebuilds to maintain effectiveness. Incremental updates can reduce overhead, but they cannot fully eliminate invalidation. When the embedding model is changed, the entire repository needs to be re-vectorized, and this cost is significant in large repositories.

This type of cost is continuous. It is not a one-time construction investment, but a long-term drain on compute and maintenance effort. In large repositories with frequent changes, maintenance cost may exceed the recall benefit. Teams should measure their own repositories rather than judging from experience. They can track metrics such as the number of files changed per week, reindexing time, model change frequency, and the retrieval failure rate.

5. A Minimal Tool-Driven Retrieval Workflow

Take a typical issue: “Order amount is not updated after a refund.” The agent’s operations could look like this:

1. Use grep to search for “refund” and find files related to the refund service.
2. Open the file and search for function names related to amount updates, such as updateOrderAmount.
3. Find all callers of that function and confirm which paths trigger a refund.
4. Read the corresponding test files to determine expected behavior.
5. Modify the code, run tests, and continue locating based on failure messages.

Each step depends on the result of the previous step and determines the next search term. This differs from traditional RAG. RAG places the most similar snippets into the prompt all at once, and retrieval and generation are two separate stages. Tool-driven retrieval makes retrieval part of reasoning, which can be called retrieval-augmented reasoning. The model decides when to search, what to search for, and how much to read, and each information acquisition has a clear purpose.

6. How to Choose Layered Retrieval

In practice, a reasonable approach is to layer retrieval by need and trigger it in order: when the previous layer is sufficient, do not enter the next layer.

Layer	Suitable need	Typical tools	Main advantages	Main risks
Keyword layer	Symbol definitions, reference lookup, path lookup	grep, rg, filename search	Precise, reproducible, low cost	Easy to miss when naming is inconsistent
Structural layer	Cross-file impact analysis	Language services, dependency parsing, call graph traversal	Uses real code relationships	Requires parsing capability for each language
Semantic layer	Code with similar functionality but different naming	Vector index	Can find implementations with different vocabulary	More false positives, and the index requires ongoing maintenance

The keyword layer covers most deterministic lookup needs. The structural layer handles questions that require understanding code relationships. Only when the first two layers cannot locate the target and missed recall has been recorded as an explicit problem is it worth introducing the semantic layer as a fallback.

7. Recommendations for Teams

• Establish a baseline with keyword and structural tools first, and measure fix success rates using historical issues from your own repository.
• Introduce a vector layer only when the baseline shows clear missed recall, and record concrete examples of those misses.
• Include the cost of index updates, model changes, and re-vectorization in maintenance accounting, and compare it with recall benefits.
• Provide agents with stable atomic tools, including search, read, edit, and test execution, and let them decide the retrieval path.

Vector databases have not become obsolete. Their position has shifted from default option to fallback. What teams really need to decide is how to design the toolset so that retrieval behavior works in concert with the model’s reasoning capabilities.