Embedding-Free Local Knowledge Bases: From Traditional RAG to Structured Retrieval

In enterprise knowledge bases, personal document assistants, and internal information retrieval, Retrieval-Augmented Generation, commonly known as RAG, has become a foundational architecture for large language model applications. RAG first retrieves relevant information from external sources and then passes that information to a language model for answer generation, reducing the risk of unsupported or fabricated responses.

However, traditional RAG is not finished when files are imported into a system. In real-world projects, the most time-consuming work is often document preprocessing, chunking strategy design, embedding generation, index maintenance, and retrieval tuning rather than model inference itself. Many teams spend weeks or even months on implementation and still end up with systems that can answer general questions but fail on important details.

In recent years, a lighter approach to local knowledge bases has gained attention. Instead of relying on a large model to parse every passage, forcing all content to be converted into vectors, or deploying specialized acceleration hardware, this approach uses original files, full-text indexes, keyword matching, entity recognition, and relationship expansion for retrieval.

This does not simply reject RAG. Rather, it revisits a basic assumption: must a knowledge base be converted into vectors before it can be searched?

1. Why Traditional RAG Often Becomes an Engineering Challenge

A typical RAG pipeline includes the following steps:

1. Read files.
2. Clean the text.
3. Split the text by length or semantic boundaries.
4. Call an embedding model to generate vectors.
5. Store the vectors in a database.
6. Receive a user query.
7. Convert the query into a vector.
8. Retrieve similar passages.
9. Rerank the results.
10. Send the context to a generative model.

A problem at any stage can affect the final answer.

1. Chunking Can Break Contextual Relationships

Consider the following information in an original document:

• Team A is responsible for Project X.
• The project budget is 5 million.
• The project was delayed because of supply chain changes.

If the document is split by a fixed character or token length, the responsible team, budget, and delay reason may end up in different chunks. Each chunk may appear relevant when retrieved independently, but the system may fail to recognize that they describe the same event.

Large chunks introduce irrelevant content and increase context length. Small chunks can break entities, timestamps, causal relationships, and hierarchical relationships. In many cases, inaccurate retrieval is not caused by insufficient model capability. The information may already have been damaged during preprocessing.

2. Vector Similarity Does Not Equal Factual Relevance

Vector retrieval is effective at finding semantically similar content, but semantic similarity and factual relevance are not identical.

For example, consider the question: “Who was responsible for the project after the second-quarter adjustment?” The system must understand several elements at once:

• The project name.
• The time range.
• The change implied by “after the adjustment.”
• The requested attribute: the person responsible.

If the knowledge base contains multiple projects, owners, and adjustment records, overall semantic similarity may retrieve text that sounds relevant but refers to the wrong period.

3. File Updates Create Cascading Maintenance Work

Traditional pipelines often require documents to be split again, re-embedded, and written back to the index. When a large file changes by only one page, the system may still process the entire document.

This creates several problems:

• Longer update times.
• Higher embedding costs.
• Potential inconsistencies when old and new indexes coexist.

For frequently updated policies, project documents, and operational records, maintenance can cost more than the initial knowledge base build.

2. The Core Idea Behind Embedding-Free Retrieval

Embedding-free retrieval does not mean that no processing is required. Instead, it shifts the focus from compressing all content into vectors to building explainable text indexes and relationship structures.

A relatively complete architecture can be divided into four layers.

2.1 Original File Layer

The system directly accepts text files, office documents, exported web content, logs, and structured records. Scanned documents and images require optical character recognition before they can be searched as text.

The key principle is to preserve both the original content and its source metadata, including:

• File name.
• Page or paragraph location.
• Last update time.
• Directory or category.
• Document version.
• Permission scope.

The more complete the source metadata, the more reliable citation, auditing, and incremental updates will be.

2.2 Full-Text Index Layer

A full-text index quickly locates passages containing keywords, phrases, numbers, or proper nouns. Common capabilities include:

• Tokenized search.
• Phrase matching.
• Prefix matching.
• Boolean conditions.
• Time filtering.
• Field weighting.
• Result highlighting.

Compared with vector retrieval, full-text search is fast, cost-effective, and easy to explain. When users search for an identifier, contract clause, employee name, or product specification, exact matching is often more reliable than semantic approximation.

2.3 Entity and Relationship Layer

To handle multi-hop questions, the system also needs to identify key entities, such as people, projects, organizations, dates, amounts, locations, and events, and then model the relationships between them.

For example, the following sentence:

Li is responsible for Project A. Project A underwent a budget adjustment in April, and its revised budget is 5 million.

can be represented as:

• Li — responsible for — Project A
• Project A — underwent a budget adjustment — April
• Project A — budget amount — 5 million

This does not necessarily require a complex knowledge graph. Lightweight relationship tables, inverted indexes, or event indexes can also support this type of retrieval.

2.4 Query Planning Layer

A user question is usually more than a simple keyword query. It often contains multiple conditions that must be resolved in sequence. A query planner can divide the question into several retrieval steps:

1. Identify entities in the question.
2. Determine the requested attributes.
3. Parse time ranges, scopes, and conditions.
4. Retrieve first-hop results.
5. Expand related entities based on those results.
6. Aggregate and rank the evidence.
7. Generate an answer with source references.

For example, for the question “Which people were responsible for the project and participated in the budget adjustment last year?”, the system can first locate the project, then find the budget adjustment event, and finally retrieve the people associated with that event. This is more reliable than depending on a single vector similarity score.

3. Why This Approach Can Be Faster

The main costs in a traditional knowledge base often come from model calls and vector computation. An embedding-free design relies more heavily on local text processing, index construction, and database queries.

For moderate data volumes, full-text indexing generally offers the following advantages:

• Building the index does not require calling a generative model for every passage.
• The system does not need to generate high-dimensional vectors for each chunk.
• File updates can process only changed content.
• Queries can use inverted indexes directly.
• Most preprocessing can be performed on CPUs.

“Faster” is not an absolute conclusion. Data volume, document formats, index implementation, query complexity, and hardware all affect performance. Large collections of unstructured content, highly complex semantic expressions, and multimodal data may still require vector or hybrid retrieval.

A more precise conclusion is that embedding-free retrieval can significantly reduce indexing and maintenance costs when the local knowledge base has clear document structures, stable entities, and queries containing recognizable keywords.

4. Accuracy Depends on More Than the Retrieval Algorithm

Teams often focus on which embedding model to use or how to set a similarity threshold. In practice, accuracy also depends on several other factors.

4.1 Is the Evidence Complete?

Some answers require multiple passages to establish a fact. The system should not return only the most similar passage. It should complete the evidence based on the entities and relationships in the question.

4.2 Are Facts Distinguished from Inferences?

“Person A is responsible for Project X” is a fact. “Person A may have participated in budget approval” may only be an inference. The system should indicate the type of evidence and avoid presenting speculation as a confirmed conclusion.

4.3 Are Source Locations Preserved?

Answers should ideally include the file name, section, page number, or paragraph location. This allows users to verify the result and makes retrieval errors easier to identify.

4.4 Are Time and Versions Handled Correctly?

The same policy may have multiple versions, and the same project may undergo several adjustments. Retrieval must account for effective dates, update times, and version priority. Otherwise, the system may cite outdated information.

5. Where Embedding-Free Knowledge Bases Are a Good Fit

Embedding-free local knowledge bases are particularly suitable when:

• Documents mainly consist of text and tables.
• Queries contain many proper nouns, identifiers, or exact values.
• The data volume is moderate and local deployment is important.
• Documents are updated frequently.
• Source traceability is a priority.
• The system should avoid high model-inference costs.
• The operating environment does not have specialized acceleration hardware.

Enterprise policy search, project archive retrieval, technical documentation search, contract clause lookup, and operations record analysis are all potential use cases for full-text indexing combined with relationship retrieval.

6. When It Should Not Fully Replace Vector Retrieval

This approach also has clear limitations.

If a user’s wording is highly conversational and shares few keywords with the original documents, pure full-text retrieval may fail to find the right content. This can happen when users rely on synonyms, metaphors, or conceptual descriptions while the documents use a different set of technical terms.

Text indexes alone also cannot fully interpret images, charts, audio, and video. Vector retrieval remains useful for cross-language search, complex semantic summarization, and open-ended knowledge discovery.

In many cases, the most practical engineering solution is hybrid retrieval:

• Use full-text indexes for exact terms, identifiers, and entities.
• Use vector retrieval for semantic similarity and alternative wording.
• Use rules or relationship queries for multi-hop paths.
• Use a reranking component to combine results from multiple sources.
• Use a generative model to organize the final answer.

7. How to Evaluate the System

A few subjective questions are not enough to evaluate a knowledge base. It is better to build a small test set covering the following query types:

1. Single-fact questions.
2. Multi-condition questions.
3. Multi-hop relationship questions.
4. Time and version queries.
5. Numeric and table-based queries.
6. Questions with no answer in the source material.
7. Disambiguation between similar entities.
8. Detection of outdated content.

Useful evaluation metrics include:

• Recall.
• Precision.
• Top-result hit rate.
• Evidence completeness.
• Answer traceability.
• Average response time.
• Cost per query.
• File update time.

“No-answer” detection is especially important. When the source material does not contain the requested fact, the system should clearly state that no supporting evidence was found rather than guessing in order to produce a complete-sounding answer.

8. A Practical Implementation Path

A stable implementation process can be divided into four stages.

First, establish original file management and full-text indexing so users can quickly locate source documents.

Second, add entity recognition, field extraction, and lightweight relationship tables. Prioritize frequently used entities such as people, projects, dates, and amounts.

Third, introduce query planning and multi-hop retrieval so complex questions can be decomposed into multiple verifiable steps.

Fourth, decide whether vector retrieval is necessary based on actual error samples rather than deploying a complete complex architecture from the beginning.

This incremental approach has two advantages. Initial costs remain lower, and every module can be validated against a test set. Teams are less likely to end up with a complicated system without knowing where its problems originate.

Conclusion

The core of a local knowledge base is not adding more models. It is ensuring that information can be found consistently, accurately, and transparently. Traditional RAG addresses the problem of language models being unable to access external information directly, but its chunking, embedding, and maintenance workflows also introduce significant engineering overhead.

An embedding-free approach based on full-text indexes, entity relationships, and query planning offers another path for local knowledge bases. In suitable scenarios, it can reduce indexing costs, shorten update cycles, and make retrieval results easier to explain.

The practical choice is not to decide that one technology must replace another. Instead, document types, query patterns, deployment constraints, and accuracy requirements should determine how exact retrieval, structured retrieval, vector retrieval, and generative models are combined into a knowledge base that is maintainable, measurable, and sustainable.