A Practical Guide to LLM Hallucination Mitigation: From Evaluation to Engineering

5 hr ago0 views

Large language models often generate plausible but factually incorrect content, known as hallucination, which remains a primary barrier to enterprise deployment. This article explores a practical framework for mitigating LLM hallucinations, covering common hallucination types, how to establish an evaluation baseline with core metrics and automated workflows, and key engineering techniques such as Retrieval-Augmented Generation (RAG), structured output, and evidence verification. It also provides strategies for teaching models to express uncertainty and setting up continuous post-launch monitoring, complete with a deployment checklist.

1. Why Hallucination is the Primary Obstacle to Deployment

Large language models generate text by predicting the next most probable token. While this mechanism excels at producing fluent and contextually appropriate expressions, it does not naturally guarantee factual consistency. When faced with missing or ambiguous information in its training data, the model may still provide a seemingly complete answer—a phenomenon commonly referred to as hallucination.

For business systems, the cost of hallucination depends on the context. In casual chat or creative writing, occasional deviations might be acceptable. However, in scenarios like customer service, medical consultation assistance, legal retrieval, or financial report interpretation, a single incorrect date or amount can result in actual losses and erode user trust in the entire system. Therefore, reducing hallucination is often a prerequisite for enterprise deployment, not an optional optimization.

2. Common Types of Hallucinations

Based on their manifestation, hallucinations can be roughly categorized into four types:

Factual Fabrication: Providing non-existent details about specific people, organizations, or events.
Source Forgery: Citing non-existent literature, links, or clause numbers.
Numerical and Date Deviation: Errors in calculation results, version numbers, or time nodes.
Overconfidence: Answering questions with absolute certainty even when the model lacks confidence.

Categorizing hallucinations helps in selecting appropriate mitigation strategies. Factual fabrication and source forgery primarily rely on retrieval and citation verification; numerical errors can be handed off to deterministic programs; overconfidence requires prompt design and output-level guidance to encourage the model to express uncertainty.

3. Establish an Evaluation Baseline First

Without measurement, it is impossible to determine if improvements are effective. Before diving into optimization, teams should build an evaluation set relevant to their business.

3.1 Principles for Building an Evaluation Set

Questions should originate from real business logs, covering both high-frequency and high-risk issues.
Each question should be paired with a standard answer and a traceable reference basis.
Retain a certain proportion of unanswerable questions to test the model’s refusal capability.
Update regularly to prevent high overlap between the evaluation set and the knowledge base or training corpus, which could inflate metrics.

3.2 Core Metrics

Factual Accuracy: The proportion of answers consistent with the reference basis.
Traceability Rate: Whether key conclusions in the answer can be mapped to specific evidence fragments.
Refusal Accuracy: The proportion of correct “I don’t know” responses to unanswerable questions.
Hallucination Rate: The proportion of answers containing factual errors out of all answers.

For example, in a 100-question evaluation set, 80 questions have standard answers and 20 are unanswerable. If the model answers 72 correctly and correctly refuses 12, the factual accuracy is 72/80 (90%), and the refusal accuracy is 12/20 (60%). Tracking these two metrics separately clearly reflects the model’s performance in both “knowing how to answer” and “not answering blindly”.

3.3 Automated Evaluation Process Flow

Below is pseudo-code for a general process, independent of specific frameworks:

for case in eval_set:
    answer, evidence = system.answer(case.question)
    if case.is_unanswerable:
        record.refusal_correct += is_refusal(answer)
    else:
        record.correct += judge_consistent(answer, case.reference)
        record.traceable += has_supporting_evidence(answer, evidence)
report = aggregate(record)

Here, judge_consistent can be accomplished through rule matching, human annotation, or a calibrated judge model. The key is to maintain consistent evaluation criteria throughout the iteration process; otherwise, metrics across different stages cannot be compared.

4. Engineering Methods to Reduce Hallucination

4.1 Retrieval-Augmented Generation (RAG)

The core idea of RAG is to retrieve relevant fragments from a trusted knowledge base before answering and provide these fragments to the model as context. The model’s task shifts from “answering from memory” to “answering based on provided materials,” significantly reducing the probability of factual fabrication.

When deploying, note the following:

Knowledge base chunking must be reasonable. Chunks that are too large will introduce irrelevant information, while chunks that are too small will lose context.
Retrieval quality dictates the upper limit of overall performance; recall and ranking quality should be evaluated independently.
If no relevant content is retrieved, the system should inform the user directly rather than forcing an answer.

4.2 Constraining Answer Scope

Explicitly specify in the system prompt: only answer based on the provided materials; if information is missing from the materials, state “cannot be confirmed from the available data.” Such constraints are simple and effective but require validation through evaluation, as the model’s adherence to instructions varies with task complexity.

4.3 Structured Output and Deterministic Computation

For computable fields like dates, amounts, and quantities, require the model to output structured results, and let a program handle calculations and format validation. For instance, have the model only extract the “start date” and “number of days,” and use a date library for addition/subtraction, preventing the model from making mental calculation errors in text.

Example:

extracted = model.extract(text, schema={"start_date": "date", "days": "int"})
end_date = add_days(extracted.start_date, extracted.days)
validate(end_date >= extracted.start_date)

4.4 Citation and Evidence Verification

Require the model to append evidence identifiers after each key conclusion, and have the system verify whether the original text corresponding to that identifier actually supports the conclusion. If verification fails, it can trigger regeneration or downgrade to a refusal. This step effectively intercepts source forgery issues and facilitates manual review.

4.5 Multi-Sampling Consistency Check

Sample the same question multiple times. If the core facts of multiple answers contradict each other, it indicates the model is unstable on that question and should be flagged as low confidence. This method increases API call costs and is suitable for high-risk questions or offline review scenarios.

4.6 Using Tools to Replace Memory

For dynamic content like real-time data, account information, and inventory status, do not rely on model memory. Instead, query via an API first, then organize the language output. The model is responsible for understanding user needs and articulating results, while the factual source is left to the data system.

5. Teaching the Model to Express Uncertainty

Mitigating hallucination is not just about reducing errors, but also making errors identifiable. In practice, the following approaches can be adopted:

Specify confidence expression methods in the prompt, such as distinguishing between “Confirmed,” “Limited Evidence,” and “Cannot Confirm.”
Provide visual cues for low-confidence answers in the UI, or append verification suggestions.
Design clear scripts for refusals and provide an entry point for human handover or knowledge base queries.

The benefit of this approach is that even if the model makes a mistake, the user has the opportunity to detect it in time, keeping the overall system reliability under control.

6. Continuous Monitoring Post-Launch

An evaluation set can only cover limited scenarios; live traffic is the real test. It is recommended to establish the following mechanisms:

Sampling Audit: Extract live conversations at a fixed ratio and have business experts annotate factual correctness.
User Feedback: Provide an “Incorrect” button next to the answer and record the reasons for feedback.
Metrics Dashboard: Track trends in hallucination rate, refusal rate, and user correction rate.
Alert Thresholds: Automatically notify the responsible person and suspend related features when the error rate for a specific type of question exceeds a preset threshold.
Closed-Loop Iteration: Feed erroneous samples back into the evaluation set, and simultaneously correct knowledge base content or adjust prompt strategies.

The focus of monitoring should be on trends, not single results. A single-day fluctuation might be due to changes in traffic structure, but a continuous rise over several weeks warrants attention.

7. Deployment Checklist

Have you established an evaluation set covering high-frequency and high-risk questions, including unanswerable samples?
Are factual accuracy and refusal accuracy tracked separately?
Do key conclusions have traceable evidence?
Are computable contents like dates and amounts calculated by a program?
Are there live sampling audits and a user correction entry point?
Are alert thresholds and emergency downgrade plans defined?