Gen AI

Hallucination Mitigation

Because LLMs predict what sounds plausible rather than what is factually true, developers use grounding, retrieval, and verification pipelines to force the model to stick to reality.

An LLM generates a response based on retrieved facts. Before the user sees it, a smaller verification model checks the output against the source documents, blocking the response if an unsupported fact is detected.

Why Does This Exist?

By their nature, Large Language Models do not "know" facts; they know statistical correlations between words. When a model doesn't have enough information to answer a question, its training still pushes it to generate a fluent, confident-sounding sentence. The result is a hallucination—a statement that sounds completely plausible but is factually incorrect.

If you are using an LLM to write a fantasy novel, hallucinations are a feature (creativity). But if you are building an AI paralegal, a medical assistant, or a financial advisor, a hallucination is a catastrophic bug. Because the underlying architecture of an LLM cannot be "fixed" to stop hallucinating entirely, engineers must build systems around the model to detect and mitigate them.

Think of It Like This

A fast-talking salesperson and a meticulous fact-checker

Imagine a salesperson who is incredibly charismatic but has a bad habit of inventing features to close a deal. You can't change their personality, but you can change their workflow.

First, you give them a strict script and say, "You can only talk about the features written on this page" (Grounding/RAG).

Second, you put a meticulous fact-checker on the phone with them. Before the salesperson is allowed to hit "send" on an email to a client, the fact-checker reads it, compares it to the script, and deletes any claims the salesperson invented (Verification).

Hallucination mitigation is building the script and the fact-checker, acknowledging that the salesperson (the LLM) will always be prone to making things up.

How It Actually Works

Mitigation strategies fall into three phases: what happens before generation, during generation, and after generation.

1. Before Generation (Grounding & Prompting)

The easiest way to stop a model from guessing is to hand it the answer.

Retrieval-Augmented Generation (RAG): Instead of relying on the model's parametric memory (its weights), you retrieve relevant documents and insert them into the prompt.
Strict Prompting: Explicitly telling the model what to do if it doesn't know the answer. For example: "Answer ONLY using the provided documents. If the answer is not in the documents, reply exactly with 'I do not have enough information'."

2. During Generation (Parameters & Sampling)

You can tweak how the model selects its words.

Low Temperature: Setting the generation temperature to 0 or 0.1 forces the model to pick the most mathematically probable next token, reducing randomness and "creativity," which in turn reduces hallucinations.
Chain of Thought: Asking the model to "think step by step" before outputting the final answer. By forcing the model to write out its intermediate logic, it is less likely to jump to a statistically common but factually wrong conclusion.

3. After Generation (Verification & Guardrails)

You assume the model hallucinated, and you verify the output before showing it to the user.

Self-Correction (Self-RAG): Asking a model to review its own output and cite the specific sentences in the source document that support its claims. If it can't find a citation, it flags the claim as a hallucination.
Output Guardrails: Passing the generated text through a secondary, smaller model (like an NLI - Natural Language Inference model) whose only job is to check if Sentence B (the output) is logically supported by Document A (the source). If not, the system blocks the response or triggers a retry.

Show Me the Code

1# Mitigating hallucinations using Grounding and an Output Guardrail2def answer_question_safely(question: str, source_docs: str) -> str:3    # 1. Grounding Prompt (Before Generation)4    prompt = f"""5    Answer the question ONLY using the facts in the Source Documents.6    If the documents do not contain the answer, say "I don't know."7    8    Source Documents: {source_docs}9    Question: {question}10    """11    12    # 2. Low Temperature (During Generation)13    draft_answer = llm.generate(prompt, temperature=0.0)14    15    # 3. Output Guardrail (After Generation)16    # Use a small NLI model to verify the draft is supported by the source17    if not is_supported_by_source(claim=draft_answer, source=source_docs):18        return "I'm sorry, I cannot confidently answer that based on my sources."19        20    return draft_answer

Watch Out For

Prompting away the problem doesn't scale

You cannot fix hallucinations purely by adding "Do not hallucinate" to the system prompt. LLMs do not have an internal lie-detector; they don't know when they are making things up. Relying solely on prompts without external RAG grounding or output verification will eventually fail.

The 'I don't know' loop

If your hallucination mitigation is too strict (e.g., your verification model is overly sensitive), the system will constantly reject valid answers. Users will get frustrated if the AI replies "I don't know" to perfectly reasonable questions. Tuning the threshold of your verification guardrails is a delicate balance between safety and utility.

The Quick Version

Hallucinations are a fundamental reality of LLM architecture; they cannot be completely eliminated, only mitigated by external systems.
Grounding (RAG) provides the model with hard facts so it doesn't have to guess.
Low temperature and Chain of Thought prompting force the model to be more deterministic and logical during generation.
Output Guardrails act as automated fact-checkers, verifying the LLM's claims against the source documents before the user sees them.
Effective mitigation requires a layered approach across the entire generation lifecycle.