Skip to content
AI360XpertBeta

Reduce LLM Hallucinations

LLMs guess the most plausible next word, so you ground answers in retrieved passages and check each claim against those passages before the user sees it.

A claim-level checker scores each drafted claim against the retrieved sources at threshold 0.80, passing two claims at 1.00 and holding the answer over one unsupported claim at 0.57
A claim-level checker scores each drafted claim against the retrieved sources at threshold 0.80, passing two claims at 1.00 and holding the answer over one unsupported claim at 0.57

Why Does This Exist?

An LLM answers from patterns in its weights, not from a fact table it can point at. Ask about something thin in its training and it still hands you the most plausible-sounding sentence, with perfect grammar and total confidence. Sometimes that sentence is wrong. Why the model behaves this way is covered in why LLMs hallucinate; this page is about what you build around the model once you accept it.

The stakes decide whether you need any of this. A fantasy-name generator can invent freely. A support bot that invents refund policies, a summarizer that invents dosages, or a coding assistant that invents API flags cannot. You can't retrain the tendency away, so you wrap the model in a pipeline instead: ground it in documents, then check what it wrote before anyone reads it.

Think of It Like This

A confident tour guide and a receipt checker

You hire a tour guide who is charming, fast, and occasionally invents history. You can't change the guide, so you change the job. You hand them an approved fact sheet with one rule: say only what is on the sheet, and say "I don't know" otherwise. Then you station a checker at the exit who compares every line of the tour slip against the sheet, receipt by receipt, and holds the slip if one line has no match.

The guide is the LLM, the fact sheet is your retrieved passages, and the exit checker is the verification step. Two claims match and pass. The third adds a detail the sheet never mentions, so the slip stays at the desk.

Where it stops: a real receipt checker matches exact items, while your checker compares meanings, and meaning comparison is itself approximate. A paraphrased truth can fail the check and a smoothly reworded invention can pass it. That is why the threshold you set matters far more than the metaphor suggests.

How It Actually Works

Mitigation runs in three places: what you put into the model, how you let it generate, and what you check afterward. The layers stack. None of them holds alone.

Ground the prompt in retrieved passages

Instead of asking from memory, you fetch documents first and paste the best passages into the prompt, with an instruction to answer only from them plus an explicit abstention line such as "reply I do not have enough information." This is the RAG pattern: retrieval turns an open-memory question into a reading-comprehension question, which models handle far better.

The assumption is blunt. Your sources are correct and actually contain the answer. When they are stale or miss the question, grounding hands the model confident-sounding wrong material, and no downstream check can rescue an answer whose evidence was wrong to begin with.

Generate conservatively

Two decoding choices lower the invention rate. Low temperature, near 0, makes the model pick the most probable next token instead of sampling widely, which cuts creative detours. Step-by-step reasoning before the final answer forces intermediate claims into the open where they can be checked. Neither makes output true; they make it steadier. Skip both when you want variety, such as brainstorming, where invention is the product rather than the bug.

Check each claim, then decide

The strongest layer treats the draft as guilty until proven innocent. Split the draft into claims, score each one against the sources, and hold or retry anything below threshold. This is the same instinct behind output guardrails and claim-level citations, applied as a numeric test.

For one claim cc, let K(c)K(c) be its content keywords: names, numbers, and fact words, lowercased, with filler words dropped. Let K(S)K(S) be the keyword set of the retrieved source passages. The support score is:

s(c)=∣K(c)∩K(S)∣∣K(c)∣s(c) = \frac{|K(c) \cap K(S)|}{|K(c)|}

In plain words: the fraction of the claim's keywords that appear anywhere in the sources. ∣X∣|X| means the count of items in a set, and ∩\cap means the overlap between two sets. The decision rule uses a threshold τ\tau, say τ=0.80\tau = 0.80: the claim passes when s(c)≥τs(c) \ge \tau, and anything below is flagged for retry or refusal. Each layer costs something, so name the trade: retrieval adds lookup latency, reasoning tokens cost money, and the checker spends a second pass over every answer. You pay all three to buy factuality where it matters.

Worked example

The sources SS read: "Revenue grew 10 percent in 2024. The Berlin plant opened in March. The CEO is Ana Ruiz." Their keyword set K(S)K(S) holds 12 items: revenue, grew, 10, percent, 2024, berlin, plant, opened, march, ceo, ana, ruiz.

The draft makes three claims:

  • c1c_1: "Revenue grew 10 percent in 2024." K(c1)K(c_1) has 5 keywords and all 5 sit in K(S)K(S), so s(c1)=5/5=1.00s(c_1) = 5/5 = 1.00. Passes.
  • c2c_2: "The Berlin plant opened in March and employs 500 people." K(c2)K(c_2) has 7 keywords: berlin, plant, opened, march, employs, 500, people. Four appear in K(S)K(S); three (employs, 500, people) appear nowhere. So s(c2)=4/7≈0.57s(c_2) = 4/7 \approx 0.57, below 0.800.80. Flagged.
  • c3c_3: "The CEO is Ana Ruiz." K(c3)K(c_3) has 3 keywords, all present, so s(c3)=3/3=1.00s(c_3) = 3/3 = 1.00. Passes.

Two of three claims hold, so answer-level support is 2/3≈0.672/3 \approx 0.67. The pipeline holds the answer: retry the second claim against more passages or refuse that part. That is exactly the diagram above. Two passes, one held slip.

Code

The snippet below is the check itself, not the formula retyped: split nothing, fetch nothing, just score three claims against a source set the way the worked example did by hand.

def support_score(claim_words: list[str], source_words: set[str]) -> float:    """Fraction of the claim's keywords found anywhere in the sources."""    overlap = sum(1 for w in claim_words if w in source_words)    return overlap / len(claim_words)

def check_claims(    claims: dict[str, list[str]], source_words: set[str], threshold: float = 0.80) -> dict[str, bool]:    """Map each claim id to True when its support reaches the threshold."""    return {        cid: support_score(words, source_words) >= threshold        for cid, words in claims.items()    }

source = {    "revenue", "grew", "10", "percent", "2024", "berlin", "plant",    "opened", "march", "ceo", "ana", "ruiz",}claims = {    "c1": ["revenue", "grew", "10", "percent", "2024"],    "c2": ["berlin", "plant", "opened", "march", "employs", "500", "people"],    "c3": ["ceo", "ana", "ruiz"],}
print(round(support_score(claims["c2"], source), 2))  # -> 0.57print(check_claims(claims, source))  # -> {'c1': True, 'c2': False, 'c3': True}

The flagged claim scores 4/7≈0.574/7 \approx 0.57 and fails the 0.800.80 bar, matching the hand computation. Real checkers normalize synonyms and numbers before scoring; this one matches exact words, which is why the second pitfall below bites.

Watch Out For

Grounded in the wrong documents

Scores of 1.00 across the board, yet users report wrong answers. The checker compares the draft to the sources, so when retrieval fetches stale or off-topic passages, every claim faithfully matches bad evidence. You'll tune the checker for days while the failure sits one stage earlier, in retrieval.

Fix it there. Sample real questions, read the top passages yourself, and confirm each answer is actually in them before trusting any score. If coverage is thin, repair chunking and ranking first. The checker only works when the evidence does.

A threshold that refuses everything

After verification ships, the bot answers "I do not have enough information" to questions it handled fine before, and users stop asking. A strict threshold plus paraphrase mismatch turns good answers into refusals: the draft says "grew by a tenth" while the source says "grew 10 percent," overlap drops under τ\tau, and a true claim gets flagged.

Start near 0.80, measure the refusal rate on a fixed question set, and move the threshold only while watching misses and false blocks together. Normalize numbers and synonyms before scoring, and route held claims to a retry with more passages before you ever show a refusal.

The Quick Version

  • Hallucination is the model guessing plausibly from memory, so mitigation wraps the model in grounding plus checks rather than retraining the tendency away.
  • Retrieval turns open-memory questions into reading comprehension: paste the passages in, instruct the model to use only them, and give it an abstention line.
  • Low temperature and step-by-step reasoning steady generation, but neither makes output true on its own.
  • Score every claim as the fraction of its keywords found in the sources, and hold anything under your threshold: in the example, 1.00 and 1.00 pass while 0.57 gets held.
  • Tune retrieval and the threshold together, since wrong sources make high scores meaningless and a strict cutoff turns paraphrased truths into refusals.