Skip to content
AI360XpertBeta

Corrective RAG (CRAG)

Corrective RAG checks what your search brought back before it answers. When every passage looks weak, it asks the live web instead of letting the model guess.

Corrective RAG grades three retrieved passages as 0.82, 0.45 and 0.11 and routes the mixed batch to combine kept passages with live web search
Corrective RAG grades three retrieved passages as 0.82, 0.45 and 0.11 and routes the mixed batch to combine kept passages with live web search

Why Does This Exist?

You ask about last night's final and your index was built last week. Your vector search still returns three passages, because returning something is its job. The model then does its job too well: it stitches those near misses into a smooth answer that sounds right and is not.

That's the failure this page fixes. A plain RAG pipeline has no checkpoint between retrieval and writing, so weak evidence flows straight into the prompt. Self-RAG added a checkpoint that says no when the evidence is poor. Safe. But you've still got no answer when the answer exists somewhere else.

Corrective RAG keeps the checkpoint and adds a repair. A small retrieval evaluator grades each passage before generation. Strong batches go forward. Weak batches get rerouted to live web search. Mixed batches get combined. You pay extra latency and an extra model call, and in return you stop guessing from stale clips.

This page covers that grade and route step plus the strip down that follows. The base pipeline lives in RAG architecture. How to score retrieval on its own lives in RAG evaluation. The wider debugging order lives in RAG failure diagnosis.

The one idea you need first is the RAG contract. You index your docs once, ahead of time. At query time you embed the question, pull the closest chunks, paste them into the prompt, and generate. If the right chunk was never indexed, retrieval can't find it. No prompt trick brings it back. That missing chunk case is where correction earns its keep.

Think of It Like This

A clippings desk with a phone

You run a small news desk. An editor asks when the women's 100m final starts. Your assistant pulls three clippings from the morgue: tonight's start list, a preview from last month, and last year's marathon results.

You don't hand all three to the writer. You grade them. The start list answers the question. The preview is background at best. The marathon recap is the wrong race in the wrong year, so it goes in the bin. Because you've got one solid clipping and one gap, you keep the start list and phone a reporter on site for the missing detail. The writer gets two short strips, not a folder of maybe.

Where it stops: you understand races and dates. The evaluator doesn't. It scores surface fit between question and passage, and shared names or a confident tone can push a wrong passage over the line. And your phone call costs time and can reach a crank. Web search adds fresh evidence and fresh ways to be wrong.

How It Actually Works

CRAG inserts one decision between retrieval and writing. Retrieve as usual, grade what came back, then let the grades pick the context. The picture above shows the same three scores the example below computes.

Retrieve, then grade

You start with qq, your query text. The retriever returns d1,d2,…,dnd_1, d_2, \ldots, d_n, the top nn passages it found, with nn small (three in the example). The evaluator then reads each pair (q,di)(q, d_i) and returns sis_i, a relevance score in [0,1][0, 1] where larger means more likely to answer qq. Think of sis_i as the checker's confidence, not the retriever's distance.

You also set two cutoffs ahead of time from a labeled eval set: τhi\tau_{hi}, the high cutoff above which a passage counts as Correct, and τlo\tau_{lo}, the low cutoff below which it counts as Incorrect. Anything between them counts as Ambiguous. In the example τhi=0.70\tau_{hi} = 0.70 and τlo=0.30\tau_{lo} = 0.30. Those numbers are choices you tune, not constants from a paper.

Route on the grades

Each passage gets one label:

  • si≥τhis_i \ge \tau_{hi} means Correct, so keep it.
  • si<τlos_i < \tau_{lo} means Incorrect, so drop it.
  • otherwise Ambiguous, so hold it for now.

Then the batch takes one of three paths. All Correct means refine only. All Incorrect means throw the batch away and run web search only. Anything mixed means Ambiguous overall: keep the Correct strips, drop the Incorrect ones, run a rewritten web query to cover the gap, and combine both sources. If one search round is not enough and you want the model to keep searching, that loop lives in Agentic RAG.

Refine and generate

Knowledge refinement is the strip down. A kept passage often arrives as five chatty sentences with one useful fact. A small model extracts the one or two sentences that bear on qq and drops the rest. That shorter strip is what the final model sees, plus any web passages, each cited so you can check what came from where.

The trade-off is concrete. You add an evaluator call on every query plus a web call on weak and mixed batches, so your p95 latency rises and you now depend on a search API. What you buy is fewer confident wrong answers from stale indexes. Skip the whole thing when your corpus is stable, self contained, and latency matters more than freshness, like an HR handbook bot behind a firewall. It assumes the evaluator's scores track usefulness and the web holds a better answer. When either fails you get the worst of both: good local passages dropped or web junk pasted next to them.

Worked example

Ask q=q = "When is the women's 100m final at the current championships?" The retriever returns three passages:

  • d1d_1: schedule page naming the women's 100m final with day and time, s1=0.82s_1 = 0.82.
  • d2d_2: championships preview mentioning the 100m but giving no time, s2=0.45s_2 = 0.45.
  • d3d_3: last year's marathon recap, s3=0.11s_3 = 0.11.

Grade them against τhi=0.70\tau_{hi} = 0.70 and τlo=0.30\tau_{lo} = 0.30:

s1=0.82≥0.70→Correct,0.30≤s2=0.45<0.70→Ambiguous,s3=0.11<0.30→Incorrects_1 = 0.82 \ge 0.70 \rightarrow \text{Correct}, \quad 0.30 \le s_2 = 0.45 < 0.70 \rightarrow \text{Ambiguous}, \quad s_3 = 0.11 < 0.30 \rightarrow \text{Incorrect}

The batch is mixed, so the route is combine. Drop d3d_3. Refine d1d_1 from five sentences to the two that state the event and time, discarding three. Rewrite qq for the web ("women's 100m final start time championships"), fetch the top two web passages, and generate from two refined local sentences plus two web passages. The marathon recap never reaches the prompt, which is the whole point.

Code

The point here is the routing rule, not the model call. This tiny grader shows the same thresholds and the same three way branch as the example, so you can see why a mixed batch combines instead of picking one side.

def grade(score: float) -> str:    if score >= 0.70:        return "Correct"    if score >= 0.30:        return "Ambiguous"    return "Incorrect"

def route(grades: list[str]) -> str:    if all(g == "Correct" for g in grades):        return "refine-only"    if all(g == "Incorrect" for g in grades):        return "web-only"    return "combine"

scores: list[float] = [0.82, 0.45, 0.11]grades: list[str] = [grade(s) for s in scores]print(grades)  # -> ['Correct', 'Ambiguous', 'Incorrect']print(route(grades))  # -> combine

The prints match the hand computation: 0.82 clears 0.70, 0.45 sits between the cutoffs, 0.11 falls below 0.30, and a mixed list falls through to combine. Change the cutoffs and the route changes, which is why you tune them on labeled pairs instead of guessing.

Watch Out For

A confident wrong passage clears your cutoff

The evaluator scores fit, not truth. A passage that repeats the question words in a confident tone can score 0.78 and sail past a 0.70 cutoff while answering the wrong year or the wrong race. The symptom is specific: no web fallback fires, the answer cites a real doc, and the cited doc does not support the claim.

Don't fix this by nudging the cutoff by hand. Collect query and passage pairs from your own logs, label them Correct or not, and pick the cutoff that separates them. Log every grade and the fallback rate so you can see when the evaluator drifts. The scoring split to reuse is in RAG evaluation.

The web fallback leaks the query or returns junk

A fallback sends the user's words to a third party search API. If the query holds a name, an address, or internal project detail, that detail leaves your boundary. The other symptom is latency plus conflict: one extra search call on the slow path and two sources that disagree about the time.

Scrub before you search: strip names and IDs, rewrite to a short public form of the question, and allow only a short list of domains. Cap web use to one call with a timeout, and turn the fallback off entirely for private namespaces. Never send raw private queries to a public search API.

The Quick Version

  • Corrective RAG grades retrieved passages before generation instead of trusting the top nn blindly.
  • Each score falls into Correct, Ambiguous, or Incorrect against two tuned cutoffs such as 0.70 and 0.30.
  • All Correct refines locally, all Incorrect goes to web search only, and mixed batches combine kept strips with web results.
  • Refinement keeps the one or two sentences that answer the query and drops the rest before the final prompt.
  • The cost is an extra grading call plus web latency on weak batches, so skip it for stable private corpora where freshness adds nothing.