Reflection and Self-Correction
Do not ship the model's first draft. Make it check its own answer against a short checklist, fix exactly what fails, and stop after a couple of tries.
Why Does This Exist?
First drafts drop constraints. Ask for three lines about a lighthouse, six words per line, never the word "dark", and you will often get back three neat lines where one quietly says "dark". The shape is right and one rule is broken, which is the worst kind of wrong: it looks done.
A model writes one token at a time and cannot take a token back (that is next-token prediction). Chain of thought gives it room to work step by step, but working room is not a checking step. Reflection adds the checking step: generate a draft, grade it against a short checklist, rewrite only what failed, and stop after a fixed number of rounds.
This page covers that single-prompt loop: one draft, one verdict, one fix. Voting across many drafts instead of fixing one belongs to self-consistency. Hiring a separate model as the grader belongs to LLM as a judge. Running the same loop over a whole agent trajectory with tool feedback belongs to self-reflection agents.
Think of It Like This
Packing a suitcase against the airline checklist
You are flying budget and the rules are printed above the check-in desk: one bag, small enough to fit the metal frame, nothing sharp inside. You pack at home, then you measure the bag against the frame. Too tall. You repack once, measure again, and it slides in. You board.
The packed bag is the draft. The printed rules are the checklist. The metal frame is the critic: it does not admire your folding, it answers one question with evidence. The repack is the revision. And the flight leaving without you is the round cap: at some point you stop repacking and accept the outcome.
Where it stops: the frame measures size, never taste. A checklist verdict is exactly that narrow. It catches "line 2 says dark". It says nothing about whether line 2 is any good.
How It Actually Works
Call the task , in plain words everything you asked for. Call the first answer the draft . Call the checklist , a short list of checks that a program or a strict prompt can actually run. Call the verdict , which is either PASS or FAIL plus the evidence: which check failed and where. Call the rewrite step , which takes the draft plus the evidence and returns a new draft. Call the budget , the maximum number of revision rounds, fixed before you start.
One round goes like this: draft from , run every check in over , and read the verdict . If it says PASS, ship . If it says FAIL, feed plus the evidence into to get , and check again. The loop ends at the first PASS or when you have spent rounds, whichever comes first. Then you ship the best draft you have.
The critic must bring new evidence
Here is the part most tutorials skip: the critic has to check with something the draft did not have. A checklist the code can run (word counts, banned words, valid JSON, passing tests), a retrieved document, a tool result, a stricter prompt, or a second model. Ask the same model to re-read its own answer with no new evidence and it mostly nods along. Worse, it sometimes talks itself out of an answer that was right.
So reflection is not "ask twice". It is "check with something". No checkable signal, no loop. For a one-hop fact with nothing to verify against, a single shot plus solid hallucination mitigation habits will serve you better than two calls agreeing with each other.
Worked example
Task : write exactly 3 lines about a lighthouse, at most 6 words per line, never use the word "dark". Checklist holds three checks, each runnable by counting.
Draft :
- Line 1: "The lighthouse stands on gray rock". Count them: The(1) lighthouse(2) stands(3) on(4) gray(5) rock(6). Six words. Clean.
- Line 2: "Its lamp turns through dark water". Its(1) lamp(2) turns(3) through(4) dark(5) water(6). Six words, but there it is: "dark".
- Line 3: "Gulls rest on the white railing". Gulls(1) rest(2) on(3) the(4) white(5) railing(6). Six words. Clean.
The verdict : line count 3 of 3, pass. Lengths 6, 6, 6 against a max of 6, pass. Banned-word scan finds "dark" in line 2, FAIL, with that exact evidence. One failed check out of three, and it names the line and the word.
Revision changes one word: "dark" becomes "cold". The new line 2 reads "Its lamp turns through cold water": still 6 words, no banned word. Re-run the checklist: 3 of 3 pass. Verdict: PASS. Rounds spent: 1 of a budget .
Count the calls while you are here. A single shot costs 1 call. This round cost 3: draft, verdict, revision. Reflection roughly triples the bill per round, which is why the checklist stays short and the budget stays small.
Code
The loop below is the whole pattern with the check written as plain code, not a second model call, so its verdict is evidence you can trust. It runs the lighthouse fixture end to end with a budget of 2 rounds.
def critique(lines: list[str]) -> list[str]: """Check the draft. Returns one issue per broken rule, else [].""" issues: list[str] = [] if len(lines) != 3: issues.append(f"want 3 lines, got {len(lines)}") for i, line in enumerate(lines, start=1): words = line.split() if len(words) > 6: issues.append(f"line {i} has {len(words)} words, max 6") if "dark" in words: issues.append(f"line {i} uses banned word 'dark'") return issues
draft: list[str] = [ "The lighthouse stands on gray rock", "Its lamp turns through dark water", "Gulls rest on the white railing",]
print(critique(draft)) # -> ["line 2 uses banned word 'dark'"]
current = list(draft)rounds = 0while critique(current) and rounds < 2: current = [line.replace("dark", "cold") for line in current] rounds += 1
print(rounds) # -> 1print(critique(current)) # -> []print(current[1]) # -> Its lamp turns through cold waterOne round, one word changed, and the verdict flips to PASS. That is the entire mechanism: evidence in, pinpointed fix out, and a round cap that ends the story no matter what.
Watch Out For
The critic that only nods
Symptom: every draft passes on the first try, yet users keep reporting broken answers. The loop runs, the bill grows, and nothing improves, because the critic re-reads with the same eyes that wrote the draft.
Give the critic a real check or drop the loop. A runnable checklist, a test suite, a retrieved source, a stricter rubric, or a separate judge model (see LLM as a judge). If none of those exist for your task, reflection is decoration. Ship the single shot.
The loop that never lands
Symptom: the same verdict comes back every round, the draft shuffles words without converging, and tokens burn until something outside the loop kills the run.
Cap the rounds up front (2 is a sane default for format fixes) and stop early when the verdict stops changing between rounds. Ship the best draft, log the final verdict, and hand the rest to a human or a stronger check. Runaway loops are an agent disease; the catalogue lives in agent failure modes.
The Quick Version
- Reflection is a loop with four parts: a draft, a verdict with evidence, a pinpointed revision, and a round cap that always wins.
- The critic must check with something the draft did not have: runnable rules, a tool result, retrieved text, or a separate judge. Re-reading alone changes little.
- One failed check should name the rule and the location, like "line 2 uses banned word 'dark'", so the fix touches one word instead of rewriting everything.
- A round costs roughly three calls for one shipped answer, so keep the checklist short and the budget at 2 or 3.
- When nothing converges, stop at the cap and ship the best draft with its final verdict attached.