Hallucination in Vision-Language Models

Vision-language models describe objects that are not there with total confidence. Hallucination is the price of fluent guessing: language priors overpowering pixels.

Twenty-five invented yes answers against five misses show a yes-bias of 20 points, the measurable shape of hallucination

Why Does This Exist?

Fluent falsehood ships: medical assistants describing absent tumors, driving narrators announcing phantom pedestrians, alt-text inventing stairs that are not there. Language models learned that confident, detailed answers please raters — and vision grounding is weaker than the prior, so pixels lose the tug-of-war silently. The output reads authoritative either way.

Hallucination research measures and tames the invention: object-probing benchmarks (does the model claim absent things?), cause analysis (which prior overpowered which pixels?), and mitigations from grounding supervision to constrained decoding. The honest-testing home is open research; the QA sibling is visual commonsense reasoning.

The one idea you need first. POPE-style probing asks yes/no questions about object presence, balanced present/absent: accuracy, precision, and yes-rate separate seeing from guessing. A model answering "yes" $70\%$ of the time on $50/50$ probes hallucinates by bias alone, before any single answer is graded. CHAIR extends the idea to captions: fraction of mentioned objects actually present.

Think of It Like This

A tour guide paid by the anecdote

A guide paid per story fills empty frames with vivid paintings: eloquent, specific, believed — and absent. Fact-checkers following behind with the catalog find a quarter of the collection invented. The guide is not lying so much as optimizing: silence earns nothing, invention earns tips, and nobody checked until now.

VLM training pays per anecdote (helpful, detailed answers rate well) and rarely fact-checks against pixels. Hallucination probes are the catalog audit: present-or-absent per claimed object, yes-rates against base rates. Guides that survive audits learn a new skill — saying "not shown" — which training never previously rewarded.

How It Actually Works

Causes stack: co-occurrence priors (kitchens imply microwaves, so microwaves get "seen"), weak visual grounding (pooled features lose count and place), and decoding that compounds early guesses (one invented word conditions the next). Measurement probes presence (POPE), caption fidelity (CHAIR), and attribute correctness separately — object, count, color, and relation errors differ in cause and cure. Mitigations ground harder (region supervision, contrastive focus), constrain decoding (lookback, verifiction heads), and calibrate (refusal and hedging where evidence thins).

No mitigation holds without re-measurement: probes must rerun per release, since capability shifts move the hallucination frontier.

Worked example

$100$ balanced probes ( $50$ objects present, $50$ absent). Model answers "yes" $70$ times with this confusion: TP $= 45$ , FP $= 25$ , FN $= 5$ , TN $= 25$ .

Accuracy: $(45 + 25)/100 = 0.70$ .
Precision: $45/70 \approx 0.643$ — over a third of claimed sightings are invented.
Yes-bias: $70\%$ yes-rate against $50\%$ truth — a $+20$ -point bias toward existence.

The $25$ false positives are hallucinations proper: absent things described as seen. Cutting yes-bias (calibrated thresholds, absent-aware training) lifts precision before any architecture changes.

Code

The point of this snippet is the audit: four counts become accuracy, precision, and bias in three lines.

1def pope_audit(tp: int, fp: int, fn: int, tn: int) -> tuple:2    total = tp + fp + fn + tn3    acc = (tp + tn) / total4    prec = tp / (tp + fp)  # claimed sightings that exist5    yes_bias = (tp + fp) / total - (tp + fn) / total6    return round(acc, 3), round(prec, 3), round(yes_bias, 3)7
8
9print(pope_audit(45, 25, 5, 25))  # -> (0.7, 0.643, 0.2)

Watch Out For

Silence sold as safety

Some "fixes" suppress all uncertain claims: models refuse, hedge everything, or caption sparsely. Hallucination rates fall because claims fall — recall of real objects collapses alongside, and the product goes mute where it should inform. The metric improves; the user loses.

Track recall and informativeness beside precision: F1-style balances, not precision alone. A model that never claims never hallucinates and never helps.

Probes overfitted like any benchmark

POPE phrasing is fixed ("is there a..."), so tuning on probe style (threshold shifts, template-specific calibration) lifts scores without touching grounding. Capability looks cured; deployment on rephrased queries relapses immediately.

Vary probe phrasing, objects, and image sources per evaluation round, and hold out probe styles entirely. A cure that works on one questionnaire is a costume, not a cure.

The Quick Version

VLMs invent absent objects because language priors and co-occurrence beat weak grounding.
The worked audit gives accuracy $0.70$ , precision $0.643$ , and yes-bias $+0.20$ on balanced probes.
The $25$ false positives are hallucinations proper; cutting yes-bias lifts precision first.
Track recall beside precision or silence masquerades as safety.
Rotate probe styles per round or tuning memorizes the questionnaire instead of grounding.