Dataset Decontamination

If a benchmark's questions are in your training data, its score measures memory. At web scale they are, and checking is a search problem rather than a choice.

Eighty of 1,000 benchmark questions sitting in the training data lifts the reported score from 0.62 to 0.649, because the leaked slice is answered from memory at 0.98 instead of ability at 0.62.

Why Does This Exist?

Every public benchmark lives on the internet. The questions, usually the answers, and very often a blog post walking through them. Train on a web scrape and you train on some of that material. Not through a mistake. Through the corpus itself, unless somebody looked for it and took it out.

A contaminated benchmark does not fail loudly. It reports a higher score. That is the same reason data leakage is the hardest bug class in this band. The failure mode is a better number. The published figure then mixes ability with recall in unknown proportions, and no amount of staring at it separates the two.

So decontamination decides whether any number you report means anything. The detection machinery is shared with deduplication, pointed at a fixed target set instead of the corpus itself. And once you can measure overlap, you can gate on it, which is the same discipline as data validation. This page covers what counts as contamination, how to find it, and what to do when you find it.

Think of It Like This

A student who has seen the paper

A candidate sits an exam and scores 94 percent. Excellent, until you learn three of the twenty questions appeared in a revision guide they studied last week.

Here is what makes it awkward rather than simply cheating. They did not steal the paper. Those three questions have circulated in study guides for years, and any diligent student would have met them. Nobody did anything wrong, and the 94 percent still does not measure what they can do.

Worse, you cannot correct it after the fact. You do not know which three, and you cannot ask, because a well-revised question and a genuinely understood one feel identical from the inside.

The only fix is upstream. Check the paper against the study guides before the exam, and write fresh questions where they overlap.

Where it stops: an exam board can quarantine one paper. A training corpus absorbs the whole web, paraphrases included, so the check is statistical and never complete. You measure a rate and gate on it rather than proving zero.

How It Actually Works

What counts as contamination

More shapes than people check for. Verbatim overlap is the question text sitting in training. Answer-only overlap is the answer appearing with enough context to key on, while the question never appears. Paraphrase is a rewording, which any hash-based check misses entirely. Solution walkthroughs are tutorials explaining the reasoning, which teach the pattern and are worse than the answer alone. And derivative datasets are the sneakiest shape. Someone built an instruction-tuning set from a benchmark, you trained on the instruction set, and the original never appears anywhere in your provenance.

That last shape is why a provenance argument ("we never downloaded that benchmark") is not a decontamination claim.

Detecting it

Let $n$ be the n-gram order, the number of consecutive tokens in each shingle. Build the set of $n$ -grams for each benchmark item and flag any training document sharing more than a threshold count. Common settings are $n = 8$ to $n = 13$ , and the choice of $n$ is the whole sensitivity dial. Too small and every document matches on common phrasing. Too large and a two-word paraphrase escapes.

MinHash plus locality-sensitive hashing scales that comparison to a full corpus without comparing every pair. Embedding similarity catches paraphrase, which $n$ -grams cannot, at the price of a fuzzier threshold and many more false positives to adjudicate by hand. Canary strings are the preventive trick. Benchmark authors embed a unique random string in their files, so a model that reproduces the canary has demonstrably seen the file. Cheap, definitive when it fires, and only available when the author thought of it.

What to do when you find it

Let $c$ be the contamination rate, the fraction of benchmark items found in training. Remove the overlapping documents from training if you catch them before the run. That is the clean answer and it needs the check to happen before training rather than after. Remove the overlapping items from evaluation if you catch them afterwards, and report the clean-subset score next to the full one. Report $c$ either way. A benchmark score without a contamination figure is incomplete in the same way an estimate without an interval is. And keep a private evaluation set that has never been published anywhere. A public benchmark's contamination risk only grows, so a few hundred carefully written internal items tell you more over time than any leaderboard.

Why it keeps happening

The incentives point the wrong way. Decontamination lowers your reported number, costs engineering time, and produces nothing anyone celebrates. Nobody gets promoted for discovering that a headline result was three points softer than published. So make it a pipeline gate instead of a virtue. Run the overlap check automatically against every evaluation set you use, fail the run when $c$ exceeds a threshold you set in advance, and log the figure with the result.

Worked example

A 1,000-question benchmark. The model genuinely answers unseen questions at $a_{clean} = 0.62$ . On contaminated questions it answers at $a_{mem} = 0.98$ , because it recalls rather than reasons.

At $c = 0.08$ the reported score is $(1 - 0.08) \times 0.62 + 0.08 \times 0.98 = 0.5704 + 0.0784 = 0.6488$ , which rounds to $0.649$ . That is $0.0288$ of inflation, or 2.9 points, from 80 leaked questions. Comfortably enough to change a ranking, and small enough that nobody suspects anything.

At $c = 0.30$ , which is not unusual for an older benchmark, the reported score is $0.70 \times 0.62 + 0.30 \times 0.98 = 0.434 + 0.294 = 0.728$ . That is $0.108$ , or 10.8 points, and it would read as a substantial modelling advance. Without measuring $c$ , $0.649$ and $0.62$ are indistinguishable from the outside, and so are $0.728$ and $0.62$ . The inflation is a function of a number nobody reported.

Code

1clean: float = 0.62       # accuracy on questions the model has never seen2memorised: float = 0.98   # accuracy on questions found in training3
4def reported(clean_acc: float, memorised_acc: float, rate: float) -> float:5    """Benchmark score when a fraction rate of it leaked into training."""6    return (1 - rate) * clean_acc + rate * memorised_acc7
8for rate in (0.00, 0.08, 0.30):9    print(rate, round(reported(clean, memorised, rate), 4))10# -> 0.0 0.6211# -> 0.08 0.648812# -> 0.3 0.72813print(round(reported(clean, memorised, 0.30) - clean, 4))  # -> 0.108

The function is linear in the contamination rate, which is the useful takeaway. Inflation scales directly with overlap, so measuring the overlap tells you the size of the correction.

Watch Out For

Checking provenance instead of content

"We never downloaded that benchmark, so we are not contaminated." It sounds like a complete answer and it checks the wrong thing.

The benchmark is on the web, so it is in the scrape. Its questions appear in blog posts, forum answers, repositories of solutions, and lecture slides. Derivative datasets carry no marker, so you can train on a benchmark through three degrees of separation with a clean download log throughout.

Symptom: a confident zero-contamination claim backed by a download list. Fix: run the content check. Measure $n$ -gram overlap between every evaluation set you use and your final training corpus, at a threshold you pick in advance, and log the rate. It is the same MinHash and LSH infrastructure you built for deduplication, pointed at a much smaller target set, so it is cheap once that exists.

An n-gram threshold that catches nothing, or everything

You run a 13-gram overlap check and it flags 0.1 percent of documents. Clean, apparently. But thirteen consecutive tokens is a long exact match. Change two words, reformat a number, and the overlap vanishes while the model still learned the item. A high threshold reports a comfortable figure and misses paraphrase, which is most of the real contamination.

Drop to 5-grams and the opposite happens. Every document containing ordinary phrasing matches something, the flag rate hits double digits, and you cannot act on it.

Symptom: a contamination rate that is suspiciously close to zero, or so high it is useless. Fix: run both. A tight $n$ -gram check for verbatim overlap plus an embedding-similarity pass for paraphrase, adjudicate a sample of what each flags by reading it, and report both rates rather than the friendlier one. And keep a private evaluation set, because it is the only thing that stays uncontaminated by construction.

The Quick Version

Every public benchmark is on the web, so a web-scraped corpus contains some of it unless somebody looked.
Contamination never fails loudly. It raises your score, which is why nobody notices.
It comes in five shapes: verbatim, answer-only, paraphrase, solution walkthroughs, and derivative datasets.
Provenance is not a decontamination claim. Check the content of the corpus, not the download log.
N-gram overlap at 8 to 13 tokens is the standard check, scaled with MinHash and LSH. Embedding similarity catches paraphrase.
Canary strings prove contamination definitively when the benchmark author included one.
Remove from training if you catch it early, remove from evaluation if you catch it late, and report the contamination rate either way.
Eight percent contamination moves 0.62 to 0.649. Thirty percent moves it to 0.728. Inflation is linear in the overlap rate.
A private, never-published evaluation set is the only durable defence.
Make it a pipeline gate with a threshold, because the incentives will never make it a virtue.