Skip to content
AI360XpertBeta

Out-of-Distribution (OOD) Detection

A model only knows the world it trained on. OOD detection is the gate that scores each new input for foreignness and blocks it before the model may answer.

Training embeddings sit inside a radius 3 circle while a foreign point at distance 5.0 from the center trips the gate, which is why the model never gets to call it a dog at 99 percent
Training embeddings sit inside a radius 3 circle while a foreign point at distance 5.0 from the center trips the gate, which is why the model never gets to call it a dog at 99 percent

Why Does This Exist?

Train a classifier to separate cats from dogs and it draws a boundary between them. That boundary extends forever in every direction, because no training row ever told it where to stop. Show it an airplane and it does not say it is lost. It checks which side of the line the airplane lands on and reports dog at 99 percent.

This is the central danger in deployment. Models are sharp inside their training world and confidently wrong outside it. Out-of-distribution detection builds the gate: a second score that measures how foreign an input looks, compared against a threshold, before the main model may answer. The uncertainty split behind one scoring family lives in aleatoric vs epistemic uncertainty. The sibling problem of finding strange rows without labels lives in anomaly detection. The repair for shaky numbers on familiar inputs lives in calibration.

Think of It Like This

A sommelier and the bouncer beside him

Your main model is an expert sommelier who knows only wine. Hand him a Pinot Noir and he names it. Hand him a Merlot and he names it.

Now hand him a glass of motor oil. He does not know motor oil exists, and his job forces an answer, so he swirls it and declares a very thick Cabernet. The failure is structural: he can only answer inside his training world.

OOD detection is the bouncer beside him. The bouncer never tastes. He looks at the glass, sees black liquid that smells like fuel, and bins it before the sommelier lifts it. The tasting skill never changes. The gate decides who gets to taste.

Where it stops: a bouncer sees color and smell directly, while the gate scores foreignness through embeddings the model itself learned. A bad embedding makes near and far meaningless, and no threshold rescues it.

How It Actually Works

You cannot train on everything the model should reject, because foreignness is infinite. So every method builds a fence around familiar data and scores distance from it. Three families dominate.

First, the uncertainty family. Ask several models or several stochastic passes and read their spread. Familiar inputs draw agreement, foreign ones split. Deep ensembles use exactly this: five votes near 0.93 mean routine, the same five spread across 0.15 to 0.99 with variance 0.10 mean foreign.

Second, the distance family. Take the embedding the network computes before its final layer. Compute the center of training embeddings and the spread in each direction. Score a new input by its Mahalanobis distance, meaning standard deviations from center after correcting for stretch. Far means foreign.

Third, the reconstruction family. Train an autoencoder to squeeze training inputs down and rebuild them. Familiar inputs rebuild cleanly with low error. Foreign inputs rebuild as garbage with high error, because the compressor never learned their parts.

Worked example: center, two points, one threshold

Put training embeddings at center (0,0)(0, 0) with unit spread in each direction, so distance is plain Euclidean distance. Set the gate threshold at 3.03.0. A routine input lands at (0.5,0.5)(0.5, 0.5):

d=0.52+0.52=0.50≈0.71d = \sqrt{0.5^2 + 0.5^2} = \sqrt{0.50} \approx 0.71

That is 0.710.71, under 3.03.0, so the gate passes it. A foreign input lands at (3.0,4.0)(3.0, 4.0):

d=3.02+4.02=25.0=5.0d = \sqrt{3.0^2 + 4.0^2} = \sqrt{25.0} = 5.0

That is 5.05.0, past 3.03.0, so the gate rejects it before any label ships. The diagram draws exactly this: a circle of radius 33 around the center, one dot at 0.710.71 inside, one dot at 5.05.0 outside with an amber flag. The 99 percent dog call never happens because the airplane never reaches the classifier.

Where the threshold comes from

The threshold is not 33 by nature. Score held-out familiar inputs, pick the cutoff that passes about 99 percent of them, and freeze it before deployment. Report what that cutoff costs: the share of familiar traffic you will flag for review. A gate with no measured false-flag rate is a guess with a number attached.

Code

The point is the check: rebuild both distances from the worked example and confirm 0.71 passes while 5.0 trips the gate.

import math

def distance_from_center(x: float, y: float) -> float:    return math.sqrt(x * x + y * y)

def gate(x: float, y: float, threshold: float = 3.0) -> str:    return "PASS" if distance_from_center(x, y) <= threshold else "REJECT"

print(f"{distance_from_center(0.5, 0.5):.2f} {gate(0.5, 0.5)}")  # -> 0.71 PASSprint(f"{distance_from_center(3.0, 4.0):.1f} {gate(3.0, 4.0)}")  # -> 5.0 REJECT

The printout matches the hand computation: routine input at 0.71 passes, foreign input at 5.0 rejects. Move the foreign point to (1.0,1.0)(1.0, 1.0) and watch it pass at 1.41, which is exactly the covariate-shift trap below.

Watch Out For

Rejecting the blurry dog with the airplane

The symptom is a gate that fires all day on inputs that are still the same object: night photos after day training, a new phone sensor, a blurry but genuine dog. Operators start overriding it, then ignore it, and the real airplane walks through an open door. Not all foreign pixels are foreign objects.

Fix: separate semantic shift from covariate shift before tuning. Semantic shift changes the object and must reject. Covariate shift changes the rendering and should often pass. Test both buckets apart, widen training rendering or add a dedicated shift check, and never tune one threshold on a mixed pile.

Setting the threshold by staring at it

The symptom is a threshold of 3 because 3 looks careful, with no measured pass rate on familiar traffic. In production the gate flags a fifth of routine rows, review queues flood, and someone disables the gate to clear the backlog.

Fix: fit the threshold on held-out familiar data only, record the false-flag rate there, and recheck it after every data change. Treat the threshold like any tuned cutoff: chosen on validation, reported with its cost, revalidated on a schedule.

The Quick Version

  • Classifiers extend their boundaries forever, so foreign inputs get confident wrong labels like dog at 99 percent.
  • OOD detection scores foreignness against a threshold and blocks the input before the model answers.
  • Three families work: vote spread, embedding distance, and rebuild error.
  • Our routine point sits at distance 0.71 and passes a threshold of 3.
  • Our foreign point at (3.0, 4.0) sits at distance 5.0 and rejects.
  • Thresholds come from held-out familiar data with a measured false-flag rate, never from habit.
  • Blurry familiar inputs and truly new objects need separate buckets, or the gate either spams or sleeps.