Skip to content
AI360XpertBeta

Conformal Prediction

A single label pretends you are sure. Conformal prediction returns a small set of labels that holds the truth about eight times in ten, and the set grows when the model is unsure.

Nine calibration scores set the threshold at 0.55, so the blurry image earns the two label set of dog and fox for 80 percent coverage
Nine calibration scores set the threshold at 0.55, so the blurry image earns the two label set of dog and fox for 80 percent coverage

Why Does This Exist?

You have seen the failure. A classifier looks at a blurry animal and announces dog at 0.99. The number came from a softmax, which must sum to one, so it cannot say it does not know. In a clinic or on the road that false certainty decides what happens next, and you'll pay for it.

Start with one grounding. For input xx the model gives probabilities py(x)p_y(x), one per label yy. The top label is the guess and its probability is the confidence. Calibration asks whether 0.90 means right nine times in ten. Most models fail that check, and you'll feel it the first week live.

This page builds the next step. You'll keep your trained model, score its past mistakes on fresh data, and return a set with a stated coverage rate instead of one label. When you need a single cut with priced mistakes, use threshold selection. When inputs may be foreign, read out-of-distribution detection beside this page, because shift breaks the promise below.

Think of It Like This

The fishing net

You want fish for dinner and you don't trust a spear. You check last week's catch log, see how far your throws usually miss, and pick a net width that held fish eight days out of ten. On a clear day you use a small net and land one fish. On a murky day you cast wide and land two, plus weed you will throw back.

The log is the calibration set, the mesh width is the threshold q^\hat{q}, each cast is a new prediction, and the net width is the set size. A wide net means you were unsure, not that the lake got bigger.

Where it stops: the net never names the fish, it only promises the fish is inside. And the promise holds only in the same lake. Move to a new lake with different fish and the old mesh keeps no promise.

How It Actually Works

Split conformal prediction keeps your model frozen and adds a wrapper. You need fresh labeled data the model never trained on, called the calibration set. Exchangeability is the one assumption: calibration and future inputs come from the same process, in no special order. If that holds, the coverage promise holds. If it fails, nothing below holds.

Scores, threshold, and sets

For each calibration pair (Xi,Yi)(X_i, Y_i), meaning input ii with true label YiY_i, compute a nonconformity score s(Xi,Yi)s(X_i, Y_i), meaning how strange the true label looked to the model. For classification the standard choice is

s(x,y)=1−py(x)s(x, y) = 1 - p_y(x)

Here py(x)p_y(x) is the model's stated probability for label yy on input xx. A true label given 0.95 leaves score 0.05, meaning routine. A true label given 0.10 leaves score 0.90, meaning the model nearly missed it.

Pick a miscoverage rate α\alpha, meaning the failure budget you accept. Coverage is 1−α1 - \alpha, meaning the share of future sets that must hold the truth. Sort the nn calibration scores from small to large and take position

k=⌈(n+1)(1−α)⌉k = \lceil (n + 1)(1 - \alpha) \rceil

Here nn is the calibration size, ⌈⋅⌉\lceil \cdot \rceil rounds up, and q^\hat{q} is the kkth smallest score, meaning the threshold. The prediction set for a new xx keeps every label whose score fits under it:

C(x)={y:s(x,y)≤q^}C(x) = \{ y : s(x, y) \le \hat{q} \}

Here C(x)C(x) is the set, and its size reports uncertainty. One label means sure. Three labels means unsure. The finite sample promise is P(Yn+1∈C(Xn+1))≥1−αP(Y_{n+1} \in C(X_{n+1})) \ge 1 - \alpha, meaning the next truth lands inside at least at the stated rate when exchangeability holds.

Worked example: nine scores give threshold 0.55

Take n=9n = 9 calibration scores, already sorted:

[0.05,0.12,0.15,0.22,0.28,0.33,0.41,0.55,0.90][0.05, 0.12, 0.15, 0.22, 0.28, 0.33, 0.41, 0.55, 0.90]

Set α=0.20\alpha = 0.20, so coverage is 1−α=0.801 - \alpha = 0.80. Then (n+1)(1−α)=10×0.80=8.0(n + 1)(1 - \alpha) = 10 \times 0.80 = 8.0, and k=⌈8.0⌉=8k = \lceil 8.0 \rceil = 8. The 88th smallest score is q^=0.55\hat{q} = 0.55. That is the whole fit. No training, one sort, one pick.

A blurry new image gives probabilities [0.50,0.47,0.02,0.01][0.50, 0.47, 0.02, 0.01] over [dog, fox, coyote, rabbit], which sum to 1.001.00. Scores are one minus each:

[0.50,0.53,0.98,0.99][0.50, 0.53, 0.98, 0.99]

Keep labels with score at or under 0.550.55. Dog at 0.500.50 stays. Fox at 0.530.53 stays. Coyote at 0.980.98 goes. Rabbit at 0.990.99 goes. The set is {dog,fox}\{\mathrm{dog}, \mathrm{fox}\} with size 22.

A sharp image behaves differently with the same threshold. Probabilities [0.88,0.07,0.03,0.02][0.88, 0.07, 0.03, 0.02] give scores [0.12,0.93,0.97,0.98][0.12, 0.93, 0.97, 0.98]. Only dog at 0.120.12 fits, so the set is {dog}\{\mathrm{dog}\} with size 11. Same model, same q^\hat{q}, small set when sure and larger set when unsure. The diagram draws these numbers: nine dots with the line at 0.550.55, then four dots with two kept.

What the guarantee actually promises

The 0.800.80 is marginal, meaning average over random draws of calibration and test data. It doesn't promise 8080 percent for every image, every class, or every week. A rare class can sit at 6060 percent while a common class sits at 9090 percent and the average still reads 8080 percent. When you need per group safety, split calibration by group, which practitioners call Mondrian conformal prediction, and check slice coverage before you ship.

The price is size. A weak model keeps its promise by returning huge sets. That is honest and often useless, so track mean set size beside coverage. If size drifts up, the model or the data moved and no threshold trick will save you. Probability calibration can sharpen single numbers, and aleatoric vs epistemic uncertainty tells you whether more data or a bigger model is the fix. Use conformal prediction when a wrong single guess costs more than asking a human to pick from two or three.

Code

The point is the check: sort the same nine scores, pick the 8th, and rebuild the two label set.

import numpy as np
scores: np.ndarray = np.array([0.05, 0.12, 0.15, 0.22, 0.28, 0.33, 0.41, 0.55, 0.90])alpha: float = 0.20k: int = int(np.ceil((len(scores) + 1) * (1.0 - alpha)))qhat: float = float(np.sort(scores)[k - 1])
probs: np.ndarray = np.array([0.50, 0.47, 0.02, 0.01])names: list[str] = ["dog", "fox", "coyote", "rabbit"]chosen: list[str] = [n for n, p in zip(names, probs) if 1.0 - float(p) <= qhat]
print(f"{k} {qhat:.2f}")  # -> 8 0.55print(chosen)  # -> ['dog', 'fox']

The printout matches the hand computation: position 8, threshold 0.55, kept labels dog and fox.

Watch Out For

Coverage collapses after shift

You calibrate on daylight images and deploy on night images. Calibration promised 80 percent and the first live week lands near 62 percent, with sets that look the same size. Nothing in the wrapper warns you because exchangeability already broke.

Symptom: empirical coverage on recent labels falls ten or more points under 1−α1 - \alpha while set size stays flat, often right after a camera, prompt, or population change. Fix: freeze a rolling recent split, measure coverage there weekly, and recalibrate q^\hat{q} on it. Gate foreign inputs with out-of-distribution detection and do not trust old sets past a shift.

Fitting the threshold on training scores

Scores from training rows look too good because the model already fit them. The threshold comes out low, sets come out small, and live misses pile up past the α\alpha budget.

Symptom: notebook coverage reads 80 percent at q^=0.30\hat{q} = 0.30 while held out coverage with the same q^\hat{q} reads near 65 percent, and singleton sets miss most. Fix: fit q^\hat{q} only on data the model never trained on, or on out of fold predictions. Measure coverage and mean size on a third split neither training nor calibration has touched, as in calibration.

The Quick Version

  • Conformal prediction returns a set C(x)C(x) of labels instead of one guess, and a larger set means more uncertainty.
  • Nine calibration scores with failure budget α=0.20\alpha = 0.20 give position k=8k = 8 and threshold q^=0.55\hat{q} = 0.55.
  • The blurry image with probabilities [0.50,0.47,0.02,0.01][0.50, 0.47, 0.02, 0.01] keeps dog and fox, so the set holds two labels with size 2.
  • The 80 percent promise is marginal over random draws and needs exchangeable data, so the average can hide weak coverage on one rare class.
  • Recalibrate q^\hat{q} on recent held out data after any shift, and track mean set size because a weak model keeps coverage with uselessly large sets.