Precision-Recall Curve

Why Does This Exist?

Ten thousand subscribers, 900 of them cancelling next month. A model scores everyone for cancellation risk and the top slice gets a retention offer: two months at half price, which costs money every time it lands on somebody who was never leaving.

Two numbers matter. Of the people you sent the offer to, how many were going to cancel — that's precision, what the discount budget buys. Of the 900 leaving, how many did you reach — that's recall, the revenue you saved.

Now grow the business. Same model, same threshold, same 900 leavers, but 30,000 subscribers. On a ROC curve almost nothing happens: recall is unchanged and the false positive rate moves half a point, because the extra 20,000 land in an already-large denominator. Precision falls from 0.50 to 0.23.

One view says nothing changed. The other says the economics broke. This is the one that noticed.

Think of It Like This

How It Actually Works

Two axes, and the count missing from both

Precision is the flags that were right over everything you flagged. Recall is that same numerator over everything that really was positive. Something is absent from both: the count of negatives the model correctly left alone.

That one fact is the whole difference from ROC. The false positive rate divides your false alarms by the entire negative class, so a mountain of obvious negatives inflates that denominator and shrinks the rate. Precision divides them by the flags you made, which doesn't grow with the population.

So hold the threshold still and add 20,000 subscribers who were never going to leave. Recall can't move — its denominator is the 900 leavers. The false positive rate barely moves, because you added to both its numerator and its huge denominator. Precision falls hard, because you added to its denominator only.

The baseline moved, so the number moved

A random classifier's PR curve is a flat line at the positive rate, not at 0.5. Flag people at random and the share of your flags that were leavers is just the share of leavers in the population. On a 9%-churn base, chance is 0.09.

Which means average precision, the standard summary of this curve, is only interpretable next to the prevalence it came from. An average precision of 0.4 is strong on a 2%-positive problem, where it beats chance twentyfold. The same 0.4 on a 45%-positive problem is barely better than a coin. ROC's chance line stays pinned to 0.5 whatever the class balance does, and that stability is what makes it blind here.

The sawtooth, and which summary to trust

The curve isn't monotone, which surprises people who've only seen ROC. Walk the threshold down one prediction at a time: each is either a hit, nudging precision up, or a miss, dragging it down. Recall only climbs, so the path zigzags.

At the low-recall end the zigzag has visible teeth. Up there the threshold is high, you've flagged maybe eight people, and precision's denominator is eight, so one case flipping moves the number by more than a tenth.

Which is also why average precision beats a trapezoidal area. In precision-recall space a straight line between two operating points describes a classifier that doesn't exist, so the trapezoid comes out optimistic. Average precision instead sums each step in recall times the precision at that step. average_precision_score does that; auc(recall, precision) doesn't.

Show Me the Code

The same 900 leavers, the same threshold, and a subscriber base that keeps growing.

1import numpy as np2
3rng = np.random.default_rng(8)4
5
6def at_cut(churn: np.ndarray, stay: np.ndarray, cut: float) -> tuple[float, float, float]:7    tp, fp = int(np.sum(churn >= cut)), int(np.sum(stay >= cut))8    # neither recall nor precision ever divides by the stayers the model got right9    return tp / churn.size, tp / max(tp + fp, 1), fp / stay.size10
11
12churn = np.clip(rng.normal(0.55, 0.16, 900), 0.0, 1.0)  # the same 900 leavers every time13for n in (9_100, 29_100, 99_100):14    stay = np.clip(rng.normal(0.22, 0.16, n), 0.0, 1.0)15    recall, precision, fpr = at_cut(churn, stay, 0.45)16    print(f"{n:>6} stayers   recall {recall:.3f}   precision {precision:.3f}   fp rate {fpr:.3f}")17# ->   9100 stayers   recall 0.728   precision 0.503   fp rate 0.07118# ->  29100 stayers   recall 0.728   precision 0.232   fp rate 0.07519# ->  99100 stayers   recall 0.728   precision 0.080   fp rate 0.076

Recall is identical to three decimals across all three runs, because its denominator never changed. The false positive rate moves half a point. Precision goes from 0.503 to 0.080 — half your offers landing on real leavers, down to one in twelve. Track the false positive rate and that stays invisible.

Watch Out For

The Quick Version

• Precision on the vertical, recall on the horizontal, one point per threshold. Neither axis involves the true-negative count.
• That's the whole difference from ROC. A growing pile of obvious negatives can't inflate a denominator, so it can't flatter you.
• Hold the threshold and add negatives: recall untouched, false positive rate barely moves, precision falls hard.
• Chance sits at the positive rate, not 0.5, so average precision only means something next to its prevalence.
• The curve saw-tooths at the low-recall end, where precision's denominator is a handful of cases.
• On rare-event problems, report this curve. Keep ROC for comparing rankings.

What to Read Next

• ROC and AUC is the curve this one replaces under imbalance, and worth knowing for what it does see.
• Threshold Selection turns this curve into the operating point you deploy.
• Classification Metrics covers precision, recall and F-beta at a single cut.
• Imbalanced Data is where this page stops being a preference.
• Definitions worth a look: Class Imbalance, SMOTE, and Stratified Sampling.