Confusion Matrix

Four counts in one grid sort every prediction by what was said and what was true, and every classification score divides those cells.

Four cells sort 12000 bottles into 168 catches, 72 misses, 384 false alarms and 11376 passes, with recall dividing the really cracked row

Why Does This Exist?

A camera watches bottles roll past and calls each one cracked or fine. Take one shift as a worked fixture: 12,000 bottles, 240 genuinely cracked, 552 flagged by the camera. You can't tell from those three totals whether the camera is careful or trigger happy.

Only four things can happen to a bottle. The camera says cracked and it was, says cracked when it was fine, says fine when it was cracked, or says fine and it was. Count each outcome and you hold the whole story, because every percentage in a model report divides those four counts by a different total.

Names trip people up, so fix them now. Positive never means good, it means the model said yes. True or false says whether that call was right. A false positive is a false alarm, a good bottle in the reject bin. A false negative is a miss, a cracked bottle heading out on a pallet. A false alarm costs one bottle, a miss can cost a customer and a batch recall, and nothing inside the model knows that trade.

The grid also assumes someone inspected those 12,000 bottles by hand. That hand count is ground truth, and the table is only as honest as it is. This page builds the four counts and shows three divisions from them. Classification metrics picks between those divisions, and model evaluation puts the choice beside splits, thresholds and slices.

Think of It Like This

The smoke alarm in your kitchen

You mount one alarm above the stove. Burnt toast sets it off and you wave a towel, and that is a false alarm. A pan fire it sleeps through is a miss. Sounding means the alarm said yes, fire present means the truth, and the two cross exactly like the bottle grid.

You keep the kitchen alarm jumpy on purpose because towel waving is cheap and a house fire is not. Move the same device to an aircraft cabin and the costs flip, since a false alarm can divert three hundred people, so you tune it to trip much later and accept a slower catch of real smoke. Same two failures, opposite setting, because the price of each mistake changed.

Where it stops: the alarm watches one stream and rings now, it never sorts four kinds of bottles, and it never reports precision. When you need named error kinds across many classes, you leave the kitchen and count the grid.

How It Actually Works

Call the four counts $TP$ , $FP$ , $FN$ and $TN$ . $TP$ counts true positives, catches the camera got right. $FP$ counts false positives, false alarms. $FN$ counts false negatives, misses. $TN$ counts true negatives, good bottles passed correctly. Rows hold what was true, columns hold what the camera said.

The four counts, on one shift

Of the 240 cracked bottles the camera caught 168 and missed 72, so $TP = 168$ and $FN = 72$ . Of the 11,760 good ones it passed 11,376 and binned 384, so $TN = 11376$ and $FP = 384$ . Add across and you get the shift back: $168 + 72 = 240$ really cracked, $384 + 11376 = 11760$ really fine, $168 + 384 = 552$ flagged, $72 + 11376 = 11448$ passed, $240 + 11760 = 12000$ in all. The diagram draws these same four numbers, with the really cracked row accented because recall divides by it.

Rows stay fixed by reality, since 240 bottles were cracked whether the camera ran or not. Columns stay fixed by the camera, since it flagged 552. That split explains nearly every argument about scores. A row answers what happened to the bottles that were cracked, while a column answers what the flags really were.

Worked example: the same 168 over three totals

Recall divides catches by the row they sit in:

\text{recall} = \frac{TP}{TP + FN} = \frac{168}{240} = 0.70

Seven in ten cracked bottles got caught. Precision divides those same catches by the column instead:

\text{precision} = \frac{TP}{TP + FP} = \frac{168}{552} \approx 0.30

Three in ten flags were justified. Accuracy divides both correct cells by everything:

\text{accuracy} = \frac{TP + TN}{TP + FN + FP + TN} = \frac{11544}{12000} = 0.962

One numerator, three denominators, three numbers that disagree with none of them lying. Accuracy looks high only because good bottles dominate the total. Choosing between these divisions is what classification metrics is for.

More than two classes

Nothing needs exactly two classes. Sort bottles into fine, cracked, underfilled or askew label and you get a $K \times K$ table, where $K$ is the number of classes. Rows stay the truth, columns stay the call, and the diagonal holds what the model got right.

The off diagonal cells carry the value because mistakes cluster. Underfilled bottles read as fine far more often than askew labels do, which tells you to fix fill lighting and leave the label checker alone. No single score hands you that pattern, and the whole table still describes one threshold.

Code

The point is the check: rebuild the three scores from the same four counts and confirm the hand arithmetic.

1tp: int = 1682fn: int = 723fp: int = 3844tn: int = 113765
6recall: float = tp / (tp + fn)7precision: float = tp / (tp + fp)8accuracy: float = (tp + tn) / (tp + fn + fp + tn)9
10print(f"{recall:.2f} {precision:.2f} {accuracy:.3f}")  # -> 0.70 0.30 0.962

The printout matches the hand computation above: recall 0.70, precision 0.30, accuracy 0.962.

Watch Out For

Reading the grid with the axes silently swapped

Some tools put truth on rows and some put it on columns, and a few plotting helpers transpose to fit their labels.

Symptom: you point at 384 and call it misses when it is false alarms, so precision and recall swap and every fix you try pushes the wrong error.

Fix: label both axes in words on every figure you share, then check row totals against class counts you know independently. Here rows must sum to 240 and 11760. If they sum to 552 and 11448 instead, the table is transposed.

Treating the matrix as a property of the model

A confusion matrix freezes one threshold on one dataset, yet teams quote it like a parameter count.

Symptom: a report says the model has 384 false alarms, then someone quiets the reject bin by moving the cut and the count drops without any training, so last week and this week stop comparing.

Fix: write the threshold beside every matrix and compare models only at the same cut. When the cut itself is the open question, sweep it with ROC and AUC and the precision recall curve instead of freezing one grid.

The Quick Version

A confusion matrix crosses what the model said with what was true into TP, FP, FN and TN counts.
Positive means the model said yes, so a false positive is a false alarm and a false negative is a miss.
One bottle shift sorts 12000 bottles into 168 true positives, 72 false negatives, 384 false positives and 11376 true negatives.
Recall divides 168 catches by the 240 really cracked row for 0.70, precision divides 168 by 552 flagged for 0.30, and accuracy divides 11544 correct by 12000 for 0.962.
With more classes the off diagonal cells name which classes get mistaken for which, which no single score provides.
A confusion matrix describes one threshold on one dataset, so moving the cut moves all four cells together.
Row totals must equal class counts in the data, so check axes before trusting any matrix you did not draw.