Clustering Evaluation: Scores Without Labels

No answer key exists for clusters, so each score defines good in its own way. The same grouping can please one score and fail another.

Silhouette rewards compact round groups, so a wrong straight cut through two crescents scores 0.46 while the true arcs score 0.31

Why Does This Exist?

Suppose you cluster soil samples into management zones so a spreader can vary its rate as it drives. Someone asks if the zones are any good. With a classifier you would check answers. Here nobody wrote down the true zone for each sample.

So every check is indirect, and each one carries its own idea of what a cluster looks like. Score a result with a metric that shares your algorithm's bias and it approves of itself. A round blob method judged by a round blob score always looks right.

This page covers how to judge a grouping once you have one. K-means and DBSCAN cover how to make one, and model evaluation covers the supervised case where labels exist and a held out set settles arguments.

Think of It Like This

The round hole gauge

A workshop checks parts by dropping each one through a 40 mm round hole. Parts that fall through pass. It is fast, it needs no judgement, and within a year everyone calls it the quality test.

Then the shop starts making brackets. A perfectly machined bracket will not go through, so it fails. Nothing is wrong with the bracket and nothing is wrong with the hole. The hole only ever measured roundness, and someone read roundness as quality.

That is silhouette scoring a crescent. It is not broken. It answers whether groups are compact and well separated, and a correct crescent is neither. There is also a second check the shop never runs: hand the same batch to another shift and see if the same parts pass. If the verdict moves with who is on duty, the process is not a process.

Where it stops: holes and shifts cannot tell you if the zones matter in the field. Only a difference in something you left out, like yield, can do that.

How It Actually Works

Three families answer three different questions. Internal scores see only the distances. External scores borrow labels you happen to have. Stability asks if the answer survives a new sample.

Internal scores see only the distances

For each sample, let $a$ be the mean distance to the other samples in its own group, and let $b$ be the mean distance to the samples in the nearest other group. The silhouette of that sample is:

s = \frac{b - a}{\max(a, b)}

It runs from minus 1 to 1. Near 1, the sample sits deep inside its own group. Near 0, it sits on a boundary. Negative means it would be happier elsewhere. Average $s$ over every sample for the score of the whole grouping.

Two cousins use the same ingredients. Calinski Harabasz is a ratio of between group variance to within group variance, so larger is better and it has no fixed range. Davies Bouldin averages each cluster's similarity to whichever cluster it most resembles, so smaller is better. All three are built from distances to centres or averages of pairwise distances, so all three are happiest with compact, round, well separated blobs.

External scores borrow labels you happen to have

Sometimes you do have labels, like last season's hand drawn zones. The adjusted Rand index counts pairs of samples the two groupings agree about, placed together or kept apart, then removes the agreement chance alone would give. Random grouping scores 0. Perfect agreement scores 1. Normalised mutual information asks how much knowing one grouping tells you about the other, from 0 to 1.

Both validate a method on ground you know. Neither judges a grouping you plan to use. If labels exist for the rows you care about, train a classifier instead.

Worked example: four points on a line

Take points at 0, 1, 5 and 6 on a line, grouped as $[0, 1]$ and $[5, 6]$ . For the point at 0, $a$ is the distance to 1:

a = 1.0

$b$ is the mean distance to 5 and 6:

b = \frac{5.0 + 6.0}{2} = 5.5

So its silhouette is:

s = \frac{5.5 - 1.0}{5.5} = \frac{4.5}{5.5} \approx 0.82

The point at 1 gives $a = 1.0$ and $b = (4.0 + 5.0) / 2 = 4.5$ , so $s \approx 0.78$ . The mirror on the right gives 0.78 and 0.82. The mean over all four is about 0.80. Tight pairs far apart score near 1, as they should.

Why round blobs win on crescents

Now the same formula on shape it was never built for. Take two interlocking arcs of 200 points each, radius 3, the second shifted by $(1.6, 1.2)$ . The true arcs score 0.306. A straight left right split at the mean $x$ of 0.8 scores 0.464. The wrong answer wins by about 0.16.

The diagram draws exactly this: two arcs, one straight cut, neighbours on one arc landing on opposite sides, 0.46 against 0.31. Maximise silhouette here and you ship the straight cut every time. Hierarchical clustering with the wrong linkage and Gaussian mixture models with round covariances fall into the same trap.

The check people skip is stability. Resample your rows, split them in half, recluster, and see if the same samples land together, with adjusted Rand between the two runs as the ruler. Repeat many times. A grouping that reshuffles across resamples describes one sample, not the field. The bar that settles it is downstream: do the groups differ on something you never clustered on, like last season's yield map.

Code

The point is the check: rebuild the four point line from above and confirm the hand computation of 0.82 for the first point.

1from math import hypot2
3pts: list[tuple[float, float]] = [(0.0, 0.0), (1.0, 0.0), (5.0, 0.0), (6.0, 0.0)]4
5a: float = hypot(1.0, 0.0)6b: float = (5.0 + 6.0) / 2.07s: float = (b - a) / b8print(f"{a:.1f} {b:.1f} {s:.2f}")  # -> 1.0 5.5 0.82

The printout matches the hand steps: own distance 1.0, other mean 5.5, silhouette 0.82.

Watch Out For

Picking k by maximising silhouette

The loop looks responsible: fit k from 2 to 12, keep the best silhouette. Then the zones come back as two giant round blobs while the field visibly has four long strips, and the four zone answer scored lower.

Symptom: the metric picking its favourite shape, not a discovery about your data. Fix: plot silhouette across k next to a stability curve and the BIC from a mixture model, note where they disagree, then pick k for a reason you can say aloud. Four because the spreader has four rate settings beats five because 0.41 beat 0.39.

Trusting labels that stop at deployment

You score 0.93 adjusted Rand against last year's hand drawn zones on two fields and ship to forty. The score was real. It measured that drawn zones can be recovered from the same soil columns.

Symptom: high external score, weak zones on every new field. Fix: treat an external score as a pipeline check only, then validate where you will use it with resample stability plus a gap in a held out column. If labels exist everywhere you need them, stop clustering and train a classifier.

The Quick Version

Clustering has no answer key, so each score defines good in its own way.
Silhouette of one sample is (b minus a) over the larger of the two, from minus 1 to 1, with 0.82 for an edge point in our four point line.
Calinski Harabasz grows with round separated blobs while Davies Bouldin shrinks, so both favour the same shape as silhouette.
Adjusted Rand removes chance agreement and gives 0 to random groupings and 1 to perfect agreement.
Two true arcs score 0.31 while a wrong straight cut scores 0.46, which shows internal scores punish correct non round shape.
Stability across resamples plus a gap in a held out column decides if zones are worth keeping.