Distance and Similarity Metrics

Why Does This Exist?

Every vector database setup asks you to pick a metric from a dropdown, offers no explanation, and then never mentions it again. That dropdown decides your search quality, and the three options do not merely differ in scale — they can rank the same candidates in opposite orders.

The demonstration is in the diagram. Take $a = [1, 1]$ and $b = [10, 10]$: cosine similarity says they are identical, and Euclidean distance says they are $12.73$ apart. Now take $c = [1, 0]$ and $d = [0, 1]$: Euclidean says $1.41$, closer than the first pair, while cosine says $0$ — completely unrelated. The two metrics have inverted the ranking.

This is the page behind the most common retrieval bug there is, and behind a subtler one: k-nearest neighbours and k-means both silently assume Euclidean, so feeding them unscaled features means the largest-range feature decides every neighbour.

Think of It Like This

How It Actually Works

Four measures cover nearly all practical use.

Euclidean (L2) distance is straight-line separation:

In plain words: subtract, square, sum, root. It is sensitive to magnitude, and it is the default assumption of k-NN, k-means, and most clustering.

Cosine similarity is the angle only:

In plain words: the dot product divided by both lengths, so length cancels. It is bounded in $[-1, 1]$ and completely blind to length. It is the default for embeddings. Note "cosine distance" usually means $1 - \cos\theta$, which is not a true metric — it violates the triangle inequality — so it cannot be used in algorithms that require one.

Dot product is $a \cdot b$, unnormalised. It mixes direction and both magnitudes, which is the trap: a long irrelevant vector beats a short relevant one. It is fast, and correct only when your vectors are already unit length. Lengths themselves are measured with norms.

Manhattan (L1) distance is $\sum_i \lvert a_i - b_i \rvert$. It does not square, so one large discrepancy punishes less than Euclidean. It is sometimes preferred in high dimensions for that reason.

The identity that resolves most of it

For unit-length vectors, Euclidean distance and cosine similarity give the same ranking:

In plain words: once both vectors have length $1$, squared distance is a decreasing function of cosine similarity, so sorting by one sorts by the other. Expand $\lVert a - b \rVert^2 = a \cdot a - 2 a \cdot b + b \cdot b = 1 - 2\cos\theta + 1$ to see it. This is why normalising at ingestion is such a clean move: normalise once, then the cheap dot product is cosine similarity, and Euclidean agrees with both.

Choosing

• Embeddings from a neural model — cosine, or dot product on pre-normalised vectors. Magnitude is a training artifact, not meaning.
• Physical measurements in real units — Euclidean, after standardising features so no unit dominates.
• Sparse count vectors — cosine. Document length would otherwise dominate.
• Recommender inner products — dot product deliberately, because in matrix factorisation the magnitude encodes popularity and you may want that.
• Sets rather than vectors — Jaccard, the size of the intersection over the union. Different family, and the right answer for tags or shingles.

High dimensions change the picture

In very high dimensions distances concentrate: the ratio between the nearest and farthest neighbour of a random point approaches $1$, so "nearest" becomes weakly meaningful. This is one face of the curse of dimensionality, and it hits Euclidean harder than cosine.

Real embeddings are less affected than random vectors, because they occupy a low-dimensional manifold rather than filling the space. But the effect is why cosine dominates for high-dimensional embeddings, and why approximate nearest-neighbour indexes can afford to be approximate.

Worked example

Take $a = [1, 1]$ and $b = [10, 10]$, so $b = 10a$ — exactly the same direction.

Hand-computed: $\lVert a \rVert = \sqrt{1 + 1} = \sqrt{2} \approx 1.414$ and $\lVert b \rVert = \sqrt{100 + 100} = \sqrt{200} \approx 14.142$. Dot product: $1 \cdot 10 + 1 \cdot 10 = 20$.

Perfect similarity, since they are parallel. But Euclidean distance:

Very far apart. Same pair, opposite verdicts.

Now invert it with $c = [1, 0]$ and $d = [0, 1]$. Euclidean distance is $\sqrt{1 + 1} = \sqrt{2} \approx 1.414$ — much closer than the first pair. Cosine similarity is $0 / (1 \times 1) = 0$ — orthogonal, no similarity at all.

So by Euclidean, $(c, d)$ at $1.41$ is far closer than $(a, b)$ at $12.73$. By cosine, $(a, b)$ at $1.0$ beats $(c, d)$ at $0.0$. The ranking has completely reversed, from four vectors and no ambiguity.

Finally, normalise and watch the disagreement vanish. $\hat{a} = \hat{b} = [1/\sqrt{2}, 1/\sqrt{2}] \approx [0.707, 0.707]$, so their distance becomes $0$ and their cosine stays $1$. Both metrics now agree they are identical.

Code

The snippet reproduces every number above, then shows the k-NN failure that unscaled features cause.

1import numpy as np2
3
4def cosine(u: np.ndarray, v: np.ndarray) -> float:5    return float(u @ v / (np.linalg.norm(u) * np.linalg.norm(v)))6
7
8a, b = np.array([1.0, 1.0]), np.array([10.0, 10.0])  # same direction9c, d = np.array([1.0, 0.0]), np.array([0.0, 1.0])  # orthogonal10
11print(round(cosine(a, b), 4), round(float(np.linalg.norm(a - b)), 4))  # -> 1.0 12.727912print(round(cosine(c, d), 4), round(float(np.linalg.norm(c - d)), 4))  # -> 0.0 1.414213
14# Normalising makes the two metrics agree: ||a-b||^2 == 2 - 2*cos.15u, v = a / np.linalg.norm(a), b / np.linalg.norm(b)16print(round(float(np.linalg.norm(u - v) ** 2), 10),17      round(2 - 2 * cosine(u, v), 10))  # -> 0.0 0.018
19# Unscaled features let the widest-range column decide every neighbour.20X = np.array([[1.0, 1000.0], [2.0, 1000.0], [1.0, 2000.0]])  # (3, 2)21print(round(float(np.linalg.norm(X[0] - X[1])), 3),22      round(float(np.linalg.norm(X[0] - X[2])), 3))  # -> 1.0 1000.0

Every number matches the hand calculation. The last block is the k-NN failure: rows 0 and 1 differ by $1$ in feature one, rows 0 and 2 by $1000$ in feature two, and Euclidean treats the second as a thousand times more important purely because of units.

Watch Out For

The Quick Version

• Euclidean is straight-line distance and is magnitude-sensitive. Cosine is angle only, bounded in $[-1, 1]$.
• Raw dot product mixes direction with both magnitudes, so long vectors win. It is correct only on pre-normalised vectors.
• The metrics can rank pairs in opposite orders: $[1, 1]$ vs $[10, 10]$ is cosine $1.0$ and distance $12.73$.
• For unit vectors, $\lVert a-b \rVert^2 = 2 - 2\cos\theta$, so both give the same ranking. Normalise once at ingestion and the choice stops mattering.
• "Cosine distance" $1 - \cos\theta$ is not a true metric — it fails the triangle inequality.
• Embeddings take cosine. Physical units take standardised Euclidean. Sparse counts take cosine. Sets take Jaccard.
• In high dimensions distances concentrate, so "nearest" weakens. Cosine holds up better than Euclidean.
• Standardise features before any distance-based method, or the widest-range feature decides every neighbour.

What to Read Next

• Dot Product
• Eigenvalues and Eigenvectors
• Kernel Functions