Your Vector Search Is Probably Ranking By Length

Every managed vector store asks you the same question during setup, in a dropdown, with no explanation: cosine, inner product, or Euclidean. It is presented as a performance detail. It is not — on unnormalised embeddings, that dropdown decides whether your search ranks by relevance or by document length, and nothing downstream will ever tell you which one you picked.

The mechanism is one line of algebra. A dot product factors as

$$a \cdot b = \lVert a \rVert \, \lVert b \rVert \cos\theta$$

Two magnitude terms and one direction term. Direction is the semantic part — that is what "these mean similar things" reduces to in an embedding space. The magnitudes are, for most embedding models, an artifact: longer inputs and more frequent tokens tend to produce longer vectors, and none of that is a claim about meaning.

So ranking by raw inner product means part of your ranking function is document length.

Why nobody catches it

This is the part that makes it worth a post rather than a footnote. The failure does not look like a failure.

Results stay plausible. They are topically related, mostly sensible, and merely somewhat worse than they should be. No exception is raised, no metric turns amber, and your evaluation set — if it was built by sampling documents you already know are relevant — will not isolate it. The usual symptom is soft and easy to rationalise: a handful of documents that keep appearing across unrelated queries, and a vague sense that retrieval is weaker than the demo suggested.

Teams typically respond by changing the embedding model, adding a reranker, or tuning chunk sizes. Sometimes those help, which is worse, because it buries the actual cause under a fix that works for the wrong reason.

The fix, and where to apply it

Normalise every vector to unit length at ingestion, then use inner product.

That ordering is the whole point. Once $\lVert a \rVert = \lVert b \rVert = 1$, the dot product is the cosine — the length terms are both 1 and drop out. You get direction-only ranking while still using the cheapest kernel the index offers, and you pay the division once per document instead of once per query per candidate. This is precisely why the managed services' "inner product" and "cosine" metrics return identical rankings on normalised data, and why their docs quietly recommend normalising.

Concretely:

• Normalise in your ingestion pipeline, not in your query path. Dividing by the L2 norm is the operation, and np.linalg.norm is the function — not sqrt(sum(x**2)), which overflows on large components and underflows on small ones.
• Assert it. After writing, sample vectors out of the index and check the norm is 1.0 to within tolerance. This is a two-line test that permanently closes the failure mode, and almost nobody writes it.
• Guard the zero vector. An empty or failed embedding has no direction and cannot be normalised. Decide whether that is a skip or an error, but decide.
• Re-index when you change embedding models. A new model has a different magnitude distribution, so a mixed index is a mixed metric.

For unit-length vectors, Euclidean distance and cosine similarity produce the same ranking, since $\lVert a - b \rVert^2 = 2 - 2\cos\theta$. That is a useful thing to know: once you have normalised, the dropdown genuinely stops mattering, and you can stop thinking about it. Which was the goal.

The general lesson

Managed services move real complexity behind sensible-looking defaults, and mostly that is the value you are paying for. But a default that is only correct under a precondition the service does not enforce is a trap, and "inner product" on unnormalised embeddings is exactly that shape.

When a hosted product asks you to choose something you have not thought about, that dropdown is where the assumption lives. It is worth ten minutes.