Your Vector Search Ranks By Length

Why Does This Exist?

When developers build Retrieval-Augmented Generation (RAG) applications, they usually rely on a managed vector database (like Pinecone, Weaviate, or a cloud provider's PostgreSQL pgvector extension). The workflow seems simple: embed the text, store the vectors, and run a nearest-neighbor search.

But a huge percentage of these applications are suffering from terrible recall simply because they left the default distance metric on L2 (Euclidean distance) instead of Cosine similarity.

Think of It Like This

Imagine trying to find two people who agree on politics. Cosine similarity checks if they are pointing in the same direction. L2 distance checks how physically close they are. If Person A whispers their opinion and Person B shouts the exact same opinion through a megaphone, they are pointing in the same direction, but their magnitude (volume) is different. L2 distance thinks they are enemies.

How It Actually Works

Text embedding models map semantic meaning to direction. The length (norm) of the vector often just represents the density or word count of the original chunk.

If you use Euclidean distance (L2) to search, the database penalises vectors that have a different length, even if they point in the exact same semantic direction. A short query vector will fail to retrieve a long, highly relevant document vector simply because the document vector's magnitude pushes it too far away in the multidimensional space.

Cosine similarity ignores the length of the vectors and only calculates the angle between them. For unnormalized text embeddings, this is almost always the correct metric.

Watch Out For

Some embedding models (like OpenAI's text-embedding-3) return normalized vectors by default (where every vector has a length of 1). If your vectors are strictly normalized, L2 and Cosine similarity will rank results in the exact same order. However, if you are using an open-weight model that does not normalize outputs, and your cloud database defaults to L2, your retrieval is compromised.

(Correct as of August 2026).

The Quick Version

If your embedding model does not normalize its vectors, using the default L2 distance metric in your managed vector database will ruin your search recall. Always verify whether your specific model requires Cosine similarity.

What to Read Next

Check the rag-architecture concept page to see where the vector database fits into the broader retrieval pipeline.