Semantic Search
Keyword search looks for the same words. Semantic search looks for the same meaning, so a query about a leaky pipe can find a passage about dripping plumbing with no words in common.
Why Does This Exist?
You've asked a help center how to fix a leak and gotten nothing, even though the answer sits one click away under different words. Your query says "how do I fix a leaky pipe" and the passage says "repairing dripping plumbing". A keyword index files those as unrelated. No shared tokens, no match, no help.
That miss is the vocabulary mismatch problem. People say one thing dozens of ways, and BM25 and inverted indexes only match literal tokens. Semantic search flips the comparison from words to meaning: every text becomes a point in one vector space, and closeness means relatedness. You need it whenever queries and documents might paraphrase each other.
This page covers the dense loop itself: one encoder, cosine ranking, and one worked ordering. How those vectors get trained is embedding models. How the ruler works is similarity metrics. Storing the vectors at scale is vector databases, and the pipeline that consumes the winners is RAG architecture.
The one idea you need first. An embedding turns any text into a list of numbers positioned so related meanings land close together, and cosine similarity scores that closeness from the angle between two lists. Length is ignored; direction is the story. The full treatment with its own example is embeddings.
Think of It Like This
A repair wall sorted by job, not by name
You walk into a hardware shop holding a broken washer from under your sink. You do not know its name. One aisle lists every part alphabetically, so "washer" sits far from "seal" even when both would stop your drip. Useless to you.
The back wall works differently. Every fix is grouped by the job it does. All drip stoppers sit together no matter what each is called. You hold up your broken piece, the staffer finds the spot it belongs, and hands you the nearest handful. Query in hand, wall of candidates, nearest wins. That handful is top-k.
Where it stops: the staffer understands plumbing. An encoder doesn't understand anything. It only places texts where its training pairs taught it to place them, the axes carry no human labels, and closeness is learned geometry rather than comprehension. Wrong pairs teach the wrong shelf order.
How It Actually Works
One model, two lanes
Write the encoder as . It maps your query to a query vector , where is a list of numbers standing in for what you asked. It maps chunk to a chunk vector , the same kind of list standing in for that passage. Same on both sides, or the comparison means nothing.
Indexing runs offline. Split documents into chunks, short passages sized so one vector holds one idea, embed each chunk once, and store the vectors. How to split is chunking strategies. Retrieval runs per question. Embed the query with the same model, score it against the stored vectors, keep the top , where is simply how many chunks survive into the answer. Small stays cheap and focused. Larger covers more ground and drags in noise.
Cosine decides the order
The score for chunk is the cosine of the angle between the two lists:
Here is the dot product (multiply the lists element by element, then add), and are the lengths of the two lists (square each entry, add, take the square root), and is the resulting score. Near 1 means almost the same direction. Near 0 means unrelated.
You buy paraphrase recall and you pay in literal precision. Exact codes, names, and quoted errors can drown among semantic neighbours, since vectors blur rare tokens. So don't use dense retrieval alone when the answer must match a token exactly, like a part number or an error string. That condition calls for keyword search, and when both signals matter on your corpus the answer is hybrid search. The method assumes similarity tracks relevance and your queries resemble the encoder training pairs. Drifted slang, heavy jargon, or a new domain breaks that assumption, and no larger repairs it.
Worked example: three passages, one query
Toy 2-D vectors so you can check every step. Real models use hundreds of dimensions; the arithmetic is identical.
- Query , standing in for "how do I fix a leaky pipe". Length .
- D1 , standing in for "repairing dripping plumbing". Zero content words shared with the query.
- D2 , standing in for "how to fix a paint bucket leak". Shares words, wrong job.
- D3 , standing in for "electric car battery range". Unrelated.
Dots first: . Next . Then .
Lengths: . Next . Then .
Now the cosines:
- D1:
- D2:
- D3:
Ranking: D1 first, D2 second, D3 last. With the system returns the paraphrase that shares no words and skips the decoy that shares several. Those are the three arrows and the three scores drawn above, and the snippet below reruns them.
Code
The point of this snippet is the ranking, not the formula: three cosines from scratch on the worked fixture, deciding which passage wins.
import numpy as np
def cosine(a: np.ndarray, b: np.ndarray) -> float: """Angle score between two vectors, near 1 for same direction.""" return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))
query = np.array([1.0, 0.0]) # how do I fix a leaky piped1 = np.array([0.9, 0.3]) # repairing dripping plumbingd2 = np.array([0.6, 0.8]) # how to fix a paint bucket leakd3 = np.array([0.1, 0.9]) # electric car battery range
scores: dict[str, float] = { "D1": cosine(query, d1), "D2": cosine(query, d2), "D3": cosine(query, d3),}ranked = sorted(scores, key=lambda k: scores[k], reverse=True)print([f"{k} {scores[k]:.2f}" for k in ranked]) # -> ['D1 0.95', 'D2 0.60', 'D3 0.11']D1 wins at 0.95 despite sharing no content words with the query. D2 trails at 0.60 with its shared words, and D3 sits last at 0.11.
Watch Out For
Exact codes vanish into the average
Symptom: a query like "TX-9942-B keeps jamming" returns general printer advice while the exact row with that serial sits unretrieved. Or "E4021 at checkout" brings back payment guides with no code in sight. Nothing errors; the right document simply never makes top-k.
The cause is structural. Embeddings average every token into one direction, so a rare code contributes little against the pull of common words around it. The model was never trained to preserve serials verbatim. Fix it by pairing signals: run BM25 and inverted indexes for the literal half and fuse with dense ranks as hybrid search does, or gate exact fields with a metadata filter before vector search. Never trust pure dense retrieval for IDs, names, or quoted errors.
A new model turns the old index into noise
Symptom: relevance drops right after an embedding upgrade, with no error anywhere. Scores bunch together or look random, and queries that worked last week start missing. The code changed one string, the model name, and everything else stayed put.
Two models build unrelated coordinate systems, so a query from the new space compared against documents from the old one measures nothing. Dimensions can even match and still mean different things. Fix: version-pin the encoder, re-embed the entire corpus with the exact model and version the queries use, and refuse writes from anything else. Budget that pass before you promise the upgrade, since it is the real cost. The training side of this rule is embedding models.
The Quick Version
- Semantic search embeds the query and every passage with one encoder and returns the closest vectors, so meaning decides rank instead of shared words.
- Cosine scoring divides the dot product by both lengths, so direction decides the order and vector length does not.
- On the worked fixture the paraphrase scores 0.95, the word overlap decoy scores 0.60, and the unrelated passage scores 0.11.
- Dense retrieval blurs exact codes and names, so IDs and error strings still need keyword search or hybrid search.
- Query vectors and stored vectors must come from the same model version, or every distance in the index is noise.