Hypothetical Document Embeddings (HyDE)
You ask a short vague question, the model drafts a fictional answer, and you search with that draft because answer shaped text lands closer to the real passage than your question does.
Why Does This Exist?
Dense retrieval ranks passages by the angle between vectors. Your query is short and interrogative, something like "how do I fix my login?" The passage that answers it is long and declarative, something like "open Okta settings." Different words, different shape. So the two vectors land apart even when the meaning matches. That mismatch is called the query to document gap, and that's why a store can hold the right answer yet return the wrong passages.
You could retrain the encoder, but that needs labels and time. Hypothetical document embeddings (HyDE) takes the zero-shot route instead. It asks a chat model to draft what an answer might look like, then searches with the draft. The draft may get facts wrong. It still carries answer style wording, which is what the search actually compares.
This page covers that draft then search trick inside a RAG pipeline. For the encoder itself, see embedding models. For how closeness is scored, see similarity metrics.
The one idea you need first: an encoder turns text into a list of numbers, a vector. Cosine similarity measures the angle between two vectors: 1.00 means identical direction, 0.00 means unrelated. Retrieval keeps the passages with the largest cosine to the search vector.
Think of It Like This
A police sketch artist
A witness gives you one vague sentence: "he was tall with a scar on his cheek." That sentence is your query. You cannot match a sentence against a wall of mugshots, so you call a sketch artist. The artist is the chat model. The artist draws a full face from that sentence, inventing hair and eyes to fill the gaps. That drawing is the fictional document.
Now you match face against faces. The drawing finds the real mugshot because pictures compare well with pictures, even though half the invented details are wrong.
Where it stops: the artist can draw the wrong person with total confidence. If the witness meant the Python language and the artist draws a snake, you search the wrong wall. HyDE fails the same way, and for the same reason you never convict on the sketch alone. The final answer must come from the real passage, never from the fiction.
How It Actually Works
HyDE runs before normal retrieval, in two steps. Generation first, then search with the generated text.
The two steps
Call the user query . That is just the raw question string. Call the encoder . That is the model mapping any text to a vector in , where means a list of real numbers and counts how many numbers each vector holds.
First, prompt a chat model with and ask for a short passage that answers it. Say guessing names is fine. It returns one or a few hypothetical documents , where each is one drafted passage and counts how many drafts you asked for. Encode each draft as , meaning the vector for draft . The HyDE search vector is the average of those draft vectors. With a single draft, is just that one vector. No labels, no retraining. That's why people call it zero-shot.
Second, score every stored chunk , where is chunk number in your corpus and is its precomputed vector from index time. The score for chunk is the cosine between the HyDE vector and the chunk vector:
In plain words: the dot product on top, divided by the two vector lengths below. Keep the top chunks, where is how many passages you pass onward. Then run normal generation grounded in those real chunks. The fiction never reaches the final prompt as fact. It only steers the search.
Worked example
Use tiny 2-D vectors so you can check every step. Let the query vector be , the true passage be , and the fictional answer be . Think of as the draft that guessed "portal" where the truth says "Okta": wrong noun, right shape.
Query against passage:
Fiction against the same passage:
So the raw question sits at 0.800 while the fiction sits at 0.997. Same passage, same metric. The only change is what you search with. Those are the two numbers drawn in the diagram above.
Why the fiction helps, and when it does not
Encoders place same shaped texts near each other. A question and its answer differ in wording and form, but two answers share vocabulary, length, and structure. The fiction borrows that shape, so its vector starts the race halfway to the target. That's the whole bet.
Name the price plainly. HyDE buys recall on short vague queries and pays with a full chat model call: extra latency, extra cost, one more part that can break. Skip it when the query already reads like a passage, when the lookup is an exact code or product name, or when your latency budget has no room. For exact terms, hybrid search beats a fiction. For cleaning up the question itself, query rewriting is the cheaper sibling: it rewrites the question, while HyDE converts the question into an answer.
The assumption underneath: the draft's terms point roughly toward the truth. When it locks onto the wrong entity, the vector marches confidently the wrong way, and no reranker downstream can rescue passages that never made the shortlist.
Code
The point of this snippet is the check behind the diagram. It recomputes both cosines from the worked example with plain vectors, so you can see the gap is arithmetic, not marketing.
import numpy as np
def cos_sim(a: np.ndarray, b: np.ndarray) -> float: return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b))) # angle between vectors as one number
q: np.ndarray = np.array([1.0, 2.0]) # short question embeddingd: np.ndarray = np.array([2.0, 1.0]) # true passage embeddingh: np.ndarray = np.array([2.0, 1.2]) # fictional answer embedding
print(f"{cos_sim(q, d):.3f}") # -> 0.800print(f"{cos_sim(h, d):.3f}") # -> 0.997The prints agree with the hand computation: 0.800 for the raw question, 0.997 for the fiction. Swap in your own encoder outputs and you'll see whether HyDE helped that query.
Watch Out For
The fiction names the wrong entity and drags retrieval with it
Symptom: your top hits all share a term from the draft that was never in the query. The user asked about the Python language and every result discusses snakes, because the draft guessed the animal and the search obeyed.
Fix: keep the draft short and cool-headed with a low temperature, ask for two or three drafts and average their vectors instead of trusting one, and log the draft text during development so you can see the wrong guess. If wrong entity guesses keep happening, that query belongs on hybrid search, not HyDE.
Paying for an LLM call on queries that needed no help
Symptom: latency and cost climb while quality stays flat. Long detailed queries and exact identifier lookups score well with direct embedding, so the extra generation step adds nothing but delay.
Fix: route only short vague queries through HyDE and embed the rest directly. A simple length or intent check in front saves most of the cost. Never wrap every query in HyDE by default and hope the budget sorts itself out.
The Quick Version
- HyDE drafts a fictional answer to your question and searches with the draft, because answer shaped text lands closer to stored passages than the question does.
- The draft's facts may be wrong; the search still works when its wording and structure point the right way.
- In the toy example the raw question scores 0.800 against its passage while the fiction scores 0.997 under the same cosine metric.
- It costs one extra chat model call per query, so it trades latency and money for recall on short vague questions.
- It fails when the draft picks the wrong entity, and it adds nothing for long detailed queries or exact term lookups.