Gen AI

Embedding Models

An embedding model is trained specifically so that related things land close together, and the strongest retrieval models use two different encoders for short queries and long documents.

An asymmetric encoder maps a short query and a long document through two different learned paths into the same vector space, so a question and its answer can land close together despite looking nothing alike as text

Why Does This Exist?

Embeddings established that meaningful vector positions are the whole point of an embedding — but nothing about a generic neural network guarantees that property on its own. A model trained purely to predict masked words, say, has no particular reason to place "how do I reset my password" close to "forgot login credentials" in its internal representation space; it was never asked to. Getting embeddings that are actually useful for retrieval requires training a model specifically toward that goal, and the specific choices made during that training — what objective, what architecture, what data — determine how well the resulting vectors actually serve a real search system.

This page is about those choices: how an embedding model is actually trained to produce a useful space, and the specific architectural decision — separate encoders for queries and documents — that turned out to matter enormously for retrieval quality.

Think of It Like This

Training a matchmaker instead of a librarian

A librarian catalogs books by subject and author — organized, but organized around the book's own description of itself. A matchmaker does something different: given a person's stated interests, find the other person, described completely differently, who's actually a good fit. The matchmaker's whole skill is bridging two different kinds of description — a short list of preferences on one side, a long, detailed profile on the other — and recognizing when they point at the same underlying match despite looking nothing alike on the page.

A general-purpose language model is closer to the librarian: good at describing text, not specifically trained to bridge a short question against a long answer. An embedding model built for retrieval is trained to be the matchmaker — explicitly optimized so that a short query and the long passage that actually answers it land close together, even though as raw text they share almost nothing.

How It Actually Works

Contrastive training: teaching "close" and "far" directly

Embedding models for retrieval are typically trained with a contrastive objective: given a query and a passage that actually answers it (a positive pair), and a query paired with an unrelated passage (a negative pair), the loss directly pushes positives closer together and negatives farther apart. This is a fundamentally different signal from next-token prediction — not "predict what comes next," but "position these two things correctly relative to each other," which is exactly what retrieval needs.

The quality of negatives matters enormously. Easy negatives (cooking paired with astrophysics) teach little, since almost any representation already separates them. Hard negatives — passages superficially similar to the query but not actually relevant — force the fine-grained distinctions that make retrieval work well. Mining good hard negatives is, in practice, most of the difficulty in training a strong embedding model.

Why queries and documents get different encoders

The diagram above shows the key architectural choice behind the strongest retrieval-focused embedding models: an asymmetric setup, where queries and documents pass through separately trained encoder paths before landing in the same shared vector space. This addresses a real structural mismatch — a query is typically a handful of words, phrased as a question or fragment, while a document is a long, complete passage written to inform, not to be searched against. Forcing both through an identical path asks one representation to be equally good at two different jobs. Letting them diverge, while still training them to land close for matching pairs, consistently beats a single symmetric encoder.

Dimensionality: a real tradeoff, not a free parameter

Dimensionality trades retrieval quality against storage and compute directly — a 1,536-dimension vector captures more nuance than a 384-dimension one from the same family, but costs four times the storage and roughly four times the per-comparison cost at scale. Some models support nested dimensions: a single trained model produces a full-length vector that can also be truncated to a shorter prefix, with graceful rather than catastrophic quality loss, serving multiple cost/quality points without training separate models.

Domain fit is not automatic

A general-purpose embedding model, trained mostly on broad web and book text, can perform noticeably worse on a specialized domain — legal contracts, medical records, internal jargon — where vocabulary and what counts as "related" differ from training. Fine-tuning on domain-specific contrastive pairs is a real, separate decision: it needs labeled positive and negative pairs, and every existing vector in the index has to be recomputed once the model changes.

Show Me the Code

A minimal contrastive loss, showing what "push positives together, negatives apart" actually looks like as an objective.

1import numpy as np2
3
4def cosine(u: np.ndarray, v: np.ndarray) -> float:5    return float(u @ v / (np.linalg.norm(u) * np.linalg.norm(v)))6
7
8def contrastive_loss(query: np.ndarray, positive: np.ndarray, negative: np.ndarray, margin: float = 0.2) -> float:9    """Penalize when the negative is not sufficiently farther than the positive."""10    pos_sim = cosine(query, positive)11    neg_sim = cosine(query, negative)12    return max(0.0, margin - (pos_sim - neg_sim))    # zero once positive beats negative by `margin`13
14
15rng = np.random.default_rng(0)16query = rng.normal(size=16)17close_positive = query + rng.normal(scale=0.1, size=16)     # nudged slightly -- a real "match"18far_negative = rng.normal(size=16)                            # unrelated direction19
20print(round(contrastive_loss(query, close_positive, far_negative), 4))  # -> 0.0 -- already separated enough21print(round(contrastive_loss(query, far_negative, close_positive), 4))  # -> large -- roles reversed, big penalty

Swapping which vector plays "positive" and which plays "negative" flips the loss from near-zero to large — the function is directly rewarding exactly the separation retrieval needs, nothing else.

Watch Out For

Using a symmetric encoder and expecting query-document parity

A model trained to embed similar-length, similar-style text pairs (two sentences from the same article, say) doesn't automatically transfer well to the query-document asymmetry retrieval actually needs. A short, informally phrased question and a long, formally written passage are stylistically very different inputs, and a model never trained on that specific asymmetry can underperform even when it's otherwise a strong general embedding model. Check whether an embedding model was specifically trained for asymmetric retrieval before assuming it's a good fit for a search use case.

Fine-tuning without budgeting for the re-index

Deciding to fine-tune an embedding model on domain data is a decision with a hidden, often underestimated second cost: every vector already stored in the index was computed with the old model, and the new model's space is unrelated to it. Fine-tuning isn't a drop-in improvement — it requires a full re-embedding pass over the entire corpus before the new model can be used safely, which for a large corpus is a real, sometimes multi-hour or multi-day operational cost that has to be planned for, not discovered after the fact.

The Quick Version

Embedding models for retrieval are trained with a contrastive objective, directly pushing matching pairs together and non-matching pairs apart.
Hard negatives — superficially similar but actually irrelevant passages — do most of the work in training a genuinely useful model.
Asymmetric encoders, with separate paths for queries and documents, consistently outperform a single symmetric encoder for retrieval.
Dimensionality trades quality against cost; nested-dimension models let one trained model serve multiple points on that tradeoff.
Domain-specific fine-tuning can meaningfully improve retrieval quality, but it requires re-embedding the entire existing corpus.