Maximal Marginal Relevance (MMR)
Top-k by relevance alone hands you the same idea five times. MMR picks the next chunk by what it adds, not just how well it matches.
Why Does This Exist?
Say you ask a RAG system for the causes of an event and it retrieves the top five chunks. All five come from the same section of one document. Each scores high against your question, so plain top-k thinks it did great. But you've burned five context slots on one paragraph restated five ways, and your model still misses the second cause and the counterargument it needed.
Maximal Marginal Relevance (MMR) exists for that moment. It is a reranking step that builds your final set one pick at a time. Each new pick earns points for matching your question and pays a tax for overlapping picks you have already kept. You have seen the two-stage idea before in reranking, and where this step sits in the pipeline in RAG architecture.
You only need one prerequisite here. Query and chunks live as points in the same embedding space, and cosine similarity tells you how close two points are. If that is hazy, start with embeddings, then come back. This page covers the greedy pick rule with numbers you can check by hand.
Think of It Like This
Packing books for a long flight
Your carry-on holds three books. You pick the first by pure desire: the thriller everyone recommends. Now you eye the sequel. You want it, but it repeats the same characters and twists you just packed. The travel essays score a little lower on raw desire, yet they add long airports, strange cities, quiet mornings. All new ground. You pack the essays second and leave the sequel on the shelf.
That is the trade. Relevance is how much you want the book. Redundancy is how much it repeats books already in the bag. The bag is your selected set. Your taste for variety is the lambda dial. And you pack greedily, one book at a time, because repacking every combination would take all day.
Where it stops: you judge overlap by gut. MMR can't. It needs two numbers on the same zero to one scale, computed from the same embeddings. And your taste shifts mid-flight while lambda stays fixed until you retune it.
How It Actually Works
MMR runs after fast retrieval. A vector search over vector databases hands you a candidate pool, call it , with twenty or so chunks. You want of them, with much smaller than the pool. MMR builds a selected set, call it , starting empty and adding one chunk per round until holds .
The selection rule
At each round you pick the candidate that maximizes this:
In plain words, on first use:
- is your query, as an embedding.
- is one candidate chunk you are scoring.
- means candidates not yet picked.
- is the set you have already picked.
- is relevance: cosine similarity between candidate and query, zero to one.
- is redundancy: the similarity to the closest chunk already in . One near-twin is enough to tax it. When is empty there is no tax.
- is your dial between zero and one. At one you get plain top-k. At zero you ignore the query entirely, which you never want alone.
The first pick is pure relevance. Every pick after subtracts overlap. That max matters. A candidate that duplicates any single kept chunk gets taxed, even if it differs from the rest.
A worked pick with real numbers
Take four candidates with these query similarities:
- at , at , at , at
And these chunk to chunk similarities:
- , near-duplicates
- , different angles
- , different again
- ,
Set , so relevance gets weight and redundancy gets weight . Round one has no penalty, so you pick at . Now .
Round two:
- :
- :
- :
You pick at . The diagram above draws exactly this round. loses despite higher raw relevance because its overlap with drags it to .
Round three, with :
- redundancy is , score stays
- redundancy is , score is
You pick . Final set is . Plain top-k would have handed you . You swapped a duplicate for a chunk that covers new ground. That swap is the whole method.
The lambda dial and the price you pay
Lower lambda buys coverage with relevance. That is the trade, stated plainly: broader evidence, lower average query match. Push it too far and you retrieve interesting strangers that do not answer the question.
So don't reach for MMR on autopilot. Skip it when , since one pick has nothing to be diverse from. Skip it when your shortlist is already deduped by better chunking strategies. Skip it for narrow factoid questions where three agreeing passages beat three different ones. And note what it assumes: both similarities live on the same zero to one scale from the same embeddings. Mix raw lexical scores with cosine and lambda stops meaning what you think. Hybrid search solves a different job, merging word match with meaning match, and you can stack the two: fuse first, diversify second.
Cost is one more reason to keep it second-stage. Each round compares each leftover against each kept pick, so the work grows with pool size times picks. Rerank a shortlist of tens, never the whole index.
Code
This snippet replays the worked example with nothing but dictionaries, so you can trace each score against the hand math above.
relevance: dict[str, float] = { "D1": 0.90, "D2": 0.88, "D3": 0.85, "D4": 0.60,}
pair_sim: dict[tuple[str, str], float] = { ("D1", "D2"): 0.95, ("D1", "D3"): 0.20, ("D1", "D4"): 0.10, ("D2", "D3"): 0.18, ("D3", "D4"): 0.12,}
def get_sim(a: str, b: str) -> float: if a == b: return 1.0 return pair_sim.get((a, b), pair_sim.get((b, a), 0.0))
def mmr_rank(candidates: list[str], lam: float = 0.7, k: int = 3) -> list[str]: selected: list[str] = [] remaining: list[str] = list(candidates) while remaining and len(selected) < k: best_doc: str = remaining[0] best_score: float = float("-inf") for doc in remaining: rel: float = relevance[doc] red: float = 0.0 if not selected else max(get_sim(doc, s) for s in selected) score: float = lam * rel - (1.0 - lam) * red if score > best_score: best_score = score best_doc = doc selected.append(best_doc) remaining.remove(best_doc) return selected
print(mmr_rank(["D1", "D2", "D3", "D4"])) # -> ['D1', 'D3', 'D4']print([round(0.7 * 0.88 - 0.3 * 0.95, 3), round(0.7 * 0.85 - 0.3 * 0.20, 3)]) # -> [0.331, 0.535]The first line returns the diverse set, not plain top-k. The second line prints the round-two scores for and , matching the hand calculation and the diagram.
Watch Out For
Mixing score scales so lambda lies
You fuse lexical scores with cosine similarities and MMR either never diversifies or picks strangers. Symptom: moving lambda from 0.3 to 0.8 changes nothing, or the winners look random. Raw BM25 and dot products are unbounded while cosine sits between zero and one, so one term drowns the other.
Normalize first. Put both similarities on zero to one, computed from the same embedding space, and check a few triples like the ones above before you tune. Don't tune lambda until raw relevance and raw overlap share a scale.
Tuning for variety until answers drift
You crank diversity and citations look varied while answers go vague. Symptom: the model cites the 0.60 chunk and ignores the 0.88 it needed, and side-by-side evals prefer plain top-k. You bought coverage you did not need.
Raise lambda back toward relevance and compare the two sets on real questions, not instinct. If the duplicate-heavy set answers better, your chunks were not redundant in the first place.
The Quick Version
- MMR builds the final set one pick at a time, scoring each leftover by query relevance minus overlap with picks already kept.
- The first pick is pure relevance, since an empty selected set charges no redundancy tax.
- The redundancy term uses the closest kept chunk, so one near-twin is enough to sink a candidate.
- Lambda sets the trade: one means plain top-k, zero ignores the query, and the middle buys coverage with relevance.
- MMR helps when top-k wastes context on near-duplicates and hurts when k equals one or the shortlist is already diverse.
- Both similarity scores must share the same zero to one scale, or the lambda dial stops meaning what you think.