A retrieval-augmented generation (RAG) system can retrieve highly relevant passages and still build poor context. The problem appears when several top results say almost the same thing. Sending all of them to the language model consumes context without adding much evidence, while a slightly lower-ranked passage containing a different useful fact may be excluded.

Maximum marginal relevance (MMR) is a selection strategy for this situation. Instead of choosing passages only by their relevance to the query, MMR repeatedly chooses a passage that is both relevant and sufficiently different from passages already selected.

This article develops the mental model behind MMR, works through a small numerical example, and explains where the method helps, what its trade-offs are, and when simpler top-k retrieval is preferable.

The problem is redundancy, not relevance alone

Suppose a developer asks a product-support assistant:

How do I rotate an API key, and what happens to the old key?

A retriever returns these passages in similarity order:

A: Open Settings > API Keys and choose Rotate.
B: To rotate a key, open Settings > API Keys and select Rotate.
C: The old key remains valid for 15 minutes after rotation.
D: API keys can be renamed from the same settings page.

If the context budget holds only two passages, ordinary top-2 selection chooses A and B. Both are relevant, but they contribute nearly the same information. Passage C is also relevant and adds the missing answer about the old key.

The useful set is therefore closer to A + C than A + B.

This distinction matters whenever retrieval candidates contain near-duplicates, overlapping chunks from the same document, repeated documentation, or several passages covering one part of a multi-part question.

Think of MMR as greedy coverage

MMR starts with a candidate set produced by a normal retrieval stage. It then constructs the final set one passage at a time.

At each step, it asks two questions:

  1. How relevant is this candidate to the query?
  2. How similar is this candidate to something I have already selected?

A common MMR scoring form is:

score(d) = lambda * sim(query, d)
           - (1 - lambda) * max sim(d, s)
                                 s in selected

Here, d is a candidate passage and selected is the set already chosen. The first term rewards query relevance. The second penalizes redundancy with the most similar selected passage.

lambda controls the balance. With values near 1, selection behaves more like relevance-only ranking. Lower values give diversity more influence. The useful value depends on the embedding model, similarity scale, candidate set, and task, so it should be tuned on representative retrieval examples rather than treated as a universal constant.

The exact formula can vary between implementations. What defines the approach is the repeated relevance-versus-redundancy trade-off, not a particular library API.

Work through the smallest useful example

Assume three candidate passages have these query similarities:

A: 0.90
B: 0.88
C: 0.82

A is selected first because it has the highest query similarity. Now suppose similarity to A is:

sim(B, A) = 0.95
sim(C, A) = 0.20

B is slightly more relevant to the query than C, but B is almost a duplicate of A. Let lambda = 0.7.

For B:

0.7 * 0.88 - 0.3 * 0.95
= 0.616 - 0.285
= 0.331

For C:

0.7 * 0.82 - 0.3 * 0.20
= 0.574 - 0.060
= 0.514

MMR selects C next. It gives up a small amount of raw query similarity in exchange for substantially less redundancy.

This example also shows what MMR does not do. It does not declare C globally better than B. It says C has more marginal value after A has already been selected.

MMR sits after candidate retrieval

MMR is normally a selection step, not a replacement for retrieval.

A practical pipeline can look like this:

query
  |
  v
retrieve 30 candidates
  |
  v
optional relevance reranking
  |
  v
MMR-select 6 passages
  |
  v
build prompt context
  |
  v
LLM

The first stage needs enough recall to include the useful evidence. MMR cannot select a passage that never entered the candidate set.

A reranker and MMR also solve different problems. A reranker tries to improve the ordering by relevance. MMR tries to choose a useful set whose members are relevant without being unnecessarily repetitive. A system can use either one or both.

When both are used, be explicit about which scores feed MMR. For example, relevance might come from a reranker while redundancy is measured with embedding similarity. Those scores may have different numeric ranges, so blindly combining them in one formula can make the lambda parameter misleading. Calibration, normalization, or a ranking-based formulation may be needed depending on the implementation.

Implement the selection loop

The core algorithm is small. The following pseudocode assumes that larger similarity values mean more similarity and that query and pairwise similarities are comparable enough for the chosen weighting scheme:

selected = []
remaining = candidates

while len(selected) < k and remaining is not empty:
    if selected is empty:
        choose candidate with highest query_similarity
    else:
        for candidate in remaining:
            redundancy = max(
                similarity(candidate, chosen)
                for chosen in selected
            )
            score = lambda * query_similarity(candidate) \
                    - (1 - lambda) * redundancy
        choose candidate with highest score

    move chosen candidate from remaining to selected

For a small candidate pool, computing these pairwise similarities directly is often acceptable. For larger pools, the repeated comparisons add work. In many RAG systems this is controlled by retrieving a moderate candidate set first and applying MMR only to that set.

The important implementation invariant is that the redundancy penalty is recomputed as the selected set grows. Computing one static diversity score up front is not equivalent, because a candidate’s marginal value depends on what has already been chosen.

Choose similarity and chunking carefully

MMR is only as meaningful as the similarities it receives.

If the same embedding representation is used for both query relevance and passage-to-passage redundancy, implementation is straightforward. But a representation optimized for query-document matching is not automatically ideal for measuring document-document redundancy. Test whether near-duplicate passages actually receive high pairwise similarity and whether meaningfully different evidence is separated.

Chunking also changes the problem. Heavy overlap between neighboring chunks can cause many candidates to contain nearly identical text. MMR can reduce the resulting repetition, but it should not be used to excuse pathological chunking. Reducing unnecessary overlap or deduplicating exact copies earlier may be cheaper and more predictable.

Metadata can provide another useful signal. If diversity across documents, products, time periods, or sources is a hard requirement, enforce that requirement directly rather than hoping vector dissimilarity will produce it. Semantic diversity and business constraints are not the same thing.

Understand the main trade-offs

Diversity can remove useful repetition

Repeated evidence is not always waste. Two independent sources may support the same claim, and multiple nearby chunks may jointly preserve context that was split at chunk boundaries. An aggressive diversity penalty can remove supporting material that the generator needs.

This is why MMR should optimize downstream usefulness, not diversity for its own sake.

Lower lambda can favor irrelevant novelty

A passage can be very different from selected passages because it is about something else. The relevance term must remain strong enough to prevent novelty from dominating the selection.

A practical evaluation should therefore inspect both coverage and relevance. If selected passages are diverse but answer unrelated aspects of the corpus, the system has overcorrected.

Candidate size affects quality and cost

A larger candidate pool gives MMR more alternatives and can improve coverage when relevant evidence appears deeper in retrieval results. It also increases retrieval and selection work. If the initial retriever already returns a clean, nonredundant top-k set, expanding the pool solely for MMR may add latency without useful gains.

Context order is a separate decision

MMR determines which passages enter the selected set. The order in which those passages should appear in the final prompt can be handled separately. For example, a system might select with MMR and then order the chosen passages by relevance, document structure, or chronology.

Do not assume that greedy selection order is automatically the best presentation order for the language model.

Evaluate MMR as a set-selection change

Offline evaluation should use queries that expose redundancy, not only single-fact lookups where ordinary top-k already works well.

Useful cases include multi-part questions, queries whose answer is distributed across several sections, and corpora with repeated or overlapping text. For each query, compare the evidence selected by plain top-k and by MMR under the same final context budget.

Measure outcomes that match the application. Depending on the system, these can include whether all required facts are present, retrieval recall, answer correctness, citation coverage, context tokens consumed, and latency. Diversity itself can be a diagnostic metric, but lower pairwise similarity is not the product goal.

Also inspect failures manually. A numeric improvement can hide cases where MMR drops a crucial supporting passage because the embedding model considers it redundant with a superficially similar one.

Common mistakes

Applying MMR to a weak candidate set. Diversity cannot repair missing evidence. Improve first-stage recall if relevant passages are absent.

Treating lambda as portable. Its effect depends on the score distributions and similarity functions in the specific pipeline.

Mixing incompatible score scales. A cross-encoder relevance score and cosine similarity are not necessarily directly comparable just because both are numbers.

Using diversity to enforce hard source rules. If the application requires at least two independent sources, encode that constraint explicitly.

Optimizing average pairwise distance. A highly diverse set can still be irrelevant. The objective is useful evidence coverage under a context budget.

When MMR is worth using

MMR is a good candidate when relevant retrieval results are frequently redundant and the final context budget is much smaller than the candidate pool. It is especially useful when questions need evidence covering several distinct aspects of a topic.

Plain top-k is simpler and often preferable when candidate results are already diverse, the context budget is generous, or each query usually needs one narrow piece of evidence. Exact deduplication is also a better first tool when the main problem is literal duplicate content rather than semantic overlap.

The practical mental model is simple: retrieval finds individually promising passages, while MMR chooses a collectively useful subset. When context is scarce, the difference between those two objectives can determine whether the model sees repeated evidence or the full set of facts it needs.

Artificial Intelligence RAG LLM Retrieval