A retrieval-augmented generation (RAG) system can retrieve highly relevant passages and still build a poor context. The problem appears when the top results repeat the same fact in slightly different wording.

Suppose five retrieved chunks all explain how to reset an API token, while a lower-ranked chunk explains the permission change that must happen afterward. Filling the context window with the five near-duplicates gives the language model less useful evidence than selecting a smaller set that covers both parts of the task.

Maximal marginal relevance (MMR) is a reranking method for this problem. Instead of selecting candidates only by relevance to the query, MMR also penalizes candidates that are too similar to items already selected. The result is a controllable trade-off between relevance and novelty.

This article builds MMR from a small example, explains its greedy selection rule, and shows how to use it responsibly in a RAG pipeline without confusing diversity with retrieval quality.

The problem is redundant relevance

Consider a documentation assistant answering:

How do I rotate an API token and update the service afterward?

An embedding retriever might return these candidates:

A  0.94  Rotate an API token from the settings page
B  0.92  Steps for rotating an API token
C  0.90  Replace an existing API token
D  0.86  Update the service secret after token rotation
E  0.82  Verify the new token before revoking the old one

The numbers are simplified query-similarity scores. If the system sends only the top three chunks to the model, it selects A, B, and C. All three may be relevant, but they may also contain nearly the same information.

This is not necessarily a failure of the embedding model. The retriever is doing what its ranking objective asks: place individually relevant items near the top. It has not been asked whether the selected set is redundant.

MMR changes the selection question from:

Which remaining item is most relevant to the query?

to:

Which remaining item adds the best combination of relevance and new information?

That set-level view is the core mental model.

Select one item at a time

MMR builds the final set greedily. At each step, it scores every unselected candidate using two signals:

  1. how relevant the candidate is to the query;
  2. how similar the candidate is to the items already selected.

A common form is:

MMR(d) = lambda * sim(query, d)
         - (1 - lambda) * max(sim(d, s)) for s in selected

Here, d is a candidate document or chunk, sim is a similarity function, and lambda is a value between 0 and 1 that controls the trade-off.

The first term rewards query relevance. The second term penalizes redundancy with the most similar item already selected.

When lambda = 1, the redundancy term disappears, so selection is driven entirely by query relevance. As lambda decreases, similarity to already selected items receives more weight. At the extreme lambda = 0, query relevance no longer contributes to the score, which is usually not a useful retrieval policy by itself.

The exact similarity functions do not have to be identical in every implementation. For example, a system may use embedding cosine similarity for both terms, but MMR as a general criterion only requires meaningful relevance and redundancy scores whose scales are compatible with the chosen weighting.

Walk through a small example

Return to the API-token candidates. Suppose A is selected first because it has the highest query relevance.

Now assume the similarities to A are:

candidate  query similarity  similarity to A
B          0.92              0.96
C          0.90              0.93
D          0.86              0.35
E          0.82              0.42

Use lambda = 0.7. The MMR score for B is:

0.7 * 0.92 - 0.3 * 0.96
= 0.644 - 0.288
= 0.356

For D:

0.7 * 0.86 - 0.3 * 0.35
= 0.602 - 0.105
= 0.497

Although B is more similar to the query, it is also very similar to information already represented by A. Candidate D therefore receives the larger MMR score and is selected next.

After selecting D, redundancy must be recomputed against the selected set {A, D}. For each remaining candidate, the penalty uses its largest similarity to any selected item:

max(sim(candidate, A), sim(candidate, D))

This detail matters. MMR is not a one-time adjustment to the original ranking. Each selection changes what counts as redundant for the next step.

The process continues until the desired number of chunks has been selected or another stopping rule is reached.

MMR is reranking, not retrieval

In a typical RAG pipeline, MMR operates on a candidate pool that another retrieval method has already produced:

query
  -> retrieve a broad candidate pool
  -> rerank candidates with MMR
  -> select context chunks
  -> generate an answer

This separation explains an important limitation: MMR cannot recover a useful chunk that never entered the candidate pool.

If the initial retriever returns 20 near-duplicate chunks from one document and misses the relevant troubleshooting guide entirely, MMR can diversify only among those 20 candidates. Improving candidate recall requires changes earlier in the pipeline, such as better embeddings, keyword retrieval, hybrid retrieval, metadata filtering, chunking, or a larger candidate pool.

A useful design therefore distinguishes two sizes:

candidate pool size: how many items retrieval gives to MMR
final context size:  how many items MMR selects

If both values are 5, MMR has almost no room to make a different choice. Retrieving more candidates than the final context needs gives the reranker alternatives, although a larger pool also increases similarity computations and may admit more weakly relevant material.

Implement the selection loop carefully

The algorithm itself is compact. The following pseudocode intentionally avoids any library-specific API:

selected = []
remaining = candidates

while remaining is not empty and len(selected) < k:
    if selected is empty:
        choose the candidate with highest query relevance
    else:
        for each candidate in remaining:
            redundancy = max(sim(candidate, item) for item in selected)
            score = lambda * relevance(candidate, query) \
                    - (1 - lambda) * redundancy
        choose the candidate with highest score

    move chosen candidate from remaining to selected

For embedding-based retrieval, cosine similarity is a common choice when the representation model is intended to support it. If vectors are unit-normalized, their dot product equals cosine similarity, which can simplify computation.

Do not assume that every retriever score can be inserted directly into the formula. A distance where smaller values are better, an unbounded lexical score, and cosine similarity have different meanings and scales. If the relevance and redundancy terms are not comparable, lambda no longer expresses the trade-off you think it does.

One practical approach is to derive both terms from a compatible embedding similarity. Another is to transform or calibrate scores deliberately and validate the resulting ranking. The correct choice depends on the retrieval system; MMR itself does not define a universal score-normalization rule.

Tune lambda against the task, not intuition

lambda controls a real quality trade-off. A value near 1 preserves the original relevance ranking more strongly. A lower value gives the novelty penalty more influence.

Neither direction is universally better.

Imagine a query asking for one exact configuration option. Several similar chunks may all contain the same correct answer, and aggressive diversification could replace a highly relevant chunk with a less relevant one simply because it looks different. In that case, diversity has little value.

Now imagine a broad diagnostic query with several possible causes. A context containing five paraphrases of one cause may be much less useful than a set covering authentication, networking, permissions, and configuration. More diversification may help.

Treat lambda as a parameter to evaluate on representative queries. For each candidate value, measure downstream behavior rather than only inspecting whether the selected chunks look varied.

Useful retrieval-level measurements can include:

  • whether known relevant evidence appears in the final set;
  • redundancy among selected chunks;
  • coverage of distinct required facts or subtopics when such labels exist;
  • the rank or inclusion rate of evidence needed to answer the query.

Then measure the RAG outcome that actually matters, such as answer correctness, evidence support, or task completion. A prettier retrieval list is not the goal.

Evaluate candidate-pool size together with lambda

lambda does not operate independently of the candidate pool.

Suppose the final context contains five chunks. Compare two configurations:

configuration A: retrieve 5, select 5
configuration B: retrieve 30, select 5 with MMR

Configuration A gives MMR no meaningful choice. Configuration B gives it alternatives, but some of those alternatives may be less relevant.

Increasing the pool can therefore help until additional candidates mostly add noise or unacceptable computation. The useful range depends on corpus size, retriever quality, chunk granularity, latency budget, and final context size.

Evaluate pool size and lambda together. A diversity setting that works with 20 strong candidates may behave differently with 200 candidates containing many weak matches.

Understand the cost

Pure top-k retrieval can simply take the first k results from an existing ranking. Greedy MMR performs additional candidate-to-selected similarity comparisons as it builds the set.

For a candidate pool of n items and a final selection of k items, a straightforward implementation performs on the order of n * k candidate-to-selected comparisons. Exact work depends on implementation details, caching, vectorization, and how candidates are removed as selection proceeds.

For modest RAG candidate pools, this may be small compared with embedding generation or language-model inference. At larger scales or under tight latency targets, it can matter. Precomputing candidate embeddings, vectorizing similarity calculations, and keeping the MMR pool bounded are common ways to control the reranking cost.

Do not use MMR as a replacement for efficient first-stage retrieval across an entire large corpus. Its natural role is usually to rerank a manageable candidate set.

Watch for common failure modes

Diversity can reward irrelevance

If the novelty penalty is too strong, a candidate can win because it is different rather than because it is useful. Lowering lambda does not create better evidence; it changes the balance between two signals.

Keep the initial candidate pool reasonably relevant and validate the final selections against the task.

Similarity is only a proxy for redundancy

Two chunks can have high embedding similarity while contributing different critical details. Conversely, two chunks can use different wording while making the same claim.

MMR sees the similarity function you provide, not semantic redundancy in an absolute sense. Domain-specific evaluation is necessary when small factual differences matter.

Chunk boundaries can distort diversity

Adjacent chunks from the same source may overlap heavily because of the chunking strategy. MMR can reduce some of that repetition, but excessive overlap is often better addressed at chunking or indexing time.

Likewise, forcing diversity across badly fragmented chunks cannot reconstruct context that was lost when documents were split.

Source diversity is not guaranteed

Embedding dissimilarity does not necessarily mean source diversity. If the application needs constraints such as at most two chunks per document, evidence from multiple repositories, or a required date range, encode those rules explicitly. MMR is a scoring criterion, not a general constraint solver.

Duplicate removal and MMR solve different problems

Exact duplicates are usually better removed directly. MMR is useful for softer redundancy where two distinct candidates are similar enough that selecting both may waste context.

Running exact or near-exact deduplication before MMR can make the candidate pool cleaner and leave MMR to handle the relevance-versus-novelty trade-off.

Know when a simpler approach is enough

MMR is useful when the final result is a small subset, the candidate pool contains meaningful redundancy, and diversity can expose additional evidence that helps the downstream task.

It is less compelling when the query has one narrow intent, the retriever already returns non-redundant results, or the application needs a strict learned relevance ordering. In those cases, ordinary top-k selection may be simpler and easier to reason about.

It is also not a substitute for a learned reranker. A cross-encoder or another learned ranking model can estimate query-document relevance using richer interactions than embedding similarity. MMR addresses a different question: how much new information does a candidate add relative to what has already been selected?

The two ideas can be combined. For example, a system can retrieve candidates, apply a stronger relevance reranker, then use MMR or another diversification method to choose a compact final context. Whether that extra stage helps should be established by evaluation rather than assumed.

Conclusion

RAG context quality depends on the usefulness of the selected set, not only on the relevance of each chunk in isolation. When top-ranked candidates repeat one another, spending context tokens on all of them can crowd out evidence that covers another part of the query.

Maximal marginal relevance makes that trade-off explicit. It greedily rewards query relevance while penalizing similarity to already selected items. Used on a sufficiently broad but relevant candidate pool, it provides a simple way to reduce redundant context.

The practical discipline is to treat MMR as a reranking tool, keep its score scales meaningful, tune lambda together with candidate-pool size, and evaluate downstream answer quality. Diversity is valuable when it brings useful new evidence—not merely when the retrieved chunks look different.

Artificial Intelligence RAG Retrieval Embeddings