Fuse Keyword and Vector Search with Reciprocal Rank Fusion

A RAG system often needs two kinds of retrieval at once. Keyword search is good at exact strings such as product codes, error messages, and names. Vector search can recover passages that express the same idea with different words. Running both is easy; combining their scores correctly is where many implementations become fragile.

Reciprocal rank fusion (RRF) solves that problem by ignoring the raw scores and combining rank positions instead. That matters because a BM25 score and a vector similarity score do not share a meaningful numeric scale. This article builds the RRF mental model, works through a small example, and shows what to tune and measure when using it for hybrid retrieval.

Why adding raw retrieval scores is unreliable

Imagine a documentation search for:

reset error E104 after token refresh

A keyword retriever might rank documents using BM25:

rank  document  BM25 score
1     A         18.7
2     B         14.2
3     C          9.6

A vector retriever might return cosine similarities:

rank  document  cosine similarity
1     C         0.89
2     D         0.84
3     A         0.80

It is tempting to add 18.7 + 0.80 for document A. That number has no stable interpretation. BM25 and cosine similarity are produced by different scoring functions, and their ranges can change with the query, index, model, and implementation. Multiplying one side by an arbitrary constant merely hides the scale mismatch behind a tuning parameter.

Score normalization can be useful when it is designed and validated carefully, but it introduces assumptions about the score distributions. RRF takes a simpler route: it asks only where each document ranked.

The mental model: reward high ranks, then add the evidence

For each ranked list, RRF gives a document a contribution based on its position:

contribution = 1 / (k + rank)

The fused score is the sum of those contributions across the lists in which the document appears:

RRF(d) = sum(1 / (k + rank_i(d)))

Here, rank_i(d) is the one-based rank of document d in result list i. If the document is absent from a retrieved list, that list contributes nothing. The constant k dampens the difference between adjacent ranks. A commonly used value is 60, but it is a parameter rather than a law of retrieval quality.

The useful intuition is more important than the formula:

• a document near the top of a list receives more credit than one near the bottom;
• a document found by several retrievers receives credit from each of them;
• raw BM25 and vector scores never have to be compared.

This makes RRF especially convenient for hybrid search, where the component rankers may have unrelated score scales.

Work through the smallest useful example

Use a small k = 10 here so the arithmetic is easy to see. Production systems often use a larger value; the example is only for understanding the mechanism.

Suppose keyword search returns:

1. A
2. B
3. C

Vector search returns:

1. C
2. D
3. A

Document A contributes from both lists:

keyword: 1 / (10 + 1) = 0.0909
vector:  1 / (10 + 3) = 0.0769
A total:                  0.1678

Document C also appears in both:

keyword: 1 / (10 + 3) = 0.0769
vector:  1 / (10 + 1) = 0.0909
C total:                  0.1678

B and D appear only once:

B: 1 / (10 + 2) = 0.0833
D: 1 / (10 + 2) = 0.0833

The fused ranking is therefore A and C at the top, followed by B and D. A and C earned support from both retrieval methods even though their raw component scores lived on different scales.

A tie is possible, as it is here. A production implementation needs a deterministic tie-break rule, such as a stable document identifier or a defined secondary ranking signal. The tie-breaker should not silently reintroduce an unvalidated comparison between incompatible raw scores.

Implement RRF without tying it to a search vendor

The algorithm itself is small. The main requirement is that every result has a stable identifier shared by all retrievers.

function rrf(result_lists, k):
    scores = map(default = 0)

    for results in result_lists:
        for rank, document in enumerate(results, start = 1):
            scores[document.id] += 1 / (k + rank)

    return sort_descending(scores)

This pseudocode leaves several production decisions outside the function: how many candidates each retriever supplies, whether lists have weights, how ties are resolved, and whether a later reranker is used. Keeping those choices explicit makes evaluation easier.

Deduplicate by document identity before final ranking. If the same passage can appear under several IDs because of indexing or chunking mistakes, RRF will treat those IDs as different candidates and can crowd the result set with near-duplicates.

Candidate depth matters as much as the fusion formula

RRF can only fuse documents that the component retrievers return. If keyword search contributes its top 100 results but vector search contributes only its top 5, the keyword side has many more opportunities to place documents into the candidate pool.

That does not mean both depths must always be equal. Different retrievers can have different cost and recall characteristics. It does mean candidate depth is part of the retrieval policy and should be evaluated deliberately.

Consider a relevant passage ranked 18th by vector search and absent from keyword search. With a vector candidate depth of 10, RRF never sees it. Changing k cannot recover a candidate that was discarded before fusion.

For RAG, this distinction is useful:

retrieval depth -> determines which candidates can participate
RRF parameter k -> determines how rank positions contribute within that pool
final top N     -> determines what proceeds to reranking or the model context

When recall is poor, inspect candidate generation before spending time tuning the fusion constant.

What the k constant actually changes

The k in RRF is sometimes confused with the number of nearest neighbors requested from vector search. They are separate quantities.

In the RRF formula, a smaller k makes differences near the top of each list more pronounced. A larger k flattens those differences, so moving from rank 1 to rank 2 changes the contribution less.

For example:

k = 10
rank 1: 1/11 = 0.0909
rank 2: 1/12 = 0.0833

k = 60
rank 1: 1/61 = 0.01639
rank 2: 1/62 = 0.01613

The absolute RRF score is not a probability of relevance. Its value depends on k, the number of fused lists, and where the document appears in those lists. Treat it as a ranking signal, not as calibrated confidence.

A default such as k = 60 is a reasonable starting point because it is widely used, but the operating point should still be checked on your retrieval task. If quality is sensitive to small changes in k, that can be a sign that candidate generation or the evaluation set deserves closer inspection.

Add weights only when you have evidence for them

Plain RRF gives each ranked list equal influence. Some systems need a weighted form:

weighted_RRF(d) = sum(w_i / (k + rank_i(d)))

A larger w_i gives retriever i more influence. This can make sense when evaluation shows that one retriever is consistently more useful for the workload.

Weights can also hide retrieval defects. If vector search performs poorly on exact identifiers because those identifiers are badly represented in the embedding space, increasing the keyword weight may improve aggregate metrics while leaving semantic queries worse. Segment evaluation by query type before deciding that one global weight is the right fix.

If you do not have enough labeled queries to justify weights, equal weighting is easier to reason about and harder to overfit.

RRF is fusion, not a relevance model

RRF combines rankings; it does not inspect the query or document text itself. That creates useful boundaries around what it can fix.

If both retrievers rank an irrelevant document highly, RRF will usually reinforce that mistake. If neither retriever finds a relevant passage, fusion cannot invent it. If your chunks are poorly formed, embeddings are mismatched to the domain, or lexical indexing drops a critical field, RRF does not repair those upstream problems.

It also cannot learn that some queries should rely more heavily on exact lexical evidence while others should rely more on semantics unless you add weighting or another query-dependent mechanism.

For those reasons, RRF often works well as a first fusion stage rather than the final word on relevance.

Combine RRF with reranking when the extra cost is justified

A practical RAG retrieval pipeline can look like this:

query
  |-- keyword retrieval --\
  |                      RRF -> top candidates -> reranker -> context
  |-- vector retrieval  --/

RRF is inexpensive because it works on ranks and document IDs. A cross-encoder or another model-based reranker can then inspect the query-document pairs more deeply, but that extra scoring costs compute and latency.

The two stages solve different problems. RRF broadens the candidate pool by combining complementary retrievers. A reranker tries to order that smaller pool more precisely. Sending hundreds of candidates directly to an expensive reranker may waste latency; fusing and trimming first can keep the reranking workload bounded.

A reranker is not mandatory. If RRF already meets your relevance and latency targets, the simpler pipeline has fewer moving parts and is easier to operate.

Evaluate retrieval before evaluating generated answers

A RAG answer can fail because retrieval missed the evidence or because the generator mishandled evidence that was retrieved correctly. Measure retrieval separately so those failure modes do not get mixed together.

Build an evaluation set containing realistic queries and judgments about which passages are relevant. Include the cases that motivated hybrid search: exact identifiers, paraphrases, acronyms, rare names, and queries where lexical and semantic signals disagree.

Compare at least:

keyword only
vector only
RRF fusion

Use retrieval metrics that match the application. Recall at a candidate cutoff is useful when a later reranker or generator needs at least one relevant passage in the pool. Metrics such as mean reciprocal rank or normalized discounted cumulative gain can be useful when the position of relevant results matters directly. The metric should reflect what the downstream stage actually consumes.

Also measure latency and candidate counts. Running two retrievers can improve retrieval quality while increasing query work, memory traffic, or service cost. Parallel execution can reduce wall-clock impact, but it does not make the second retrieval free.

Common mistakes make hybrid retrieval look better than it is

The first mistake is evaluating only a handful of friendly queries. Hybrid retrieval is most useful because query types differ, so the test set should preserve that variety.

The second is tuning k, candidate depths, and weights on the same queries used for the final quality report. That encourages overfitting to the evaluation set. Keep a separate held-out set when the dataset is large enough to support it.

Another mistake is assuming a higher RRF score means a document is relevant with higher probability. RRF scores order candidates; they are not calibrated probabilities and should not be used as confidence thresholds without separate validation.

Finally, do not use fusion as a substitute for access control or metadata constraints. If a user is not allowed to retrieve a document, enforce that rule in retrieval or filtering. Ranking logic is not an authorization boundary.

When reciprocal rank fusion is a good fit

RRF is a strong baseline when you have multiple useful retrievers, their raw scores are not directly comparable, and you want a fusion method that is simple enough to inspect and implement.

It is less compelling when one retriever already satisfies the task, because a second retrieval path adds complexity and cost. It may also be too limited when relevance depends on rich query-document interactions that neither component ranking captures. In that case, learned fusion or model-based reranking may earn its additional complexity, provided you have the data and evaluation process to support it.

A practical next step

Start with two ranked lists and equal-weight RRF. Keep the candidate depths explicit, use a stable document ID, and record the component ranks alongside the fused rank so failures are explainable.

Then test keyword-only, vector-only, and fused retrieval on the same held-out queries. If RRF wins for the query mix you actually serve, you have a useful hybrid baseline. If it does not, inspect which retriever misses which queries before adding more tuning knobs. The point of reciprocal rank fusion is not to make retrieval complicated; it is to combine complementary evidence without pretending unrelated scores are comparable.