Reciprocal Rank Fusion k=60: what the constant does, measured on real BM25 + vector runs

Every Reciprocal Rank Fusion (RRF) tutorial uses k=60 and few explain why. What does the constant do, and does changing it help?

Answer: k controls how much a top rank is worth compared with a mid-table rank. Small k rewards “rank 1 in one list”. Large k flattens everything toward “appears in both lists”. On my data, k=1 beat k=60 by 4.1 points of mean MRR@5 (0.733 vs 0.692), and every value from 100 to 1000 was within 0.002 of k=60. The default is safe, not optimal.

What k does

RRF scores a document as the sum of 1 / (k + rank) over each ranked list it appears in. Rank starts at 1. It was introduced in the Cormack, Clarke and Buettcher paper, which uses k=60. For the basics of hybrid retrieval see the free retrieval lesson.

The ratio of the weight of rank 1 to the weight of rank r, computed from the formula:

k	rank 1 vs 2	rank 1 vs 5	rank 1 vs 10	rank 1 vs 50
0	2.00x	5.00x	10.00x	50.00x
1	1.50x	3.00x	5.50x	25.50x
10	1.09x	1.36x	1.82x	5.45x
60	1.02x	1.07x	1.15x	1.80x
200	1.00x	1.02x	1.04x	1.24x
1000	1.00x	1.00x	1.01x	1.05x

At k=60, rank 1 is worth only 1.15 times rank 10. Fusion becomes mostly a vote count: a document that shows up in both lists beats one that is first in a single list. A worked case: document A is rank 1 in one list and missing from the other; document B is rank 5 in both. A wins at k=0 (1.000 vs 0.400) and k=1 (0.500 vs 0.333). B wins at every k from 5 up (at 5: 0.167 vs 0.200; at 60: 0.016 vs 0.031).

Setup

Same corpus, chunker, BM25 and MiniLM embeddings as the hybrid search post. The corpus was re-fetched today and has grown slightly (2,771 chunks vs 2,713 there), so numbers differ a little from that post. Fusion is plain equal-weight RRF, no alpha. A document absent from one retriever’s top 50 contributes 0 for that side. Three query sets:

• Heading (780): section headings as queries. Easy for BM25, because the heading path is indexed.
• Identifier (309): code identifiers that occur in exactly one chunk.
• Paraphrase (30): hand-written questions, a hit is any chunk of the one right file.

Results

Single retrievers, MRR@5:

set	BM25	vector
heading	0.859	0.493
identifier	0.773	0.285
paraphrase	0.566	0.768

Fused, equal weights, MRR@5:

k	heading	identifier	paraphrase	mean
0	0.751	0.704	0.741	0.732
1	0.745	0.710	0.743	0.733
5	0.723	0.705	0.747	0.725
10	0.704	0.700	0.747	0.717
20	0.686	0.690	0.731	0.702
40	0.679	0.688	0.714	0.694
60	0.679	0.684	0.712	0.692
100	0.678	0.683	0.712	0.691
200	0.677	0.683	0.712	0.691
1000	0.676	0.683	0.712	0.690

How often the fused top result is the top result of at least one retriever:

k	= BM25 top-1	= vector top-1	either
0	73.2%	59.2%	100.0%
5	67.6%	54.1%	89.3%
60	63.1%	50.6%	81.2%
1000	63.0%	50.6%	81.1%

What the numbers mean

• Small k helped most where one retriever is clearly stronger. On heading queries, k=0 beat k=60 by 7.2 points (0.751 vs 0.679). BM25 is far better there (0.859), and a small k lets its confident rank-1 survive the vote.
• Identifier and paraphrase queries barely moved: 2.0 and 2.9 points between k=0 and k=60.
• Large k stops mattering. Between k=100 and k=1000 the mean moved 0.001. Tuning in that range is wasted effort.
• At k=0, the fused top-1 is always one retriever’s top-1 (100%). At k=60 that falls to 81.2%: in about one query in five, fusion promotes a document neither retriever put first. That is the “consensus” effect, and in this data it cost accuracy more than it gained.
• Surprise: no fused setting beat the best single retriever. BM25 alone got 0.859 on headings (best fused 0.751) and vectors alone got 0.768 on paraphrases (best fused 0.747). Equal-weight fusion buys robustness across query types, not a higher peak. The mean of the best single retriever per set would be higher than any fused mean here, but you can’t know the query type in advance.

Caveats

• One corpus, one embedding model, one chunker. The 30 paraphrase queries are a small sample: one query is 3.3 points of hit rate, so treat paraphrase differences of a few points as noise.
• Heading queries overlap the indexed heading path, so they favor BM25 and likely exaggerate the small-k gain there.
• Equal weights and top-50 depth only. With a weight alpha (see the alpha sweep) or deeper lists, the best k may shift. I didn’t test that.
• Retrievers with more than two lists, or lists where one is much noisier, may behave differently.

Code

from collections import defaultdict

def rrf(rankings: list[list[str]], k: int = 60, top: int = 5) -> list[str]:
    scores: dict[str, float] = defaultdict(float)
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] += 1 / (k + rank)
    return sorted(scores, key=lambda d: (-scores[d], d))[:top]

Rule of thumb

Keep k=60 if you have no labeled queries: it is within about 4 MRR points of the best value here, and the Cormack paper’s choice. If you do have a golden set, sweep k over [1, 5, 10, 20, 60] and ignore anything above 100. Check whether fusion beats your best single retriever before shipping it; for the eval harness see RAG evaluation in CI.

Experiment files

bench_rrf_k.py (setup copied from the hybrid benchmark, plus the k sweep), results.txt and fetch_corpus.sh are in the Experiment files box below. The script runs in about three minutes on 2 CPU cores, most of it embedding the chunks.