Ask a pure vector search index for "error 0x80070057" and there's a decent chance it hands you a nicely written paragraph about Windows troubleshooting in general, while the one document that actually contains that exact error code sits at rank 40. Embeddings are great at meaning and terrible at strings. BM25 has the opposite problem: it nails exact tokens and falls apart the moment the user phrases something differently than the document does.
Hybrid search is the boring, effective answer: run both, then merge the results. No new model, no fine-tuning, usually a double-digit improvement in retrieval quality. Here's how it actually works, including the fusion step everyone hand-waves past.
Two retrievers, two failure modes
BM25 is lexical. It scores documents by term frequency and inverse document frequency, with length normalization on top. If the query says "kubectl rollout undo" and the document says "kubectl rollout undo", BM25 finds it every time. If the document says "reverting a Kubernetes deployment", BM25 finds nothing, because not a single token overlaps.
Vector search is semantic. You embed the query and every chunk into the same vector space and rank by cosine similarity. "Reverting a Kubernetes deployment" and "kubectl rollout undo" land close together, so paraphrase queries work. But embeddings compress text into a few hundred dimensions, and rare exact identifiers (part numbers, error codes, function names, people's names) get smeared out in that compression.
In my experience the split looks roughly like this:
| Query type | BM25 | Vector |
|---|---|---|
| Exact identifiers (SKUs, error codes, API names) | Strong | Weak |
| Paraphrases and synonyms | Weak | Strong |
| Short keyword queries | Strong | Mixed |
| Long natural-language questions | Mixed | Strong |
| Out-of-domain jargon the embedder never saw | Strong | Weak |
Real query traffic is a blend of all of these, which is exactly why picking one retriever means picking which subset of your users you're going to fail.
The fusion step: Reciprocal Rank Fusion
So you have two ranked lists. Now what? You can't just add the scores together: BM25 scores are unbounded and cosine similarities live between -1 and 1, so raw addition lets one system silently dominate.
Reciprocal Rank Fusion (RRF) sidesteps the whole problem by throwing the scores away and using only the ranks. For each document, sum up:
RRF(d) = sum over each retriever r of 1 / (k + rank_r(d))
where rank_r(d) is the document's position in retriever r's list (1-indexed) and k is a constant, almost always 60. A document ranked 1st by BM25 and 3rd by vector search gets 1/61 + 1/63. A document only one retriever found still gets partial credit from that single term.
The whole thing is about ten lines of code:
def rrf_fuse(rank_lists, k=60):
scores = {}
for ranking in rank_lists:
for rank, doc_id in enumerate(ranking, start=1):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)
fused = rrf_fuse([bm25_top50, vector_top50])[:10]
Why k=60? It flattens the curve so that rank 1 isn't worth vastly more than rank 10. That makes RRF forgiving: a retriever that's confidently wrong can't drag a bad document to the top on its own, because the other retriever's silence costs it.
The alternative is weighted score fusion: min-max normalize each retriever's scores to [0, 1], then compute something like 0.4 * bm25 + 0.6 * vector. It can beat RRF when you tune the weights on real relevance judgments. The catch is that normalization is fragile (one outlier score reshapes the whole distribution) and the weights drift as your corpus changes. If you don't have an evaluation set, use RRF and move on. Elasticsearch, OpenSearch, Weaviate, Qdrant, and pgvector setups all support some version of this natively now, so you often don't even write the loop yourself.
One practical note: fetch generously before fusing. Take the top 50 to 100 from each retriever, fuse, then keep the top 10. Fusion can only promote documents that made it into at least one list.
When hybrid actually wins
Hybrid is not free lunch in every scenario, so it's worth being honest about where the gains come from.
Hybrid wins clearly when:
- Your queries mix identifiers with natural language ("how do I fix CVE-2024-3094 in sshd"). This is most technical documentation, support, and codebase search.
- Your corpus contains vocabulary the embedding model barely saw during training: internal product names, niche legal or medical terms, fresh jargon.
- You can't afford a reranker. RRF gets you a chunk of the benefit of reranking at essentially zero latency cost, since both retrievers run in parallel.
Hybrid helps less when:
- Queries are purely conversational and your corpus is well-covered by the embedder. Vector alone may already be near the ceiling.
- You're adding a cross-encoder reranker anyway. The reranker fixes a lot of what fusion fixes, though feeding it hybrid candidates instead of vector-only candidates still tends to help, because the reranker can only rerank what retrieval surfaced.
Anecdotally, the single most common RAG retrieval bug I see is not a bad embedding model. It's vector-only retrieval quietly whiffing on exact-match queries, and nobody noticing because the demo queries were all paraphrases.
Where live web search slots in
Everything above assumes the answer is already in your index. Often it isn't. Prices changed, a library shipped a breaking release, the news happened after your last crawl. No amount of fusion fixes an index that doesn't contain the answer.
The pattern that works is treating live web search as a third retriever. Your BM25 and vector indexes cover your private corpus; a web search covers everything else, and the results get fused or simply appended as a separate context block. A search API built for LLMs returns structured results you can drop straight into that pipeline, and the fetch side returns pages as clean Markdown so the web-sourced chunks look like the rest of your corpus instead of raw HTML soup.
Two ways to wire it:
- Always-on: every query hits local hybrid search and web search in parallel, and RRF fuses all three lists. Simple, higher cost, best freshness.
- Fallback: run local hybrid first; if the top fused score is weak or the query mentions dates, versions, or prices, escalate to web search. Cheaper, and covers most freshness cases.
If you go this route, the same chunking and cleaning rules apply to web content as to your own documents. I've covered both in how chunkers work and in the walkthrough on building a RAG pipeline with live web data.
A sane default stack
If you're starting from zero, this setup is hard to beat for the effort involved:
- BM25 and vector retrieval over the same chunks, top 75 each
- RRF with k=60, keep top 10
- Optional: a cross-encoder reranker on the fused top 25 if quality still isn't there
- Live web search and fetch as the freshness escape hatch
Hybrid search is one of the rare cases in this field where the robust choice is also the simple one. Two retrievers, one ten-line fusion function, and a large class of embarrassing retrieval failures just goes away.
Need the live web side of your retrieval stack? Sign up for link.sc and get 500 free requests a month for LLM-ready search and fetch.