Quick answer: Keyword search matches the exact words in your query against the words in documents (fast, precise, literal). Semantic search matches meaning by comparing vector embeddings, so it finds relevant results even when they share no words with the query. Neither wins everywhere, which is why production systems increasingly use hybrid search: BM25 keyword scoring combined with vector similarity, taking the strengths of both.
The short version: keyword search asks "which documents contain these words", and semantic search asks "which documents mean the same thing". Those are different questions, and the right one depends on what you are searching for.
The core difference
Keyword search, in its classic form, builds an inverted index (a map from each word to the documents that contain it) and scores matches with a ranking function like BM25 that weighs term frequency and rarity. If you search "python list comprehension", it finds documents with those tokens.
Semantic search embeds both the query and the documents into vectors, then ranks by similarity. It can match "how do I make a new list from an existing one in a single line" to a document about list comprehensions even though the words barely overlap. It trades literal precision for conceptual recall.
| Keyword search | Semantic search | |
|---|---|---|
| Matches on | Exact words / tokens | Meaning (vectors) |
| Handles synonyms | No | Yes |
| Handles typos | Poorly | Somewhat |
| Exact IDs, codes, names | Excellent | Weak |
| Rare / out-of-vocab terms | Excellent | Can miss |
| Infrastructure | Inverted index (BM25) | Embedding model + vector store |
| Explainability | High (you see the matched terms) | Low (a similarity score) |
| Cost per query | Very low | Higher (embed + ANN search) |
Where keyword search wins
Do not write off keyword search. It is still the right default for a large class of queries.
- Exact identifiers. Order numbers, SKUs, error codes, function names, legal citations. When the user knows the precise string, exact match is what they want, and semantic search will "helpfully" surface near-matches they did not ask for.
- Rare or brand-new terms. A product name coined last week may not embed well because the model never saw it. BM25 finds it instantly.
- Speed and cost at scale. No embedding call, no vector search. When latency and budget are tight and queries are literal, keyword search is hard to beat.
- Explainability. You can show exactly which terms matched, which matters for compliance and debugging.
Where semantic search wins
- Natural-language questions. Users phrase things a hundred ways. Semantic search maps them all to the same intent.
- Synonyms and paraphrase. "Car" finds "automobile", "reset password" finds "recover login".
- Cross-lingual and conceptual matches. Multilingual embedding models can match a query in one language to a document in another.
- Discovery. When the user does not know the exact words, meaning-based retrieval surfaces relevant material they could not have queried for by keyword.
Hybrid search: use both
The mature answer is not to choose. Hybrid search runs a keyword query and a vector query, then fuses the two ranked lists into one. It catches the exact-match cases keyword search nails and the meaning-based cases semantic search nails.
A common fusion method is Reciprocal Rank Fusion (RRF), which combines rankings without needing the two scores to be on the same scale:
def reciprocal_rank_fusion(keyword_hits, vector_hits, k=60):
scores = {}
for rank, doc_id in enumerate(keyword_hits):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
for rank, doc_id in enumerate(vector_hits):
scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
return sorted(scores, key=scores.get, reverse=True)
# keyword_hits: doc ids ranked by BM25
# vector_hits: doc ids ranked by cosine similarity
final = reciprocal_rank_fusion(keyword_hits, vector_hits)
RRF is popular because it is simple and it does not care that BM25 scores and cosine similarities are incomparable numbers. It only looks at rank position. Many vector databases and search engines now offer hybrid search as a built-in feature, so you often do not implement the fusion yourself.
When semantic search fails
Semantic search is not magic, and treating it as such causes real bugs.
- Exact-string queries. As above, it will fuzz an order number into similar-looking ones. Bad.
- Negation and precise logic. "documents that do not mention pricing" is not something similarity scoring understands. It sees "pricing" and pulls pricing docs.
- Domain jargon a general model never learned. If your field uses terms the embedding model treats as noise, recall suffers. Test on your data.
- Freshness. Your vector index is only as current as your last ingest. A brand-new document is invisible until embedded and stored.
That last point matters for anything answering questions about the current world. If embeddings power the retrieval layer under this, the mechanics are worth understanding: see what are vector embeddings.
How this connects to web search for agents
For an AI agent, the "index" is often the live web, and the freshness problem dominates. No static vector store contains today's news or a price that changed an hour ago. So agents lean on a web search API that already does the ranking (a blend of keyword and semantic signals) and returns current results.
The difference that matters for agents is what comes back. Traditional search APIs return snippets, which are too thin to reason over. An LLM-oriented API pairs the search with a fetch that returns full-page content:
# 1. Search: returns ranked results in serpData.results, each with a targetUrl
curl -X POST https://api.link.sc/v1/search \
-H "x-api-key: lsc_..." \
-H "Content-Type: application/json" \
-d '{"q": "who won the latest F1 race and by how much", "engine": "google"}'
# 2. Fetch: pull the full page content for a result's targetUrl
curl -X POST https://api.link.sc/v1/fetch \
-H "x-api-key: lsc_..." \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/f1-race-report", "format": "markdown"}'
Full content means the agent can extract the specific fact instead of guessing from a two-line preview. We cover this pattern end to end in real-time web search for LLMs.
The takeaway
Keyword search matches words; semantic search matches meaning. Keyword wins on exact strings, rare terms, cost, and explainability. Semantic wins on natural language, synonyms, and discovery. In practice, hybrid search with BM25 plus vectors gives you both, and for agents working against a changing world, a full-content web search API solves the freshness problem that any static index cannot.
Building an agent that needs fresh, full-content web results? Try link.sc free.