The short answer: if your knowledge lives on the open web and changes often, live web search can feed your RAG app on its own, and a vector database is optional overhead. If your knowledge is a fixed, private corpus you query the same way over and over, a vector database earns its place. Most real apps sit somewhere in between, and the mistake is defaulting to a vector store because every tutorial starts there.
RAG got taught as "embed your documents into a vector database, then retrieve." That framing bakes in an assumption most people never question: that retrieval means similarity search over vectors you precomputed. It does not have to. Retrieval just means "get the relevant text before you generate," and a web search at query time is a perfectly good way to get relevant text.
What each approach is actually good at
A vector database stores embeddings of text you have already ingested. You chunk your documents, run each chunk through an embedding model, and store the vectors. At query time you embed the question and pull the nearest chunks by cosine similarity. It is fast, it works offline, and it shines when the same corpus answers many different questions.
Live web search skips all of that. You take the user's question, run a search, fetch the top pages as clean text, and hand those to the model as context. There is no index to build, no embeddings to store, and nothing to keep in sync. The tradeoff is that you pay a fetch cost per query and you depend on the query returning good pages.
Here is the honest comparison:
- Freshness. A vector database knows only what you indexed and when. Web search is current by definition. If your answers depend on this week's pricing, a product that launched yesterday, or a docs page that got rewritten, a static index is confidently wrong and web search is right.
- Coverage. A vector store can only retrieve what you put in it. Web search reaches the long tail: the rare question about a source you never thought to ingest.
- Cost shape. A vector database front-loads cost. You pay to embed and store once, then reads are cheap. Web search has near-zero setup and a per-query cost instead. Which is cheaper depends entirely on your read-to-write ratio.
- Privacy and control. If your corpus is internal, proprietary, or not on the public web at all, web search cannot see it. A vector database is the only option for private knowledge.
- Latency. Similarity search over a warm index is single-digit milliseconds. A live fetch adds a network round trip. For most apps that extra latency is invisible next to the LLM generation time, but it is real.
When a vector database is the wrong default
The vector database is the wrong default when three things are true, which is more often than you would think.
First, your knowledge is public. If the answer is out there on the web, you are duplicating the internet into a database you now have to maintain. Second, your knowledge changes. Every change means re-fetching, re-chunking, re-embedding, and expiring old vectors, which is a whole freshness pipeline you have to build and babysit. Third, your query space is broad and unpredictable. If you cannot guess in advance what people will ask, you cannot know what to pre-index, so you end up indexing everything just in case.
When those hold, a vector database is a large, stateful, slowly-going-stale copy of information you could have fetched fresh on demand. You inherit an embedding bill, an infrastructure component, and a staleness problem, all to reproduce a search you could have run at query time.
The tell is the moment you find yourself writing a cron job to re-crawl your sources and re-embed them so the index does not rot. That job is a signal that your "knowledge base" is really a cache of the live web, and you might be better off just querying the live web.
When you genuinely want vectors
None of this means vector databases are obsolete. They are the right tool when your corpus is private and stable: internal wikis, support tickets, contracts, a product's own documentation, a body of research papers. That content is not searchable on the open web, it does not change every hour, and the same corpus fields endless different questions. Embedding it once and querying it cheaply forever is exactly the workload vector search was built for.
Vectors also win when you need semantic recall that keyword or web search misses, like matching "how do I cancel" to a passage titled "terminating your subscription." That fuzzy conceptual matching is the whole point of embeddings, and it is worth the indexing cost when your corpus is worth indexing.
The pattern most production apps land on
The strongest architecture is usually not one or the other. It is a small vector store for the private, stable knowledge you own, plus live web search for everything current and public. The router is simple: if the question is about your own product, docs, or data, hit the vector store; if it needs fresh or open-web facts, run a search and fetch.
This hybrid keeps each side small and honest. Your index only holds what genuinely belongs in an index, so it stays cheap and does not rot. Your web path handles freshness and the long tail without you pre-indexing the internet. We walk through this exact loop, including chunking and freshness tiers, in how to build a RAG pipeline that uses live web data.
What live web retrieval needs to actually work
If you lean on web search, the retrieval quality lives or dies on the content you feed the model. Two things matter most.
Clean text, not raw HTML. If you fetch a page and pass the raw markup, half your context is navigation, cookie banners, and footer boilerplate. That dilutes the signal and wastes tokens. You want the page reduced to clean markdown that is almost all content, which is also why heading-based chunking becomes easy and why token usage stays sane.
A retrieval call that returns usable text in one step. This is the whole job of link.sc. Give it a URL and /fetch returns clean markdown. Give it a question and /search returns results with full-page markdown already attached, so you skip a separate fetch round trip. Either way you get model-ready text without running a headless browser or writing an extraction layer yourself.
import requests
HEADERS = {"x-api-key": "lsc_your_key_here"}
def retrieve(question):
r = requests.post(
"https://api.link.sc/v1/search",
headers=HEADERS,
json={"q": question},
)
results = r.json()["results"]
# clean markdown is already attached, ready to drop into the prompt
return "\n\n---\n\n".join(x["content"] for x in results[:3])
def answer(question, llm):
context = retrieve(question)
prompt = f"Answer using only the context below.\n\n{context}\n\nQuestion: {question}"
return llm(prompt)
That is a complete RAG retrieval path with no vector database, no embedding model, and no index to keep fresh. For a public, fast-changing domain it is not a shortcut, it is the more correct design.
So do you need a vector database? Only if you have private, stable knowledge that many different questions will hit. If your app's answers live on the open web and move over time, live web search can carry your RAG loop by itself, and skipping the index is the simpler and fresher choice.
Ready to feed your RAG app clean, live web data without an index to maintain? Get a link.sc API key and start with 500 free credits every month.