What Is RAG (Retrieval-Augmented Generation)? A Practical Guide

Quick answer: RAG (retrieval-augmented generation) is a pattern where you first retrieve relevant documents for a query, then feed those documents to an LLM as context so it generates an answer grounded in real data instead of its training memory. It beats fine-tuning for fresh, private, or frequently changing information because you update the data, not the model. The core pipeline is chunk, embed, retrieve, generate, and it is straightforward to build.

Language models are frozen at their training cutoff and they do not know your internal docs. RAG fixes both problems without touching model weights. You keep the knowledge in a store you control, pull the right pieces at query time, and let the model do what it is good at: reasoning over text you hand it.

The retrieve-then-generate pattern

Every RAG system is two steps stapled together.

1. Retrieve. Take the user's question, search a knowledge store, and pull back the handful of passages most likely to contain the answer.
2. Generate. Put those passages into the prompt as context, along with the question, and ask the model to answer using only that context.

That second constraint is the whole point. When you tell the model to answer from the provided passages and to say so when the answer is not there, you get responses grounded in your data with citations you can trace. The model stops inventing and starts summarizing evidence.

Why RAG beats fine-tuning for fresh and private data

Fine-tuning and RAG solve different problems, and people reach for the wrong one constantly. Fine-tuning changes how a model behaves. RAG changes what a model knows at answer time. Here is the honest comparison.

Dimension	RAG	Fine-tuning
Best for	Fresh, private, changing facts	Style, format, task behavior
Update cost	Add a document	Retrain the model
Time to update	Seconds	Hours to days
Traceable sources	Yes, cite the retrieved chunk	No, knowledge is baked in
Handles data changing daily	Yes	No, stale immediately
Upfront cost	Low	Higher

If your knowledge changes (prices, docs, news, support tickets), RAG wins because updating is just writing a new document to the store. Fine-tuning would mean a retrain for every change, which is neither fast nor cheap. Reach for fine-tuning when you need to change tone, enforce an output format, or teach a specialized task, not to teach facts.

The pipeline: chunk, embed, retrieve, generate

A working RAG system has four stages. Two happen ahead of time (indexing), two happen per query.

1. Chunk

You cannot embed a whole document usefully, so you split it into passages of a few hundred tokens. Chunk too big and retrieval gets imprecise; too small and each chunk loses context. The split strategy matters more than people expect, and we go deep on it in what are chunkers and text chunking.

2. Embed

Each chunk goes through an embedding model that turns text into a vector: a list of numbers that captures meaning. Similar text lands near similar text in vector space. You store these vectors in a vector database.

3. Retrieve

At query time, you embed the question with the same model, then find the chunks whose vectors are closest to the question's vector. Those are your candidate passages.

4. Generate

You assemble the retrieved passages into a prompt and send it to the LLM with instructions to answer from that context.

User query
   |
   v
[embed query] --> [vector search] --> top-k chunks
   |                                        |
   +----------------> [prompt] <------------+
                          |
                          v
                      [LLM] --> grounded answer

A minimal code sketch

Here is the shape of a RAG loop, kept deliberately small so the flow is visible.

# Indexing (once, ahead of time)
chunks = chunk_document(doc, size=400, overlap=50)
vectors = embed(chunks)              # embedding model -> list of vectors
store.add(vectors, metadata=chunks)  # vector database

# Querying (per request)
def answer(question: str) -> str:
    q_vec = embed([question])[0]
    hits = store.search(q_vec, top_k=5)    # nearest chunks
    context = "\n\n".join(h.text for h in hits)

    prompt = (
        "Answer using only the context below. "
        "If the answer is not in the context, say so.\n\n"
        f"Context:\n{context}\n\nQuestion: {question}"
    )
    return llm.complete(prompt)

That is the entire idea. Everything else (reranking, hybrid search, metadata filters) is refinement on top of this loop.

Where live web data fits

Most RAG tutorials assume your knowledge is a folder of PDFs sitting on disk. Real systems often need current information the model never saw and that no static corpus contains: today's news, a competitor's live pricing page, a documentation site that changed this morning. That is where a fetch-and-search layer plugs into the retrieve stage.

Instead of (or alongside) a static vector store, you retrieve from the live web, then feed the results into the same generate step:

curl https://api.link.sc/v1/search \
  -H "x-api-key: lsc_..." \
  -H "Content-Type: application/json" \
  -d '{"q": "current EU AI Act enforcement timeline", "engine": "google"}'

The response lists ranked results under serpData.results, each with a targetUrl. To turn a result into prompt-ready context, fetch it:

curl https://api.link.sc/v1/fetch \
  -H "x-api-key: lsc_..." \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/eu-ai-act-timeline", "format": "markdown"}'

Because the fetch response comes back as clean full-page Markdown rather than snippets, it drops straight into your prompt without a separate scrape-and-clean pass. You can also fetch a specific known URL the same way and chunk it into your store on the fly. For the full build with real-time sources, see building RAG pipelines with real-time web data.

Common failure modes

RAG is simple to start and easy to get subtly wrong. The usual culprits:

• Bad chunking. Passages split mid-sentence or too large to be specific. Retrieval degrades before the LLM ever sees the query.
• Retrieval, not generation, is usually the weak link. If answers are wrong, inspect what got retrieved before you blame the model.
• No grounding instruction. Without "answer only from context", the model happily blends retrieved facts with training memory and you lose traceability.
• Stale index. If your data changes and you do not re-index, RAG confidently serves yesterday's answer.

The takeaway

RAG is the default architecture when you want an LLM to answer over data it was not trained on, especially data that changes. You keep control of the knowledge, you get citations, and you update by writing documents instead of retraining. Start with the four-stage loop, get retrieval quality right first, and add live web data when your questions outrun any static corpus.

---

Building a RAG system that needs current, clean web content? Try link.sc free.