Ask a plain vector RAG system "what does this contract say about termination notice periods?" and it does fine. Ask it "which suppliers are indirectly exposed if this one factory shuts down?" and it falls apart, because the answer isn't sitting in any single chunk. It's spread across a chain of relationships that similarity search was never designed to follow.
GraphRAG is the attempt to fix that. It's also one of the easiest ways to triple your pipeline complexity for a use case that didn't need it. Let's cover both halves honestly.
What GraphRAG Actually Is
GraphRAG (the term was popularized by Microsoft Research's 2024 paper and open-source project) replaces or augments the "embed chunks, search by similarity" step with a knowledge graph built from your corpus.
The indexing side looks like this:
- Split documents into chunks, same as regular RAG.
- Run an LLM over each chunk to extract entities (people, companies, products, concepts) and relationships between them ("Acme supplies batteries to Volta Motors").
- Merge those extractions into a graph: nodes are entities, edges are relationships.
- Optionally, detect communities (clusters of related entities) and have the LLM write a summary of each cluster.
At query time, instead of just pulling the top-k most similar chunks, the system can walk the graph: find the entities the question mentions, follow their edges to related entities, and pull in facts the question never named directly. For broad questions, it can answer from the community summaries instead of raw chunks.
That last part is the underrated bit. Plain vector RAG has no concept of "the whole corpus." GraphRAG's hierarchy of summaries gives it one.
Where Vector RAG Genuinely Fails
Vector search retrieves chunks that look like the question. That works when the answer lives in one or two passages. It breaks in two specific, predictable situations.
Multi-hop questions. "Which board members of Company A also advise startups funded by Investor B?" No chunk contains that sentence. The answer requires joining fact 1 (board membership) with fact 2 (advisory roles) with fact 3 (funding relationships), each from a different document. Vector similarity will retrieve chunks about Company A and maybe Investor B, then hope the LLM stitches it together from incomplete context. Usually it hallucinates the join.
Global questions. "What are the main themes across these 500 customer interviews?" Top-k retrieval returns k chunks. Twenty chunks out of 500 documents is not "the main themes," it's a biased sample. Microsoft's own benchmark motivation was exactly this class of question, which they call global sensemaking, and their reported win was on comprehensiveness and diversity of answers, not on simple factual lookup.
If your queries are mostly "find the passage that answers this," you don't have either problem. Be honest about that before you build anything.
The Cost Math Nobody Puts in the Demo
Here's the part that gets skipped in every GraphRAG demo: indexing now costs LLM inference on every chunk.
Plain vector RAG indexes with an embedding model, which is cheap. Embedding a million tokens costs pennies. GraphRAG's entity extraction runs a full LLM prompt over every chunk, often with a long extraction prompt and multiple gleaning passes. Depending on the model, indexing the same corpus can cost 10x to 100x more than embedding it. Early adopters of the Microsoft implementation regularly reported three and four figure indexing bills on corpora that would have cost a few dollars to embed.
And that's just the build. The ongoing costs are structural:
| Vector RAG | GraphRAG | |
|---|---|---|
| Indexing cost | Embedding calls, cheap | LLM extraction per chunk, expensive |
| Incremental updates | Re-embed changed chunks | Re-extract, then merge entities into existing graph |
| Infrastructure | One vector store | Vector store plus graph store (or a combined engine) |
| Query latency | One similarity search | Entity linking, graph traversal, possibly summary lookups |
| Failure modes | Bad chunks, weak recall | All of those, plus wrong entity merges and stale edges |
| Debuggability | Inspect retrieved chunks | Inspect extraction quality, merge logic, traversal path |
Entity resolution deserves its own warning. The extractor will produce "Acme Corp," "Acme Corporation," and "ACME" as three nodes. Merge them too aggressively and you fuse different companies; too conservatively and your graph is fragmented and traversals dead-end. This is a real data engineering problem, and it never fully goes away.
My rule of thumb: GraphRAG earns its cost when at least a meaningful fraction of your query load is multi-hop or global, when your corpus is entity-dense (contracts, supply chains, research literature, intelligence analysis), and when the corpus is stable enough that you're not rebuilding the graph weekly. A support chatbot over product docs meets none of those criteria. Start with vector RAG, log the questions it fails on, and let that failure log make the case for a graph.
There's also a middle path worth trying first: keep vector retrieval, but add lightweight metadata (entities tagged per chunk, cross-references) and do one join in application code. It covers a surprising share of "multi-hop" needs at almost no extra cost.
Feeding the Graph: Where the Data Comes From
Everything above assumes you have clean text to extract from. If your graph sources include the web (company sites, news, filings, product pages) that assumption is doing a lot of work.
Entity extraction is unusually sensitive to input noise. If your fetched page is 40 percent navigation links, cookie banners, and footer boilerplate, the extractor will happily emit garbage nodes: "Privacy Policy" as an entity, "Read More" related to everything. Every junk node pollutes traversals later, and unlike a bad chunk in vector RAG, a bad node doesn't just fail to be retrieved. It actively creates wrong paths.
So the pipeline front-end matters more here, not less:
import requests
r = requests.post(
"https://api.link.sc/v1/fetch",
headers={"x-api-key": "lsc_your_key"},
json={"url": "https://example.com/suppliers", "format": "markdown"},
)
clean_md = r.json()["content"]
# feed clean_md to your entity extraction prompt
Clean markdown in, plausible entities out. The link.sc fetch API handles the fetch-and-clean step in one call, and because extraction is the expensive stage, every boilerplate token you strip before it is money saved twice: once on extraction cost, once on graph cleanup you don't have to do.
The refresh problem also bites harder with graphs. A supplier relationship extracted in January may be false by June, and a stale edge is worse than a stale chunk because traversals treat it as fact. If you're building graphs from web sources, schedule re-fetches by source volatility, the same discipline covered in our RAG pipeline guide for live web data, and version your edges with an extracted-at timestamp so you can expire them.
The Short Version
GraphRAG is real, not hype: for multi-hop and corpus-wide questions over entity-dense data, it retrieves things vector search structurally cannot. But it converts a cheap indexing step into an expensive LLM extraction pipeline, adds a graph store and an entity-resolution problem, and pays off only when your query mix actually needs relational reasoning.
Start with plain vector RAG and good chunking. Collect the questions it fails. If the failures are joins and themes rather than lookups, you have a genuine GraphRAG use case, and you'll know exactly which relationships the graph needs to capture.
Building a RAG or GraphRAG pipeline on web data? Get a free link.sc API key and start with 500 free credits a month for clean, LLM-ready markdown from any URL.