Graph RAG
Plain vector search finds chunks that sound like your question. Graph RAG turns those chunks into a graph of entities and how they connect, then walks the links to answer questions no single chunk can answer.
Why Does This Exist?
Flat retrieval is superb at one job: finding the passage that looks most like your question. Ask for a single fact kept in one place and vector databases hand it over. You break it the moment the answer lives nowhere in particular.
Take three short reports. One says Ava leads Helio and Helio makes Volt. One says Volt needs Kite chips. One says Northwind supplies Kite chips. Now ask how Ava is linked to Northwind. No single report holds both names, so the top Andreea is always half the story. The missing work is not ranking better. It is following connections across reports.
Graph RAG does that follow-up structurally. At index time it converts chunks into a knowledge graph: people, products, and companies become nodes, and each stated relationship becomes a typed edge between them. At query time it walks those edges instead of re-ranking chunks, so a four-hop chain like the one above arrives as one connected argument.
This page covers the graph index and the walk: what gets extracted, how retrieval traverses it, and what it costs. The plain pipeline underneath lives on RAG architecture. The sibling trick of splitting the question instead of structuring the index lives on query decomposition.
The one idea you need first. A knowledge graph stores facts as triples, one (head, relation, tail) tuple per fact. The triple (Ava, LEADS, Helio) says exactly one thing and points at exactly two nodes. Everything below is operations on those tuples.
Think of It Like This
A corkboard with photos and string
You are tracking how a founder connects to a supplier, and your desk holds a stack of interview transcripts. Reading the stack in order never shows the link, because each transcript mentions only its own two characters.
So you pin a photo of each person and company to a corkboard and run string between the photos each transcript connects: founder to startup, startup to product, product to part, part to supplier. When someone asks the question, you do not reread the stack. You put one finger on the founder's photo and follow the string, knot by knot, until you reach the supplier.
Where it stops: the board knows only what you pinned. A wrong transcript makes a wrong string, two photos of the same person split the trail in half, and nobody updates the board when the world moves on. Those three failures have names below: bad extraction, missed aliases, and a stale graph.
How It Actually Works
Two phases, and both differ from flat RAG: an extraction index you pay for once, and a traversal retrieval you run per question.
The graph you build
Call the graph . Here is the set of entities (every distinct person, product, and company), and is the set of typed edges (every stated relationship, with its type kept, not thrown away). One fact is one triple , where is the head entity the fact starts from, is the named relation, and is the tail entity it points at.
Indexing runs chunk by chunk. An extraction model reads chunk and emits the triples stated there, and each triple keeps the id of the chunk it came from, so every edge carries its provenance. Then comes the step everyone underestimates: resolution. "Northwind" in one chunk and "Northwind Ltd" in another must become one node before loading, or the graph splits and later walks die at the gap.
Large deployments add one more layer. Community detection groups densely linked nodes, and a summarizer writes a paragraph per community, building upward from specifics to themes. Those summaries are what global questions read: instead of retrieving thousands of sentences, the system retrieves the summary of the community the question points at.
How a question is answered
Write the question as . A linker maps the names inside onto nodes, giving the seed set . From there, retrieval has two gears. Local search expands the -hop neighbourhood , meaning every node reachable within edges of a seed, and returns the subgraph it finds. Global search matches against the community summaries and returns the best-fitting ones. Either way, the generator receives connected evidence with a path it can quote, not a pile of lookalikes.
In our running example every edge points forward along the answer, so a plain directed walk suffices. Most implementations index each edge in both directions, because real questions arrive from either end of a relationship.
Worked example
Three chunks, four triples:
- : "Ava leads Helio. Helio makes Volt." That is (Ava, LEADS, Helio) and (Helio, MAKES, Volt).
- : "Volt needs Kite chips." That is (Volt, NEEDS, Kite).
- : "Northwind supplies Kite chips." That is (Kite, FROM, Northwind).
Count the graph: entities (Ava, Helio, Volt, Kite, Northwind) and typed edges. Now the question : "How is Ava linked to Northwind?" The two names in are Ava and Northwind. Chunk overlap, counted in names: holds 1 of 2, holds 0 of 2, holds 1 of 2. Flat retrieval tops out here. Its best chunk is half the answer by construction.
The graph walk starts from seeds . One hop from Ava reaches Helio. One hop from Northwind reaches Kite. No overlap yet. Two hops from Ava reach Helio, then Volt. Two hops from Northwind reach Kite, then Volt. The two sides meet at Volt. The returned path is , with length edges. That is the diagram on this page: three isolated boxes on the left, one four-hop chain on the right.
What it costs and when to skip it
Here's the trade-off, stated plainly. You pay an extraction model call per chunk at index time, plus resolution and possibly summary passes, where flat RAG pays one cheap embedding per chunk. What you buy is multi-hop recall and questions about the whole corpus that flat search cannot even attempt.
Skip the graph when every question is a single fact that flat search already finds, when the corpus is small enough to fit in context, or when sources change faster than you rebuild. A stale edge misleads with full confidence, which is worse than no edge at all. And when answers go wrong despite the graph, debug in stage order on RAG failure diagnosis before blaming the walk.
Code
The snippet below is the walk itself, stripped to its bones: breadth-first search over the four triples, returning the path flat lookup could not assemble.
from collections import deque
def bfs_path(graph: dict[str, list[str]], start: str, goal: str) -> list[str]: """Return the shortest path from start to goal as a node list.""" queue: deque[tuple[str, list[str]]] = deque([(start, [start])]) seen: set[str] = {start} while queue: node, path = queue.popleft() if node == goal: return path for nxt in graph[node]: if nxt not in seen: seen.add(nxt) queue.append((nxt, path + [nxt])) return []
graph: dict[str, list[str]] = { "Ava": ["Helio"], "Helio": ["Volt"], "Volt": ["Kite"], "Kite": ["Northwind"], "Northwind": [],}
path = bfs_path(graph, "Ava", "Northwind")print(path) # -> ['Ava', 'Helio', 'Volt', 'Kite', 'Northwind']print(len(path) - 1) # -> 4Five nodes in, five nodes out, path length 4. That 4 is the same number in the worked example and on the diagram.
Watch Out For
One company, three nodes
The symptom is specific: a question with a provable two-hop answer comes back claiming no connection, while each hop answers fine on its own. You check the graph and find "Northwind" in one place, "Northwind Ltd" in another, and "the supplier" in a third. Three nodes, split edges, no path between them.
Resolve before you load. Normalize case and punctuation, merge aliases onto one canonical node, and keep the raw spellings as display attributes. Re-run resolution whenever new chunks arrive, because aliases drift as sources pile up.
Edges nobody wrote
The symptom is a crisp answer with a relationship you cannot find. The response states that Northwind owns Helio, you open the cited chunks, and ownership appears in none of them. The extractor filled a gap with a plausible triple and the graph indexed it like a fact.
Give every edge provenance. Store the source chunk id with each triple, traverse only edges that point at real chunks, and show those chunks beside the answer. A triple without a source is a guess: drop it or flag it, never index it silently.
The Quick Version
- Graph RAG indexes text as a knowledge graph of (head, relation, tail) triples instead of a pile of chunks.
- A question becomes a walk: link its names to seed nodes, expand hop by hop, and return the connected path.
- In the running example, 3 chunks yield 5 entities and 4 edges, and the answer is the 4-hop path from Ava to Northwind, with the two sides meeting at Volt.
- Flat search caps out where the example starts: its best chunk names 1 of the 2 query names, so no top Andreea holds the answer.
- Budget for extraction and alias resolution at index time, and skip the graph when every question is a single fact that flat search already finds.
- Every edge must point at the chunk it came from, or plausible invented relationships will read as facts.