GraphRAG vs Vector RAG: When Knowledge Graphs Pay
Every retrieval-augmented generation system answers one question badly: questions about the whole corpus. Ask a vector RAG system “what were the three main themes across these 10,000 incident reports” and it will retrieve a handful of chunks that happen to be near your query embedding, then confidently summarise a sample it cannot see the edges of. That failure is what created GraphRAG, the most commercially significant knowledge graph pattern of the decade. This lesson gives you the family tree, the evidence for when graphs actually pay, and the cost truths vendors prefer to skip. By the end you should be able to answer the question every architect now faces: is the graph worth it, for this workload, at this budget?
The paper that started it: local to global
Microsoft Research’s “From Local to Global: A Graph RAG Approach to Query-Focused Summarization” (arXiv 2404.16130, April 2024) is the reference point. The pipeline: an LLM extracts an entity and relationship graph from the corpus, the Leiden algorithm clusters that graph into hierarchical communities, and the system pre-generates a summary for every community. A “global” question is then answered map-reduce style over community summaries rather than raw chunks. On corpus-level sensemaking questions it beat naive vector RAG on comprehensiveness and diversity, and the open-source release at github.com/microsoft/graphrag made GraphRAG a household term within months.
Notice what the graph is doing here. It is not a curated ontology; it is an index structure that gives the corpus a shape, so that summarisation can follow topology instead of embedding proximity. That distinction, index graph versus knowledge graph, will matter throughout this lesson.
The family it spawned
The successors form a readable evolutionary record, and the curated list at github.com/DEEP-PolyU/Awesome-GraphRAG tracks it, alongside the canonical 2025 survey (arXiv 2501.13958).
- LightRAG (arXiv 2410.05779, October 2024) attacked cost with dual-level retrieval, low-level entities plus high-level themes, over a graph-enhanced index with incremental updates. It became the most-forked lightweight alternative.
- LazyGraphRAG (Microsoft, November 2024, blog only, no arXiv paper) deferred all LLM summarisation to query time and reported roughly 0.1 percent of GraphRAG’s indexing cost at comparable quality. It is the standing counterexample to “you must pay for the expensive index”.
- KAG from Ant Group (arXiv 2409.13731) is the enterprise, schema-first branch: mutual indexing between text chunks and a knowledge graph, logical-form-guided reasoning, and schema constraints for professional domains such as finance and healthcare.
- Youtu-GraphRAG (arXiv 2508.19855, Tencent, August 2025) marks the agentic convergence: schema-guided construction and iterative agent-driven retrieval unified in one framework.
- Cost engineering continued through 2025: “Towards Practical GraphRAG” (arXiv 2507.03226) on industrial-scale construction with hybrid vector-plus-graph retrieval, HiRAG (arXiv 2503.10150) on hierarchical knowledge, and LinearRAG (arXiv 2510.10114) on linear-cost graph retrieval over large corpora.
Cloud platforms productised the pattern: Amazon Bedrock Knowledge Bases shipped GraphRAG generally available on Neptune Analytics in March 2025, followed by BYOKG-RAG for bring-your-own knowledge graphs in August 2025, while Microsoft rolled out Graph in Fabric from October 2025. When all three hyperscalers ship a pattern, the argument about whether it is real is over; the argument about where it pays begins.
Worked example · the Aberdeen-2 asset register
Where we left it: the register promoted to grounding layer. CQ5 is about to show exactly why.
Run CQ5 against a vector index of the work-order corpus. The retriever returns the reports most similar to “impeller swap” and “failure”: useful documents, all about the same one or two pumps. It cannot go further, for a structural reason worth stating precisely: the answer to CQ5 is written in no document. No work order anywhere says “P-204 shares a model with a pump that failed within 12 months of a swap”; that sentence only exists as a computation over entities. The register computes it as a traversal:
WO-2025-067 (impeller swap, Feb 2025)
--on--> P-117 <--on-- WO-2026-011 (failure, Jan 2026) # within 12 months
P-117 --a--> ex:HX300 <--a-- { P-101, P-204, ... } # lesson 25's level
=> answer: the HX300 fleet, minus P-117, with the work orders as evidenceFour hops, one aggregation, deterministic, and every hop exists because a specific lesson built it: the model level (22), the participation pattern (23), the typed dates (5). Honesty cuts the other way too: “what is P-101’s rated power?” is a single lookup, and standing up graph infrastructure for that class of question would be waste. The register serves the structural questions; the vector index keeps the factoids; a router in front decides, which is the hybrid the evidence supports.
State of the register: the graph half of a routed retrieval architecture. Next lesson: what happens when the graph itself is built by the model.
The evidence: where graphs pay and where they do not
For years the honest answer to “is the graph worth it” was folklore. Two 2025-2026 artefacts replaced folklore with measurement. “When to use Graphs in RAG: A Comprehensive Analysis” (arXiv 2506.05690, accepted at ICLR 2026) systematically compared graph and non-graph retrieval across query types. Its companion GraphRAG-Bench (arXiv 2506.02404) evaluates nine state-of-the-art GraphRAG methods across 16 college-level disciplines, scoring the entire pipeline: construction, retrieval, generation and reasoning coherence, not just final answer accuracy.
The findings compress into a usable decision aid. Graphs deliver on multi-hop questions, schema-bound aggregation and corpus-level sensemaking. They add little, while adding cost and latency, on single-hop factoid lookup, which is most of many production workloads. Separately, OG-RAG (arXiv 2412.15235, EMNLP 2025) showed that grounding retrieval in a curated domain ontology, rather than an auto-extracted entity graph, produced 55 percent higher fact recall and 40 percent higher correctness across four LLMs: evidence that the quality of the semantic layer, not merely its existence, drives the payoff. Treat vendor benchmarks with more caution: FalkorDB publishes results showing vector RAG near 0 percent versus GraphRAG above 90 percent on schema-bound aggregation queries, which is directionally consistent with the independent evidence but comes from a graph database vendor measuring the query class graphs are best at.
| Workload characteristic | Vector RAG | GraphRAG | Evidence |
|---|---|---|---|
| Single-hop factoid lookup | Strong, cheap | Overhead without benefit | arXiv 2506.05690 |
| Multi-hop questions across documents | Weak | Strong | arXiv 2506.05690, GraphRAG-Bench |
| Corpus-level themes and sensemaking | Samples blindly | Designed for it | arXiv 2404.16130 |
| Schema-bound aggregation (“all X related to Y via Z”) | Near-total failure | Strong | arXiv 2506.05690; vendor benchmarks agree |
| Regulated domains needing verifiable grounding | No provenance structure | Ontology-grounded wins | OG-RAG, arXiv 2412.15235 |
| High-churn corpus, tight indexing budget | Strong | Use LazyGraphRAG or LightRAG style | arXiv 2410.05779 |
Read the benchmark’s design as carefully as its scores. GraphRAG-Bench’s decision to score construction quality, retrieval relevance, generation faithfulness and reasoning coherence as separate stages is itself a lesson: a GraphRAG system can fail at any of the four, and a single end-to-end accuracy number hides which one failed. When you evaluate a vendor claim, ask which stage their number measures. A system with superb retrieval over a badly extracted graph will look good on paper and mislead in production, which is why Lesson 40 treats construction validation as its own discipline rather than a footnote to retrieval.
What breaks · GraphRAG over an unresolved graph
A rival team skips the register and auto-extracts a graph straight from the work-order texts. The extractor, meeting free prose, mints what it sees:
(P-101) --SWAPPED--> (impeller) # from WO-2022-114's text
("Pump P101") --FAILED--> (bearing) # same pump, second node
("the aft pump") --INSPECTED--> ... # same pump, third nodeOne physical machine is now three nodes, and every traversal silently forks. CQ5 run against this graph follows the swap from the “P-101” node, finds no failure there because the failure attached to “Pump P101”, and returns a confident, incomplete answer; the community summaries count the pump’s incidents twice and report a phantom trend. This is the lesson’s warning made concrete: extraction errors become permanent retrieval infrastructure, load-bearing for every future query. And the fix is nothing exotic. Entity resolution is exactly what the register already owns: stable IRIs from lesson 5, extent-based identity from lesson 26, mapping rows from lesson 29. The OG-RAG result, 55 percent better recall from a curated semantic layer, is measuring precisely the distance between this block and the one above it.
The cost and latency truths
Three truths belong in every proposal. First, indexing is where GraphRAG hurts: classic GraphRAG makes LLM calls across the entire corpus at build time, which is why LazyGraphRAG’s roughly thousandfold indexing cost reduction mattered so much, and why incremental-update designs like LightRAG exist. Budget the re-index on every corpus change, not just the first build. Second, query-time latency depends on architecture, not on “graphs” in the abstract: community-summary map-reduce is slow and thorough, local entity-neighbourhood retrieval is fast, and hybrid designs route between them. Third, the graph you extract is a liability as well as an asset: entity resolution errors and hallucinated relations become permanent retrieval infrastructure unless you validate the construction step, which is exactly the subject of Lesson 40.
Architecture patterns to carry into practice
Four patterns cover most real systems. Global-first: community summaries answering corpus-level questions, the original Microsoft design. Local-first: entity neighbourhood expansion for multi-hop precision. Hybrid routing: classify the query, send factoids to the vector index and structural questions to the graph, which the ICLR 2026 evidence directly supports. Ontology-grounded: OG-RAG’s approach, where an authored ontology defines the hypergraph of fact clusters and retrieval selects a minimal cover, the pattern of choice where correctness is contractual.
A practical note on tooling: LlamaIndex’s Property Graph Index supports schema-guided extractors and combined vector-plus-graph retrievers over backends including Neo4j and Memgraph, and Neo4j maintains the official neo4j-graphrag Python package. You do not need to build the plumbing from scratch, and free vendor training such as Neo4j GraphAcademy covers the mechanics; what vendors will not give you is the semantics-grounded judgement this course is built around.
A worked example makes the routing concrete. Suppose a compliance team asks three questions of a contract corpus. “What is the notice period in the Acme agreement” is single-hop: the vector store answers it in one retrieval, and building a graph for it would be waste. “Which of our suppliers are indirectly exposed to a sanctioned entity through any chain of subsidiaries” is multi-hop and schema-bound: vector similarity has no concept of a chain, and this is where the near-total failure documented for vector retrieval on aggregation queries bites. “What patterns of unusual indemnity clauses appeared across last year’s contracts” is sensemaking: community summaries were invented for it. One corpus, three questions, three correct architectures, and the router in front of them is where your judgement earns its keep.
Check yourself
Four questions arrive at the register’s retrieval stack. Which is the multi-hop, schema-bound class where the evidence shows vector retrieval failing near-totally?
- Which pumps share a model with a pump that failed within 12 months of an impeller swap?
- What is P-101’s rated power?
- What recurring failure themes ran across last year’s four thousand work orders?
- Summarise the report attached to WO-2022-114.
Show the answer, and why each wrong option is wrong
A is correct. Chained hops through typed relationships plus an aggregation over a class of entities: the answer exists in no retrievable passage and must be traversed. This is the query family where the vendor benchmark’s near-zero vector score, and the independent ICLR 2026 findings, converge.
B is a single-hop factoid: one attribute on one entity, answered by the cheapest index you own. Routing it to the graph costs latency and buys nothing, which is graph maximalism, the posture this lesson warns against as much as vector maximalism. C does want the graph, but through the other mechanism: corpus-level sensemaking via community summaries, not multi-hop traversal, and conflating the two families is how architectures end up with the wrong index for both. D needs no retrieval structure at all: the document is already identified, and summarising it is a pure LLM job. Four questions, four different right answers; the router, not the religion, is where the engineering lives.
The verdict
So when do knowledge graphs pay in RAG? When your questions have structure: multi-hop chains, aggregations, whole-corpus patterns, or compliance-grade grounding. When they do not, a vector store is cheaper and equally accurate, and the intellectually honest move is to say so, then design the hybrid. The strongest position for a knowledge graph engineer in 2026 is not graph maximalism; it is owning the routing decision with evidence. If you want to go deeper on the validation side that makes ontology-grounded RAG trustworthy, the Running Open Ontologies course picks up exactly where this lesson stops.
