Our RAG pilot keeps missing links between records. Knowledge graph, or better retrieval?

Tune the retrieval pipeline first, because most pilots that miss links are failing at identity and lexical matching rather than at graph topology. On a financial text-and-table benchmark, hybrid retrieval plus a reranker lifted Recall@5 from 0.587 to 0.816 against dense-only search, a larger swing than any published graph-versus-vector comparison. Add a graph when the questions genuinely chain facts across systems, and only after you can say which records refer to the same entity.

Why your pilot keeps missing links

A pilot that misses policy-to-provider or claim-to-adjuster links is usually failing at entity resolution, not at retrieval topology. The same provider appears as four different strings across a claims system, a provider directory, and a warehouse extract, and no retrieval architecture recovers a link that was never resolvable in the source data. A knowledge graph in the query path does not fix that. It encodes it.

The evidence is now direct. Deg-Rag ran entity resolution over LLM-built graphs, removed roughly 40 percent of entities and 31 to 62 percent of relations, and found question-answering quality improved on the smaller graphs across every variant tested, including LightRAG, HippoRAG, and both Microsoft GraphRAG modes. Less graph, better answers, because what remained was trustworthy. If duplicate nodes degrade a graph that much, they were already degrading the RAG pipeline the graph was meant to replace.

Which puts identity upstream of this decision rather than inside it. A rules-plus-ML matching layer in your warehouse, or a tool like Senzing or Zingg, produces stable identifiers before anything is indexed, and it improves every path below. It is unavailable only when sources share no attribute stable enough to match on, the risk with a mainframe extract predating any common customer key.

A second failure looks like a missing link and is not. The HybridRAG authors, working on financial filings, found that graph retrieval “sometimes fails to answer questions correctly whenever there is no entity explicitly mentioned in the question.” A graph rewards queries that name their anchors, so an adjudication question phrased as a scenario rather than a policy number gets worse from traversal. Read which benchmarks favor graph retrieval and which favor vector search before scoping anything.

Is a graph worth it across a lakehouse, legacy warehouses, and a mainframe extract?

Not as the first move, because a graph built over that estate inherits every identity problem the sources have and makes it structural. An LLM extracting entities from a mainframe extract with no shared customer key produces nodes that look like distinct providers and are not, and those duplicates then get traversed with confidence. Practitioners report six to twelve weeks to stand up a graph for one domain, and every source or schema change reopens part of that work.

The version that works is narrower than the pitch. LinkedIn’s deployed system built a graph from historical support issues, preserving the structure inside each ticket and the relationships between tickets, and reported a 77.6 percent improvement in mean reciprocal rank and a 28.6 percent cut in median per-issue resolution time. What made it work was one well-understood corpus with structure the team already owned, not a graph spanning four heterogeneous systems.

What the accuracy numbers actually support

The published gains are real, roughly ten points, and confined to the query types graphs are built for. GraphRAG-Bench measured graph methods at 53.4 percent on complex reasoning against 42.9 for text chunks, and 64.4 against 51.3 on contextual summarization. On simple fact retrieval the two effectively tied, 60.1 against 60.9. Any business case that quotes the reasoning number without the retrieval number is overstating it.

The counter-evidence deserves equal weight. A 2026 study titled “UnWeaving the knots of GraphRAG” found a well-built vector pipeline using entity-based decomposition beat standard GraphRAG and came close to state-of-the-art graph systems, at a fraction of the cost. HybridRAG, on financial filings, found the graph improved faithfulness marginally, 0.96 against 0.94, while its context recall dropped to 0.85 from a clean 1.0 for vector retrieval. Graphs trade recall for precision, and in adjudication, missing a relevant document is usually the more expensive error.

What actually differs between better retrieval and a graph?

They fail in opposite directions, which is what makes this measurable rather than architectural. Better retrieval raises recall when the answer already sits in one passage; a graph buys traversal at the price of a pipeline someone has to keep working.

Your options

Tuning the retrieval pipeline you have is the cheapest experiment and the one most pilots skip on the way to a graph. On the T2-RAGBench financial corpus, BM25 beat a state-of-the-art dense retriever on nearly every metric, because identifiers like company names and ticker symbols are exactly what embeddings dilute. Adding a reranker took Recall@5 to 0.816. That is where the first ten points live when documents carry identifiers, and chunk size decides more of it than most teams measure.

GraphRAG over your corpus is what people usually mean by GraphRAG. An LLM reads every chunk, extracts entities and typed relationships, and answers over the resulting graph and its community summaries. It earns its cost on multi-hop questions and whole-corpus synthesis, and breaks on indexing bills, reconciliation after every source change, and duplicate entities the extractor invents on messy data.

A property graph in the query path is the older and more predictable version: a modeled schema in Neo4j or similar, populated by ETL you control, queried directly. You trade automatic extraction for determinism, a reviewable schema, and answer paths an auditor can follow. It breaks when the schema has to move quickly or nobody owns the ETL.

A container that resolves relationships when content is written moves the work from query time to write time. Relationships between entries are established as content lands, so an agent walks from an entry to its related entries with no graph build, and Wire is one implementation. It does not do deterministic joins over system-of-record tables, and a regulator wanting a queryable schema over policy data wants a property graph.

How to decide

What fraction of your failed queries are actually multi-hop? Label a hundred failures by hand before buying anything. If most are one fact retrieval did not surface, you have a ranking problem and a graph will not touch it. It settles the question faster than any vendor evaluation, and it is what stops retrieval failures from being mistaken for reasoning failures.

Can you already join the records? If not, entity resolution is not a step on the way to a graph, it is the project. Do it first and rerun the pilot before deciding anything else.

Does someone need to see the path? In regulated adjudication the argument is often provenance rather than accuracy: an entity and relationship path is defensible to an auditor in a way passage citations are not. That justifies the cost even when the accuracy delta is small, but it points at a modeled property graph, not an LLM-extracted one.

What to do next

Build the evaluation set before the architecture. A labeled set of failed queries, split by whether the answer lives in one passage or several, turns this from an argument into a measurement. Then run the cheap arm against it: hybrid search plus a reranker on the corpus you already have. If the multi-hop share is high and identity is clean, the benchmark detail on where graphs win tells you which variant to scope.

If the context in question is written by agents rather than owned by a warehouse, the write-time option is the one worth pricing. Wire resolves relationships between entries as they are written, so agents get traversal with nobody owning an extraction pipeline, and how its retrieval measures up is published rather than asserted. It is not a substitute for a property graph over system-of-record tables, which is the situation two of the queries above describe.

Sources: When to use Graphs in RAG: GraphRAG-Bench (arXiv 2506.05690) · Microsoft Research: LazyGraphRAG · Knowledge graph construction for AI · Less is More: Denoising Knowledge Graphs for RAG (arXiv 2510.14271) · UnWeaving the knots of GraphRAG (arXiv 2603.29875) · From BM25 to Corrective RAG (arXiv 2604.01733) · RAG with Knowledge Graphs for Customer Service QA (arXiv 2404.17723) · HybridRAG (arXiv 2408.04948)