A knowledge graph stores information as entities (people, companies, products, documents) connected by named relationships (works for, supplies, replaces, mentions). Instead of keeping facts in separate tables or text files, it keeps the links between them, so you can ask questions that follow those links: which suppliers of this part also supply our competitors, or which contracts are affected by this regulation. GraphRAG is the use of such a graph to give a large language model connected, traceable context when it answers.
How a Knowledge Graph Works
The basic unit is a statement of three parts: a subject, a relationship and an object. “Acme Ltd, supplies, Part 4711” is one statement. The W3C’s RDF standard defines exactly this: an RDF triple has a subject, a predicate and an object, and an RDF graph is a set of such triples.
There are two common ways to build one:
- RDF and ontologies. Standard formats with formal schemas (ontologies) and the SPARQL query language. Strong for data integration across organisations and for reasoning over class hierarchies.
- Property graphs. Nodes and edges that carry their own attributes, queried with languages such as Cypher or GQL. Popular in application development because they are quick to model and query.
Either way, a graph needs a schema that says which entity types and relationships exist, and a process to keep the data current.
GraphRAG: Knowledge Graphs with LLMs
Standard retrieval-augmented generation finds text chunks similar to the question using embeddings and a vector database. That works for “what does this document say about X”. It struggles with two kinds of question:
- Multi-hop questions, where the answer depends on following several links (supplier to part to product to customer).
- Whole-collection questions, such as “what are the main themes across these 10,000 reports”.
GraphRAG adds a graph to the retrieval step. Microsoft Research’s GraphRAG paper describes one approach: a language model reads the documents and extracts an entity graph, groups closely related entities into communities and writes a summary for each, then answers global questions by combining partial answers from the relevant community summaries. The authors report more comprehensive and more diverse answers than a conventional RAG baseline on collections of about one million tokens.
Other GraphRAG variants are simpler: retrieve text chunks as usual, then pull in the entities and relationships linked to them, or let an AI agent query the graph through a tool.
When to Use a Knowledge Graph
| Good fit | Probably not worth it |
|---|---|
| Questions that follow relationships across many records | Questions answered by one paragraph in one document |
| Data from many systems that must be joined by entity | A single, well-structured database already exists |
| Answers that must show the chain of evidence | A small document set that fits in the context window |
| Stable domains with clear entity types (products, parts, regulations) | Fast-changing content nobody will maintain |
Common Failure Modes
- Extraction errors. When an LLM builds the graph, it can merge two different companies with similar names or invent a relationship. Those errors then look like facts.
- Cost of indexing. LLM-based graph extraction over a large corpus costs far more than computing embeddings. Budget it and update incrementally.
- No owner. A graph that nobody keeps current becomes wrong quietly.
- Over-modelling. A schema with hundreds of relationship types is hard to populate and hard to query. Start with the few that answer real questions.
Related Terms
A knowledge graph is a form of structured data that is often extracted from unstructured data. It complements vector search rather than replacing it, and it is one of the tools of context engineering.
In Practice
The graph is modeled from the questions users ask, and only the entities and relationships those questions need go into it; a schema meant to describe everything is rarely finished. Each extracted relationship keeps a link to the passage it came from, and a domain expert checks a sample of extractions before the graph serves answers. Graph and vector retrieval are compared on the same question set, and the graph is kept for the question types where it wins, usually those that need several hops.
We build graph-backed retrieval as part of RAG and enterprise knowledge assistants and natural language processing services. For an example of grounded search over specialist content, see the medical drug search accelerator case.