From Local to Global: A Graph RAG Approach to Query-Focused Summarization

In plain words

Ordinary retrieval-augmented generation fetches a few passages that look like the question and answers from those. That works for "what does this document say about X?" but not for "what are the main themes in this whole collection?", because no handful of passages holds that answer. GraphRAG reads the whole collection ahead of time with a language model. It writes down the entities it finds and how they relate, which forms a graph, then groups the graph into clusters and writes a short report for each cluster, at several levels of detail.

At question time, each report produces a partial answer, and the partial answers are merged into one reply. The authors' tests on two large collections found that this gave more thorough and varied answers than ordinary retrieval on questions about the whole collection. Building the index is slow: 281 minutes for the Podcast collection in the authors' run.

TL;DR

GraphRAG indexes a corpus by having an LLM extract entities, relations and claims into a graph, clustering that graph with Leiden, and pre-writing an LLM summary for every community at every level §3.1. A query is answered map-reduce style over those summaries §3.1.6. Use it for corpus-wide sensemaking, not single-fact lookup. The evidence is narrow: two corpora of about 1M tokens each and 125 LLM-judged questions per corpus §4.1.

Problem

Vector RAG retrieves records similar to the query and works when the answer is localised in a few of them. It does not support sensemaking questions that need a global view of the whole dataset §1. Query-focused summarisation methods can produce such views but, the authors state, do not scale to the amount of text that typical RAG systems index §1. Common open-domain QA benchmarks test explicit fact retrieval, so they do not measure this task §2.3.

Method

GraphRAG has two phases. Indexing builds the graph, communities and community summaries once per corpus §3.1, fig. 1. Query time runs map-reduce over the summaries §3.1.6.

Indexing

1. Split each document into text chunks. Chunk size is a design choice: larger chunks cost fewer LLM calls but lose recall of early content §3.1.1.
2. Prompt the LLM to extract entities (name, type, description) and relationships (source, target, description) from each chunk. Optionally extract claims §3.1.2 app. A.1.
3. Glean: after the first extraction, ask whether entities were missed, and if so ask for the missed ones. This allows larger chunks without losing entities app. A.2 fig. 3.
4. Merge instances into graph nodes and edges by exact string match. Entity descriptions are summarised per node and edge. Duplicate relationships become edge weights §3.1.3.
5. Partition the graph with Leiden, applied recursively inside each community until the leaves cannot be partitioned further. Each level is a partition covering all nodes exactly once §3.1.4 app. B fig. 4.
6. Summarise communities bottom-up. A leaf community's summary is built from its element descriptions (node, edge, claims), added in priority order until the token budget is full. A higher community uses its element descriptions if they fit; otherwise it swaps the largest sub-community's elements for that sub-community's summary, repeating until it fits §3.1.5.

Community reports are JSON with a title, summary, an impact rating from 0 to 10 with a one-sentence explanation, and 5 to 10 findings. Each finding cites record ids and lists at most five, then "+more" app. E.2.

Global question answering

# Query-time global answer  [§3.1.6]
def global_answer(query, summaries, budget=8000):
    # Shuffle, then pack summaries into windows of at most `budget` tokens
    # (the budget is 8k in the experiments)  [§3.1.6] [§4.1.3]
    random.shuffle(summaries)
    windows = pack_into_windows(summaries, budget)

    # Map: one partial answer per window, run in parallel; the LLM also
    # gives each answer a 0-100 helpfulness score  [§3.1.6]
    partials = parallel_map(lambda w: answer_and_score(query, w), windows)

    # Drop answers with score 0  [§3.1.6]
    partials = [p for p in partials if p.score > 0]

    # Reduce: add partials best-first until the budget is full  [§3.1.6]
    partials.sort(key=lambda p: p.score, reverse=True)
    context = pack_in_order(partials, budget)
    return summarise(query, context)  # final answer, with [Data: ...] refs

Question generation (for evaluation). An LLM is given a corpus description and asked to describe K user personas, N tasks per persona, and M global questions per (persona, task) pair. With K = M = N = 5 this gives 125 questions per dataset §3.2 alg. 1.

Evaluation criteria. Answers are compared head-to-head by an LLM judge on Comprehensiveness, Diversity and Empowerment. Directness is a control criterion §3.3. A winner scores 100, the loser 0, a tie 50 per condition app. G, tab. 6. Each pairwise comparison is judged five times §5.2 app. G.

Experiment 2 (claim-based check). Factual claims are extracted from each answer with Claimify. Comprehensiveness is the mean number of unique claims per answer. Diversity is the mean number of clusters after agglomerative clustering with complete linkage on a 1 − ROUGE-L distance, reported over thresholds 0.5 to 0.8 §4.2 tab. 3 tab. 4.

Hyperparameters

Results

Author-reported. Setup: two corpora of about 1M tokens (Podcast: Behind the Tech with Kevin Scott transcripts; News: a multi-category news benchmark) §4.1.1. Six conditions: C0 to C3 (GraphRAG community levels, root to low), TS (map-reduce over source text), SS (vector RAG) §4.1.2. The introduction says the results use GPT-4 as the LLM §1; the exact model for answering and for judging is not stated.

Graph size. Podcast: 8,564 nodes and 20,691 edges. News: 15,754 nodes and 19,520 edges §5.1.

Context units and tokens per condition tab. 2:

Head-to-head win rates of the row condition over SS (vector RAG), in percent. Four metrics per dataset: Comprehensiveness (Comp.), Diversity (Div.), Empowerment (Emp.), Directness (Dir.) fig. 2:

Head-to-head win rates of the row condition over TS, Comprehensiveness and Diversity only fig. 2:

Statistical tests on all pairs, with Holm-Bonferroni correction, are in app. G. Author-stated significance: all global conditions beat SS on comprehensiveness and diversity in both datasets §5.1 app. G. Empowerment results are mixed and directness favours SS, which is the expected control result §5.1 fig. 2.

Claim-based check (Experiment 2) §5.2. Average claims per answer tab. 3:

Average number of clusters (diversity) by distance threshold tab. 4:

Claim statistics: 47,075 unique claims over all answers, about 31 per answer §4.2. Claim-based and LLM-judge outcomes agree on 78% of non-tied comprehensiveness comparisons and 69-70% of diversity comparisons, where ties were 33% and 39% of the pairs §5.2.

Token cost §5.1 tab. 2: root-level C0 needs over 97% fewer context tokens than TS on both datasets. C3 needs 26-33% fewer.

Headline numbers quoted in the paper §5.1: comprehensiveness win rate over SS 72-83% (Podcast) and 72-80% (News); diversity 75-82% (Podcast) and 62-71% (News). The text also quotes 72% comprehensiveness and 62% diversity for root-level C0 over SS; the table shows these are the Podcast comprehensiveness and News diversity values fig. 2.

Paper vs code

The paper links its prompts at commit 6d21ef2 app. A. The repository has since moved to a packages/ layout. We cloned microsoft/graphrag at HEAD c600ff8 (MIT, Copyright Microsoft Corporation). Code citations below refer to HEAD, not the paper's commit.

• Chunk size. Paper uses 600-token chunks with 100 overlap in experiments §4.1.1. Code defaults to 1200 tokens with 100 overlap defaults.py. Set 600 explicitly to match the paper.
• Gleaning check. Paper forces the continue-or-stop answer with a logit bias of 100 app. A.2. The extractor asks a plain question and treats a reply of exactly Y as "continue" graph_extractor.py. No logit bias appears in that file.
• Gleaning count. Paper says "up to a specified maximum" and gives no number app. A.2. Code defaults to one gleaning round defaults.py.
• Gleaning wording. The continuation message matches the paper's text ("MANY entities were missed in the last extraction") graph_extractor.py.
• Leiden settings. Paper gives none of: max cluster size, seed, or whether to use the largest connected component §4.1.3. Code defaults: max cluster size 10, largest connected component on, seed 0xDEADBEEF defaults.py.
• Edge weights. Paper says duplicate relationships become edge weights §3.1.3. The extractor parses the LLM's relationship strength as the weight graph_extractor.py. Where duplicates are summed is not checked here.
• Query context budget. Paper fixes 8k tokens for all query-time calls §4.1.3 app. C. Code's global-search default is 12,000 tokens defaults.py.
• Shuffle seed. Paper shuffles without a seed §3.1.6. Code seeds the shuffle with 86 by default community_context.py community_context.py.
• Sorting inside batches. Paper does not mention sorting inside windows. Code sorts each window by community occurrence weight when entities are supplied community_context.py. This changes which reports the reduce step sees first within a window.
• Zero-score fallback. Paper drops score-0 answers §3.1.6. Code drops them too, but if none remain it returns a canned "I do not know" unless allow_general_knowledge is set (default off) search.py search.py.
• Leaf prioritisation. Paper orders a leaf community's elements by combined source and target node degree §3.1.5. Code computes combined edge degree compute_edge_combined_degree.py, but the text-unit trimming sorts by entity degree sort_context.py. Check which path your build uses.
• Sub-community substitution. Matches the paper: when a community does not fit, its largest sub-community is swapped for its report context_builder.py §3.1.5.
• Score naming. Paper calls the map score "helpfulness" §3.1.6. The code prompt calls it an importance score, integer 0-100 global_search_map_system_prompt.py.
• Entity names. Paper uses exact string matching §3.1.3. Code upper-cases names and cleans the string before matching graph_extractor.py.
• Claims. Paper describes claim extraction as optional §3.1.2. Code disables it by default defaults.py.
• Map concurrency and reduce length. Code's map and reduce lengths default to 1000 and 2000 and concurrency to 32 search.py. Paper gives none of these.
• Community report schema. Matches the paper's JSON layout, 0-10 rating, and "+more" rule community_report.py community_report.py.

Pitfalls

• Win rates are judge-based. The primary metrics come from an LLM comparing answer pairs on criteria the authors wrote §3.3. The claim-based check agrees on only 78% (comprehensiveness) and 69-70% (diversity) of non-tie cases, and ties are 33-39% of pairs §5.2. The authors say the claim threshold choice changes the results §5.2.
• Empowerment does not favour GraphRAG. Win rates over SS are 43-58% on Podcast and 43-53% on News fig. 2. Results are mixed. The authors' LLM analysis of the judgements found that specific examples, quotes and citations were key to informed understanding, and suggest tuning extraction prompts to keep more of these details §5.1.
• Directness favours vector RAG. Vector RAG wins on directness by design, which is the control §3.3 fig. 2.
• Caption mismatch on token savings. The community-table caption says C0 needs "9x-43x" fewer tokens per query tab. 2. The table gives 38x (Podcast: 1,014,611 / 26,657) and 43x (News: 1,707,694 / 39,770). No 9x ratio appears in the table tab. 2.
• Gleaning ratio is looser in text than in the figure. Text says GPT-4 extracted "almost twice" as many entity references at 600 tokens as at 2400 app. A.2. The figure's values are 9,348 vs 5,761 with no gleaning (about 1.6x), falling to 27,240 vs 19,433 after three rounds (about 1.4x) fig. 3.
• Map-reduce cost scales with the summary level. At the 8k budget, the paper does not state call counts. Derived from tab. 2: C0 Podcast is about 4 windows (26,657 / 8,000) and C3 Podcast about 93 (746,100 / 8,000). Each window needs one map call, so C3 costs far more per query than C0.
• Indexing cost is large and only partly reported. Podcast indexing took 281 minutes on gpt-4-turbo §4.1.3. Token cost of indexing is not reported. Use tab. 2 for query-time token counts only.
• Generality is untested. Results come from two corpora of about 1M tokens. The authors say generalisation to other domains is not shown and that fabrication rates have not been measured §6.1.
• Grounding is prompt-only. The report and answer prompts require citing record ids and dropping unsupported claims app. E.2 app. E.4. Lists are capped at five ids plus "+more" app. E.2. The paper describes no automatic check of these citations.
• Figure and table numbers are derived. Figure and table numbers above follow the LaTeX source order (Figure 1 pipeline, Figure 2 win rates, Figure 3 gleaning, Figure 4 communities; Table 1 queries, Table 2 community counts, Tables 3-4 claims; Appendix D and G tables). Check them against the compiled PDF before publishing.
• No equations. The paper has no numbered equations. Algorithm 1 is the only formal procedure alg. 1.
• Model naming. The introduction says GPT-4 and the appendix says gpt-4-turbo §1 §4.1.3. Pin the exact model when you replicate.

Lineage

• Builds on: Leiden community detection §2.2 §3.1.4; LLM-based knowledge graph extraction §2.2; hierarchical summary indexes such as RAPTOR, named but not on the wiki §2.1; Claimify for claim extraction §4.2. Papers without a card are named only.
• Compared against: vector RAG ("semantic search", SS) and map-reduce over source text (TS) §4.1.2. Benchmarks HotPotQA, MultiHop-RAG and MT-Bench are named as explicit-fact benchmarks §2.3.
• Superseded by: not stated in the paper.