wikillm

Paper · 2026-11-07 · cs.AI, cs.CL, cs.IR

Lightfoot: Graph Retrieval-Augmented Generation with Exact, Model-Free Document Deletion

Lightfoot is a GraphRAG index built only from per-passage LLM extractions, so deleting a document needs no model call and leaves a graph equal to a fresh index. Retrieval runs Personalized PageRank from question-similar facts and passages, fused with a cross-encoder, without LLM calls. Author-reported: competitive with HippoRAG 2 on held-out retrieval, tied on answers, cheapest to index.

Graph-based RAGRetrieval-augmented generationPassage rerankingMulti-hop reasoninggraph-ragretrievaldocument-deletionpersonalized-pagerankmulti-hop-qa

arXivPDFCodeMarkdown21 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Some questions cannot be answered from one passage. The answer is spread over several documents, and the question names only the first of them. GraphRAG systems handle this by first reading documents into a web of named things and the facts that link them, then following that web to find the passages a plain search would miss.

Lightfoot is a design for that web with three practical goals: answering questions cheaply, adding documents, and removing documents. Its main idea is that every part of the web is recomputed from small notes taken from each passage. A language model reads each passage once and lists the names and short facts in it, with the sentence that supports each fact. Everything else, such as which names refer to the same thing and which names look alike, follows from those notes by fixed rules, with no further model calls. Deleting a document therefore means dropping that document's notes and recomputing only the parts they touched. The result is the same web you would get by indexing the remaining documents from scratch. Many other systems merge descriptions across documents or keep only the closest neighbours of each name, which makes that match impossible. The author shows this on a corpus of near-duplicate names.

When a question arrives, no language model is involved in finding evidence. The system starts a random walk from the names in the facts most like the question and from the passages most like the question, then reorders the walk's best passages with a small local reranker. The author reports that this is competitive with a strong open-source baseline on retrieval, ties it on answer accuracy, and builds the cheapest index of the systems compared. These are the author's own measurements on three multi-hop question sets.

TL;DR

  • Lightfoot makes every entity, link, fact and synonym record a deterministic function of per-passage extractions. Deleting documents needs no model call, and the afterwards-graph is exactly the one a fresh build over the survivors produces §3.2 §5.
  • Retrieval makes no language-model call. It runs Personalized PageRank (damping 0.9) from question-similar facts and passages; a local reranking model (bge-reranker-base) re-scores the top 30 passages, and reciprocal rank fusion merges that order with the walk's order §3.3.
  • Author-reported, held-out questions: retrieval beats HippoRAG 2 on MuSiQue recall@5 (0.660 against 0.613) and recall@10 (0.770 against 0.714), and on 2WikiMultiHopQA recall@10 (0.964 against 0.930). HotpotQA is tied. Answer accuracy ties on all three datasets tab. 1 tab. 2 tab. 3.
  • The released code does not contain the question-entity seeding that the paper's headline rows use. Its configuration matches the paper's "v1 seeding" description, which trails HippoRAG 2 significantly on 2WikiMultiHopQA recall@2 and recall@5 §5. See Paper vs code.

Problem

Plain similarity search fails when the evidence for a question sits in passages the question never names §1. GraphRAG answers this with an extracted entity graph, but deployed systems pay three costs that comparisons usually leave out: model calls at query time, the price of building the graph (extra extraction passes, LLM-written entity summaries and community reports), and the price and correctness of removing a document §1.

Deletion is the sharp one. Retention rules, contract expiry and erasure requests all demand that a document and everything derived from it disappear. When descriptions are merged across documents, a merged description has to be rewritten, and a community report cannot be traced back to its sources §1. In the author's review of 17 open-source GraphRAG frameworks, five offer document-level deletion, and none checks that the result matches a graph that never held the document §1 app. A tab. 7.

Assumption: each passage is processed by exactly one extraction call §3.1.

Method

Lightfoot has three parts: one structured extraction call per passage §3.1, a provenance ledger in which every derived record is a function of the live per-passage extractions §3.2, and a retrieval pipeline built on Personalized PageRank with cross-encoder fusion §3.3.

Extraction §3.1

  • Each passage goes through a single call to Llama 3.3 70B, an open 70-billion-parameter instruction model §3.1 §4. The released runs select this model through an environment variable run_all.sh.
  • The reply is one compact line of JSON. Up to 12 entities appear, each carrying a name, a type picked from a configurable list, and a description capped at 8 words. Up to 12 facts follow, each a four-item array: subject, predicate, object, and a supporting quote capped at 12 words §3.1. The schema is in extract.py.
  • Writing facts as arrays with no indentation trimmed the reply to 269 tokens per passage in a 32-passage test, down from 436. The average number of facts per passage went from 3.8 up to 7.7 in that test §3.1.
  • On MuSiQue, each passage's call used 442 tokens of input and 310 of output on average §3.1.
  • The cache key for an extraction includes a digest of the prompt and the schema, so editing either invalidates the cache §3.1 extract.py.

Provenance model and the erasure invariant §3.2

  • A form is one occurrence of a name in one chunk. It stores the chunk, a normalised version of the name, the type, and the chunk's own description of the entity §3.2.
  • Two same-type forms are linked when their name vectors have cosine similarity of at least τ_m = 0.92 §3.2 config.py.
  • An entity is a group of live forms joined by links. Its identifier is derived from the smallest member. The display name is whichever alias carries the most provenance; alphabetical order settles ties §3.2.
  • Synonym edges link every same-type entity pair scoring at least 0.80 on name similarity (τ_s), with no per-entity limit §3.2 config.py.
  • A fact is keyed by its subject and object forms, and it records every chunk that asserted it §3.2.
  • Proposition 1 §3.2: for any split of a document set D into A and B, deleting B from the store built on D gives the store built on A. The argument has three parts. Each stored record depends only on the surviving forms and the chunks they came from. No record comes from a model call that reads several chunks. Components and synonym edges depend only on the names that are still live §3.2.
  • The paper names three constructions that would break this property and that Lightfoot leaves out: merged model-written descriptions, a model judge for ambiguous name pairs, and capped nearest-neighbour lists §3.2.

Deletion §3.2, "Deletion in practice"

  • All requested deletions run in one transaction. Affected components and synonym edges are rebuilt a single time, and the batched result equals deleting the documents one by one §3.2.
  • Derived data also leaves the caches. Extraction results are dropped unless an identical surviving chunk exists. Passage and name vectors held in the embedder's memory are evicted. Query-trace lines that cite the deleted chunks are removed, and topic summaries that covered them are invalidated §3.2.
  • The optional model judge has a pair-decision cache, and that cache is not purged on delete. The judge is switched off in every reported configuration §3.2.

Retrieval §3.3

Given a question q:

  1. Turn the question into a vector with the bge-base-en-v1.5 encoder §3.3.
  2. Seed a Personalized PageRank walk from three sources §3.3:
    • the entities at either end of the two facts nearest to q. Each one receives the fact's similarity divided by the number of passages that mention it;
    • entities the question names. An entity counts as named when an alias of it appears word-for-word in q. The match is greedy and longest-first inside punctuation-delimited segments, and the alias must contain a capitalised or numeric word that is not the question's first word. Each named entity receives 2 divided by its passage count;
    • the 100 passages closest to q. Their scores are rescaled to the 0–1 range, and together they receive up to 0.05 of the seed mass.
  3. Run the walk with damping 0.9, then keep the 30 passages with the highest scores §3.3.
  4. Rerank those 30 with the cross-encoder bge-reranker-base. Merge the reranker's ordering with the walk's ordering by reciprocal rank fusion, k = 60 §3.3.
  5. The merged ordering is the retrieval output. The answering model is given the ten facts closest to the question, each with its evidence quote, followed by the top ten passages in order. It replies with a brief answer span as JSON §3.3.

Before the answer call, nothing calls a language model. Alias matching on the question takes 0.03 ms. The alias table is rebuilt from storage after each index or delete, which takes 0.6 s on MuSiQue §3.3.

Pseudocode

Index one passage, applied on insertion §3.1 §3.2:

Pseudocode
# Index one passage  [§3.1] [§3.2]
# Input:  chunk_id (str); text (str); entity_types (list of str)
# Output: updated store (forms, fact assertions, components, synonyms)

def index_passage(store, chunk_id, text, entity_types):
    # 1. One extraction call per passage, cached by prompt+schema  [§3.1]
    extraction = extract(text, entity_types)  # <=12 entities, <=12 facts
    # 2. Store one surface form per (chunk, name, type), with the chunk's
    #    description of the entity  [§3.2]
    for entity in extraction.entities:
        store_form(store, chunk_id, normalise(entity.name), entity.type,
                   entity.description)
    # 3. Key facts by subject and object forms; track their chunks  [§3.2]
    for fact in extraction.facts:
        store_fact_assertion(store, chunk_id, fact)
    # 4. Link same-type forms with name cosine >= 0.92; entities are the
    #    connected components of live forms  [§3.2]
    recompute_components(store, touched_forms(store, chunk_id))
    # 5. Synonym edges: same-type entity pairs with name cosine >= 0.80,
    #    no top-k cap  [§3.2]
    recompute_synonym_edges(store, touched_entities(store, chunk_id))

Delete documents, one transaction for all requested documents §3.2:

Pseudocode
# Delete documents  [§3.2, "Deletion in practice"]
# Input:  store; doc_ids (list of str)
# Output: store that matches a rebuild over the surviving documents

def delete_documents(store, doc_ids):
    # 1. Remove forms and fact assertions that came from these chunks;
    #    no model call  [§3.2]
    chunk_ids = chunks_of(store, doc_ids)
    remove_chunk_rows(store, chunk_ids)
    # 2. Recompute only the components and synonym edges touched by the
    #    removal, once each  [§3.2]
    recompute_touched_components(store, chunk_ids)
    recompute_touched_synonyms(store, chunk_ids)
    # 3. Evict cached extractions, vectors and topic summaries that cite
    #    the deleted chunks  [§3.2]
    evict_caches(doc_ids)
    # 4. Commit everything as one transaction  [§3.2]
    commit(store)

Retrieve for one question, with no language model before the answer call §3.3:

Pseudocode
# Retrieve for one question  [§3.3]
# Input:  question (str); graph (entities, passages, edges); store
# Output: ranked passage ids (list of str), best first

def retrieve(question):
    # 1. Embed the question with bge-base-en-v1.5  [§3.3]
    query_vector = embed(question)                    # (768,)
    # 2a. Subject and object entities of the two most similar facts;
    #     mass = fact similarity / passages mentioning the entity  [§3.3]
    reset = {}
    for fact in top_facts(query_vector, k=2):
        for entity in subject_and_object(fact):
            # rule for an entity shared by two facts: not stated
            reset[entity] = fact.similarity / passage_count(entity)
    # 2b. Entities named in the question: mass 2 / passage count.
    #     Paper only; absent from the released code  [§3.3] [§6]
    for entity in named_entities(question):
        reset[entity] = reset.get(entity, 0.0) + 2 / passage_count(entity)
    # 2c. The 100 nearest passages share up to 0.05 of the seed mass  [§3.3]
    similar = top_passages(query_vector, k=100)
    for passage, score in min_max_normalise(similar):
        reset[passage] = reset.get(passage, 0.0) + 0.05 * score
    # 3. Personalized PageRank, damping 0.9; keep the 30 best  [§3.3]
    scores = personalized_pagerank(reset, damping=0.9)
    candidates = top_passages(scores, k=30)
    # 4. Score candidates with the cross-encoder (bge-reranker-base)  [§3.3]
    rerank_rank = rank_by(cross_encoder(question, candidates))
    walk_rank = rank_by(scores, candidates)
    # 5. Reciprocal rank fusion of the two rankings, k = 60  [§3.3]
    return rrf_fuse([rerank_rank, walk_rank], k=60)

Answer context, after retrieval §3.3: the ten facts closest to the question with their evidence quotes, then the top ten passages in ranked order. One shared prompt asks for the briefest possible span and always a guess §3.3 §4.

Hyperparameters

NameValueWhereNotes
extraction modelLlama 3.3 70B instruct§3.1 §4released runs use meta-llama/llama-3.3-70b-instruct run_all.sh
answer modelgpt-5.6-luna, reasoning disabled§4released runs use openai/gpt-5.6-luna run_all.sh; config defaults differ, see Paper vs code
embedderbge-base-en-v1.5, 768 dimensions§3.3 §4embed_server.py
cross-encoderbge-reranker-base§3.3 §4input capped at 512 tokens in code; not stated in paper retrieve.py
entities per passageat most 12§3.1config.py
facts per passageat most 12§3.1the system prompt asks for 10 to 12 extract.py
entity descriptionat most 8 words§3.1schema says 8; system prompt says 2 to 5 words; see Paper vs code
evidence quoteat most 12 words§3.1config.py
entity link threshold τ_m0.92 cosine§3.2config.py
model judge bandoff in reported configs§3.2 §6judge_threshold equals merge_threshold, which switches the band off config.py
synonym threshold τ_s0.80 cosine, no top-k cap§3.2config.py
PPR damping0.9§3.3 §6the walk continues with probability 0.9 graph.py
PPR convergencenot stated—released code: tolerance 1e-6, at most 100 iterations config.py
fact seedstop 2 facts§3.3 §6seeding from 5 facts instead gave about 1 point less §6
question-entity seed mass2 divided by passage count§3.3 §6paper only; not in released code
passage seedstop 100 passages, mass up to 0.05§3.3config.py
candidates reranked30§3.3config.py
RRF constant k60§3.3config.py
facts sent to answer10§3.3config.py
passages sent to answer10, in ranked order§3.3config.py
context token budgetnot stated—released code: 9000 tokens, facts capped at a quarter of it config.py retrieve.py
answer max tokensnot stated—released code: 1500 config.py
sampling temperaturenot stated—no temperature setting found in released lightfoot/*.py
name-embedding batchfixed shape: 64 rows, padded to 32, 64, 128 or 512 tokens§3.2embed.py embed.py
embedder sweep, bge-largemerge 0.912, synonym 0.809§5chosen to pass the same share of names as under bge-base
embedder sweep, Qwen3-Embedding-8Bmerge 0.976, synonym 0.934§5truncated to 1,024 dimensions tab. 4
embedder sweep, text-embedding-3-largemerge 0.881, synonym 0.707§5truncated to 1,024 dimensions tab. 4
tuning splitMuSiQue questions 1 to 500; 2WikiMultiHopQA questions 1 to 500 of the 1,000-question sample§4HotpotQA was not used for tuning §4
held-out splitMuSiQue and 2Wiki questions 501 to 1,000; all 300 HotpotQA questions§4answers only on MuSiQue 501 to 800 and the 2Wiki 1,000-question sample
datasetsMuSiQue 1,000 questions over 11,402 passages; 2Wiki 300 over 2,131 and 1,000 over 6,392; HotpotQA 300 over 2,957§4follows the HippoRAG construction §4
statisticspaired bootstrap, 95% intervals, Holm correction, significance at 0.05§4separate corrections for retrieval and answer comparisons
index price, Llama$0.10 input, $0.32 output per million tokens§4fixed list price used for every system
index price, gpt-5.6-luna$0.20 input, $1.20 output per million tokens§4fixed list price used for every system
deletion auditseeded random 10% of documents (1,140 of 11,402 MuSiQue passages)§4 tab. 6every question re-run afterwards §4
hardwareone Apple M3 Pro laptop, 18 GB§4local embeddings, reranking and walks; model calls via OpenRouter §4

Results

All results are author-reported. The retrieval and answer tables use questions the settings were not tuned on, as stated in each caption.

Retrieval tab. 1 (questions 501 to 1,000 of MuSiQue and of the 2Wiki sample, all 300 HotpotQA; best value per column in bold in the paper):

SystemMuSiQue R@2MuSiQue R@5MuSiQue R@102Wiki R@22Wiki R@52Wiki R@10HotpotQA R@2HotpotQA R@5HotpotQA R@10
Lightfoot.503.660.770.723.907.964.802.955.988
Lightfoot (v1 seeding).469.640.758.691.856.935.795.943.988
HippoRAG 2.473.613.714.752.903.930.777.930.975
HippoRAG 2 +RR.460.611.713.678.794.899.848.957.988
LightRAG.313.477.584------.703.902.950
LightRAG +RR.415.547.636------.848.937.970
fast-graphrag.310.426.475------.513.732.800
Dense.398.545.625.644.714.758.768.908.955
Dense +RR.444.567.660.666.730.772.838.942.968

"+RR" marks a baseline whose top 30 passages were re-ordered by Lightfoot's cross-encoder and merged with the baseline's own order. "v1 seeding" is Lightfoot without question-entity seeding. The two baselines LightRAG and fast-graphrag appear on 2Wiki only in the 300-question subset tab. 1 §4.

Answers tab. 2 (one shared answer stage for every system; questions 501 to 800 of MuSiQue and of the 2Wiki sample, all 300 HotpotQA; first of two runs):

SystemMuSiQue EMMuSiQue F12Wiki EM2Wiki F1HotpotQA EMHotpotQA F1
Lightfoot.383.516.553.652.650.804
Lightfoot (v1 seeding).363.505.557.654.643.795
HippoRAG 2.363.504.550.643.630.776
HippoRAG 2 +RR.333.473.513.618.653.800
LightRAG.280.408----.633.779
Dense +RR.333.460.487.555.640.786

None of the Lightfoot-versus-HippoRAG 2 answer differences reaches significance, with or without the reranker tab. 2 tab. 3.

Paired differences tab. 3 (Lightfoot minus the baseline, in percentage points, with 95% bootstrap intervals; bold where significant after Holm correction):

BaselineMetricMuSiQue2WikiHotpotQA
HippoRAG 2R@2+3.0 [+0.4, +5.4]−3.0 [−5.3, −0.8]+2.5 [−1.2, +6.2]
HippoRAG 2R@5+4.8 [+2.4, +7.1]+0.4 [−1.6, +2.4]+2.5 [+0.5, +4.5]
HippoRAG 2R@10+5.5 [+3.5, +7.5]+3.4 [+1.9, +4.9]+1.3 [+0.0, +2.7]
HippoRAG 2EM+2.0 [−2.0, +6.0]+0.3 [−3.3, +4.0]+2.0 [−1.3, +5.7]
HippoRAG 2F1+1.2 [−2.4, +4.9]+0.9 [−2.6, +4.3]+2.8 [−0.1, +5.6]
HippoRAG 2 +RRR@2+4.3 [+2.3, +6.3]+4.5 [+2.9, +6.1]−4.7 [−7.8, −1.5]
HippoRAG 2 +RRR@5+5.0 [+3.0, +7.1]+11.3 [+9.5, +13.2]−0.2 [−1.8, +1.5]
HippoRAG 2 +RRR@10+5.7 [+3.7, +7.6]+6.5 [+5.0, +8.1]+0.0 [−0.8, +1.0]
HippoRAG 2 +RREM+5.0 [+1.3, +9.0]+4.0 [+0.0, +8.0]−0.3 [−3.0, +2.3]
HippoRAG 2 +RRF1+4.2 [+0.6, +7.9]+3.4 [−0.2, +7.1]+0.4 [−1.7, +2.5]

Embedder sweep tab. 4 (held-out questions; extractions reused, thresholds re-mapped, retrieval settings not re-tuned; Qwen3 and text-embedding-3-large truncated to 1,024 dimensions):

EmbedderSystemMuSiQue R@2MuSiQue R@5MuSiQue R@102Wiki R@22Wiki R@52Wiki R@10HotpotQA R@2HotpotQA R@5HotpotQA R@10
bge-baseLightfoot.503.660.770.723.907.964.802.955.988
bge-baseHippoRAG 2.473.613.714.752.903.930.777.930.975
bge-baseDense +RR.444.567.660.666.730.772.838.942.968
bge-largeLightfoot.501.664.768.724.903.965.815.952.987
bge-largeHippoRAG 2.489.642.725.750.920.945.802.942.980
bge-largeDense +RR.431.582.672.671.741.777.845.942.970
Qwen3-Emb.-8BLightfoot.507.683.795.724.905.967.808.960.992
Qwen3-Emb.-8BHippoRAG 2.499.669.755.732.910.936.777.933.988
Qwen3-Emb.-8BDense +RR.469.637.737.674.753.799.863.947.967
te-3-largeLightfoot.514.694.805.718.893.969.825.962.990
te-3-largeHippoRAG 2.502.663.745.754.918.939.775.940.983
te-3-largeDense +RR.443.628.721.677.763.815.838.942.968

Across 54 comparisons in this sweep, with Holm correction, the author reports Lightfoot ahead of HippoRAG 2 on MuSiQue recall@10 under all four embedders, by 4.0 to 6.1 points §5.

Index cost and calls tab. 5 (normalised list prices; LightRAG's MuSiQue cost is an estimate):

ApproachMuSiQue2WikiHotpotQAIndex LLM calls per passageQuery-time LLM calls
Lightfoot$1.64$0.29$0.461.00
HippoRAG 2$2.23$0.42$0.622.01
fast-graphrag$4.89$0.93$1.371.11
LightRAG≈$9.5$1.36$2.232.51

Figure 1 shows each dataset's normalised index cost alongside its recall@5 values. Its caption says the MuSiQue points use all questions and Lightfoot's v1 seeding. It is described here and not reproduced fig. 1.

Deletion tab. 6 (a seeded random 10% of MuSiQue passages, 1,140 of 11,402, removed; recall@5 scored against every gold passage):

ApproachTimeLLM callsKept questions, beforeKept questions, afterDeleted-evidence questions, afterMatches rebuild
Lightfoot14 s0.648.661.369yes
HippoRAG 29 s0.625.639.333yes (MuSiQue only)
LightRAG13,294 s2,187.491.507.292not tested
fast-graphrag----------no deletion API

The post-deletion fingerprint test covers 68,455 entities, 71,195 aliases, 106,472 mentions, 92,608 facts, 94,032 fact sources and 63,514 synonym edges. Lightfoot's post-deletion store matched a fresh build of the 10,262 surviving documents in every table §5 tab. 6. HippoRAG 2 matched except for 16 synonym weights that differ by about 10⁻⁷ §5.

On a constructed corpus of 160 passages with near-duplicate customer account names, deleting every fourth document left HippoRAG 2 with 7,558 synonym edges against 8,123 in a rebuild. Lightfoot matched its rebuild §5. The authors call this a synthetic case and did not measure how frequently the cap binds in real catalogues §7.

Appendix A reviews 17 open-source GraphRAG frameworks for index calls, query calls, incremental insertion and document deletion. Only five offer document deletion app. A tab. 7.

Paper vs code

Checked against the official repository at code_commit (MIT, Uday Allu). The repository agrees with the paper on the extraction limits, the thresholds 0.92 and 0.80, PPR damping 0.9, the top-30 candidate pool, the top-100 passage seeds with mass 0.05, the reciprocal rank fusion with k = 60, the two-fact seed, and entity–entity edges weighted by fact count. The discrepancies are below.

  1. Question-entity seeding is not in the release. The paper defines it in §3.3, uses it for every headline Lightfoot row tab. 1 tab. 2, and credits it with most of the 2Wiki gain §5 §6 §8. Retrieval in the released code seeds only from the top two facts and the top 100 passages retrieve.py. Seeding is selected by seed_mode: hipporag config.py. No alias matching is performed on the question. The alias table is written at index time ledger.py, but no query-time code reads it. I searched lightfoot/ and harness/ for question-entity matching and for the alias lookup and found none. The released configuration matches the paper's description of "v1 seeding" §4.
  2. The answer stage differs from the paper's shared stage. The paper says every system uses one answer stage, with the top ten passages labelled [P1] to [P10] and a single prompt §4. The Lightfoot adapter calls the system's own query with answering enabled lightfoot_adapter.py. That path builds the answer context from facts and passages retrieve.py and sends Lightfoot's own system prompt answer.py. I found no [P1] labelling in the repository, so Table 2 cannot be reproduced from the released code.
  3. Query-time model calls are configurable. The paper states that retrieval never invokes a language model before answering §3.3. The config default is enable_decomposition: True config.py. When the confidence gate fails, the retriever sends the question to the language model for decomposition and retrieves again retrieve.py. Both released Lightfoot runs set this to false lightfoot_v5a_musique_1000.yaml lightfoot_v5b_musique_1000.yaml. The paper does not describe the gate.
  4. Global questions call a model. When the question matches the global-question pattern, query adds topic summaries api.py. Missing or stale summaries are generated with a language-model call topics.py. The paper's retrieval path makes no model call, and it does not evaluate global questions §7.
  5. Description length differs between schema and prompt. The schema allows at most 8 words per entity description extract.py. The system prompt asks for 2 to 5 words extract.py. The paper says 8 §3.1.
  6. Stricter retry prompt, not described in the paper. If an extraction is cut off, a retry prompt caps output at 8 entities, 8 facts, 5-word descriptions and 8-word evidence spans extract.py.
  7. Entity identifier. The paper derives the identifier from the lexicographically smallest member name §3.2. The code takes the smallest form key, which is the type followed by the separator and the name resolve.py. This differs only when members of one component have different types.
  8. Default models differ from the paper. Without environment variables, the config defaults are claude-haiku-5-5 for extraction and claude-sonnet-5-5 for answers config.py. The paper's models come only from the environment settings in run_all.sh.
  9. Reported metrics cover a different question set. The result file for the v5b run reports recall@5 of 0.646 over all 1,000 MuSiQue questions result.json. Table 1 reports questions 501 to 1,000. I did not recompute the held-out subset from the per-question files, so the match with Table 1 is unverified.

The graph layout and the fixed-shape name-embedding batching show no discrepancy embed.py.

Pitfalls

  • Headline gains need the missing seeding step. The paper says that without question-entity seeding, Lightfoot trails HippoRAG 2 significantly on 2Wiki recall@2 and recall@5 §5. Someone who follows only the released code gets the "v1" behaviour. Decide which one to build, and label the result.
  • Alias matching is under-specified. The paper gives greedy longest-first matching inside punctuation-delimited segments and a capitalised-or-numeric rule §3.3, but it does not say how case is normalised, which punctuation counts, or how overlapping aliases are resolved. None of this is in the released code.
  • Shared entities. When both top facts mention the same entity, the paper gives no rule for combining their seed mass. The released code averages the two contributions retrieve.py.
  • Masses are renormalised. The released PPR divides the reset vector by its sum before the walk graph.py. Only the ratios between seed masses matter, so the absolute values in §3.3 are not what the walk sees.
  • Damping convention. Here 0.9 is the probability of continuing the walk along an edge, so the restart probability is 0.1 graph.py. Check other systems' damping values against this convention before comparing numbers across papers.
  • Asymmetric query encoding. Facts, entities and the question-level dense search use the plain query embedding retrieve.py. Passages use the bge query instruction prefix retrieve.py embed.py. The paper says only "embed q" §3.3.
  • Thresholds are tied to the embedder. The 0.92 and 0.80 thresholds were chosen for bge-base. Under bge-base, 4.5% of extracted surface names have a same-type neighbour above 0.92 and 27.2% above 0.80. For other embedders the paper matches those shares without labels §5. Changing the embedder without remapping changes which names link.
  • Determinism is part of the method. Name embeddings must come from fixed-shape batches, and the cosine is computed with exactly rounded sums. Otherwise links near the threshold can change with batch composition §3.2 resolve.py. The paper reports differences of about 10⁻⁷ in HippoRAG 2 weights from batch composition §5.
  • The judge's cache survives deletion. If the optional judge is enabled, its pair-decision cache is left in place on delete §3.2. The released config also needs judge_threshold set to 0.80 to open the judge band config.py.
  • Context is truncated by tokens. The released context is capped at 9,000 tokens, facts at a quarter of that, and the last passage can be cut mid-text retrieve.py retrieve.py. Ten facts and ten passages can therefore become fewer when passages are long.
  • Candidate padding. If fewer than 30 candidates remain after the walk, the released code pads the ranking from the remaining walk scores retrieve.py. The paper does not mention this.
  • Reranker input is capped. The cross-encoder is loaded with a 512-token maximum retrieve.py. The paper does not say this.
  • Temperature is unset in the code. No temperature is set in the released lightfoot/*.py, so the provider default applies. The paper does not state a temperature.
  • Answer differences are mostly noise. The two answer runs disagreed on 9 to 22 of the 300 answers for each system and dataset §5. A gap of a few points on 300 questions cannot be told apart from re-run noise.
  • Costs use list prices. Index costs are computed from fixed per-million-token prices, not invoices. The MuSiQue figure for LightRAG is an estimate built from its call count tab. 5.
  • Small comparison sets. On 2Wiki, LightRAG and fast-graphrag appear only in the 300-question subset, so no held-out 2Wiki comparison exists for them §4.
  • Retrieval latency is laptop-scale. The median retrieval-only latency on MuSiQue was 0.53 s for Lightfoot, measured on one laptop with local embeddings and reranking §5 §4.
  • Exactness is tested on one corpus. The full-scale fingerprint test is on MuSiQue after one deletion §5. LightRAG's deletion was not fingerprinted §7.

Lineage

  • Builds on: HippoRAG 2 (arXiv:2502.14802). Lightfoot keeps its PageRank over extracted facts but drops the model-based fact filter and the capped synonym lists §2 §3.2. The paper also names HippoRAG (NeurIPS 2024) §2.
  • Related: Microsoft GraphRAG, which summarises communities with model-written reports, the design Lightfoot argues against for deletion §2 §1.
  • Compared against: LightRAG (arXiv:2410.05779), fast-graphrag and dense retrieval with bge-base §4.
  • Superseded by: not known.

Source and license

Adapted from “Lightfoot: Graph Retrieval-Augmented Generation with Exact, Model-Free Document Deletion” by Uday Allu (arXiv:2611.11010), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/udayallu-ml/Lightfoot-GraphRAG at commit fbba969e73ea (MIT). Any quoted code is shown under that license.