---
kind: concept
slug: retrieval-augmented-generation
title: Retrieval-augmented generation
summary: >-
  Gives a language model passages fetched at answer time, so its answer rests on material outside its training. Variants differ in how they find passages, from top-k embeddings to keyword counts or a model that decides when to retrieve. Often beats fine-tuning on the same documents, author-reported. Trade-off: answers depend on retrieval, thresholds need calibrating, and measurements are thin.
tags: [retrieval, llm, grounding, hallucination, question-answering, agents]
papers: ["2205.13760", "2206.06520", "2209.14375", "2211.08411", "2202.03629", "2112.04426", "2212.10509", "2302.00083", "2302.11713", "2303.09136", "2303.14070", "2305.13661", "2308.08155", "2308.10144", "2309.10892", "2310.11511", "2311.00176", "2402.06196", "2403.11169", "2404.13501", "2407.13193", "2409.06857", "2503.10265", "2505.10468", "2507.07426", "2302.12813", "2305.15334", "2404.16130", "2404.18416", "2210.02627", "2307.02694", "2401.01313", "2312.05934"]
related: [knowledge-injection, hallucination-mitigation, hallucination-detection, in-context-learning, llm-evaluation, tool-use, llm-agent, agent-memory, model-editing, multi-agent-systems, rlhf, domain-adaptation, data-poisoning, dense-retrieval, monte-carlo-tree-search, self-consistency, multimodal-language-model, chain-of-thought-prompting, multi-hop-reasoning, language-model-pretraining, nearest-neighbor-search, passage-reranking, automated-fact-checking, decoding-strategies, controllable-text-generation, self-refinement, graph-based-rag, llm-as-a-judge, self-training, uncertainty-estimation]
status: draft
updated_at: 2026-11-01
---

## Overview

Retrieval-augmented generation (RAG) answers a question in two steps: it first
finds material relevant to the question in a store, then gives that material to
a language model as context. The answer can then draw on material the model was
not trained on, such as a course's own files [arxiv:2309.10892 §3.2]. The
survey treats this as one way of supplying evidence to a model, next to other
input augmentation [arxiv:2202.03629 §5.1.3]. A 2025 review of AI agents lists
RAG as one of ten mitigation mechanisms for agent systems, next to tool use
and memory architectures [arxiv:2505.10468 §6.1]. A 2024 survey of
hallucination mitigation sorts retrieval by when it happens: before, during or
after generation, or end to end [arxiv:2401.01313 §2.1.1].

A survey of retrieval for NLP groups the ways retrieved text reaches the model
into four routes: text pasted into the prompt, a blend of output distributions,
passage features fed into the model's internals, and per-document add-on
weights [arxiv:2407.13193 §4]. Its comparison of the four is qualitative
[arxiv:2407.13193 tab. 2].

One card asks a different question: should new facts reach a model through
retrieval, or through more training on the same documents? Its tests use 7B
models, Wikipedia text, four MMLU subjects and 910 current-events questions.
Retrieval beat training on the original text in nearly every setting. Training
on several paraphrases of each chunk improved on the base model for all three
models, though it stayed below retrieval [arxiv:2312.05934 tab. 1] [tab. 2].
Training on the original text gave small or mixed gains, and for one model it
lowered the current-events score [tab. 2]. The results are author-reported, and
the number of passages retrieved in the main tables is not stated [tab. 1] [§5].

The cards describe several ways to find the material. A course chatbot embeds
every chunk and ranks chunks by cosine similarity [arxiv:2309.10892 §3.2.2,
§3.2.3]. A medical chat model has the language model write search terms and
ranks sections by how many terms they contain [arxiv:2303.14070 p. 4]. A
chip-design assistant fine-tunes its retriever on synthetic query-passage
pairs [arxiv:2311.00176 §2.4]. A dialogue agent runs web searches and shows the
excerpts to raters [arxiv:2209.14375 §2.4]. A medical Gemini model searches only in rounds where its own sampled answers disagree [arxiv:2404.18416 §2.1]. An agent retrieves its own past
successful attempts as examples [arxiv:2308.10144 §4.2.1]. A protein model
mixes a retrieved sequence alignment into its own predictions [arxiv:2205.13760
§4.2]. A visual question system names the pictured entity before it reads
Wikipedia [arxiv:2302.11713 §4.2]. A drug-repositioning agent places retrieval
inside a tree search over agent actions [arxiv:2507.07426 §2.3]. Newer cards add retrieval inside
pretraining [arxiv:2112.04426 §2.4], one passage prepended every few tokens to
a frozen model [arxiv:2302.00083 §3.2], retrieval steps interleaved with
reasoning [arxiv:2212.10509 §3.1], and a fact-checking pipeline that ranks web
pages by publisher tier [arxiv:2403.11169 §Approach]. A model trained to emit
reflection tokens decides for itself when to retrieve and grades its own
passages and sentences [arxiv:2310.11511 §3.1]. A tool-calling model gets the documentation of the
API it should call [arxiv:2305.15334 §3.2], and a graph index answers
questions about a whole collection from community summaries [arxiv:2404.16130
§3.1.6]. Another card trains the retriever together with the generator and rebuilds
the stored passage vectors during training [arxiv:2210.02627 §3.3]. The shared
design question is how much to trust the retrieved text. The answer is only as good as
what retrieval returns, and errors can pass from one stage to the next
[arxiv:2303.09136 §6.1]. The cut-off in the course chatbot, which decides what
counts as "close enough", is not justified [arxiv:2309.10892 §3.1.1].

The evidence is thin and varies in kind. The course chatbot reports no
retrieval accuracy, answer correctness, latency or user study [arxiv:2309.10892
§4]. The medical chat model reports one BERTScore comparison with ChatGPT,
author-reported, and does not say whether its scored model used retrieval
[arxiv:2303.14070 tab. 1] [p. 7]. The chip-design assistant reports a gain
with and without RAG on an internal chatbot test [arxiv:2311.00176 app.
Engineering Assistant Chatbot Human Evaluation]. The dialogue agent's gains come
from a full pipeline that includes reinforcement learning, so they do not
isolate search [arxiv:2209.14375 §3.2]. The survey lists RAG among LLM
mitigations without measuring it [arxiv:2202.03629 §13.3]. The protein model's
overall gain over EVE is 0.003 Spearman, and the tables give no uncertainty
estimates [arxiv:2205.13760 §6.2]. The visual system's pipeline
numbers come from a split the main table does not name [arxiv:2302.11713 §5.1].
The newer cards are author-reported too: RETRO and In-Context RALM report
perplexity and exact match on their own setups, and IRCoT reports answer F1 on
four multi-hop QA sets [arxiv:2112.04426 app. F.1] [arxiv:2302.00083 tab.
1] [arxiv:2212.10509 tab. 4]. Self-RAG reports 7B and 13B models against
ChatGPT and Ret-ChatGPT, and its ablations appear only in a figure
[arxiv:2310.11511 tab. 2] [fig. 3]. Gorilla's results are author-reported on
its own API benchmark, with no seeds or error bars [arxiv:2305.15334 §4].
GraphRAG's wins are judged by an LLM. Its claim-based check agrees with the
judge on 78% of non-tied comprehensiveness pairs [arxiv:2404.16130 §5.2].
Med-Gemini's search gains are author-reported on the authors' own setup. Its
search engine is not named, and its results per query and retrieval date are
not stated [arxiv:2404.18416 §2.1]. RAG-end2end reports one run per setting,
with no seeds or variance [arxiv:2210.02627 §4.2].

## Variants

### Chunk, embed, top-k by cosine, with a similarity cut-off

This is the pipeline of [VirtualTA](/papers/2309.10892), a Canvas-integrated
course assistant.

- **Sources.** Course files from the learning platform, and transcripts of
  lecture recordings made with Whisper [arxiv:2309.10892 §3.2]. Resources are
  ranked, with primary ones such as lectures above secondary ones such as
  external readings. The ranking mechanism is not specified [arxiv:2309.10892
  §3.2.1].
- **Chunking.** About 800 characters per chunk, keeping words and sentences
  whole. Overlap between chunks is not stated. Structured platform records such
  as announcements and assignments are embedded without chunking
  [arxiv:2309.10892 §3.2.2].
- **Embedding.** `text-embedding-ada-002`, 1,536 dimensions [arxiv:2309.10892
  §3.1.1].
- **Retrieval.** Every chunk is scored by cosine similarity against the query
  embedding. The ten best are kept, then only those above 0.75 [arxiv:2309.10892
  §3.1.1, §3.2.3]. The order is top ten first, then the filter, so at most
  ten chunks reach the prompt and possibly none [arxiv:2309.10892 §3.2.3].
- **Generation.** A fine-tuned classifier first routes the query to a feature.
  The model is then prompted with the retrieved context and an instruction to
  say "I'm not sure" rather than guess. The paper does not measure how well this
  works [arxiv:2309.10892 §3.1, §3.2.3].
- **Disclosure.** Answers show the source file and a percentage. How that
  percentage is computed is not stated [arxiv:2309.10892 §4.1.3, fig. 5]. When
  nothing passes the cut-off, the paper says the answer was generated without
  course material [arxiv:2309.10892 §4.1.3].

### Domain-tuned dense retriever, top-k passages

This is the retriever of [ChipNeMo](/papers/2311.00176), an adapted LLaMA2 for
chip design. Its base is e5-small-unsupervised, fine-tuned with contrastive
learning on 3,000 generated samples [arxiv:2311.00176 §2.4]. A sample is a
passage, a query that a language model writes for it, and hard negatives: top
retrieved passages that the same model judges do not answer the query, topped
up with random passages if too few remain [arxiv:2311.00176 app. Sampling
Procedure]. At answer time the top 8 passages are fetched from a store of about
67K passages, each about 512 characters [arxiv:2311.00176 §3.6]. On top-8 hit
accuracy the tuned retriever beats the untuned e5-small by about 2x and a
Sentence Transformer by about 30%. The per-category values appear only in a
figure [arxiv:2311.00176 §3.6]. Learning rate, epochs and batch size for the
retriever are not stated [arxiv:2311.00176 §2.4].

### Retriever and generator trained together, with a background index refresh

This is RAG-end2end from [Improving the Domain Adaptation of RAG Models](/papers/2210.02627),
aimed at domain text. The question encoder, the passage encoder and a BART
generator are fine-tuned together through the generator's loss. The gradient
reaches the passage encoder through the passage-selection probability
[arxiv:2210.02627 §3.3] [eq. 2]. A changed passage encoder makes the stored
vectors stale, so a re-encoding job runs on other GPUs while training goes on,
and a re-indexing job builds a new index once every vector is saved. The
training loop loads each new index when it is ready [arxiv:2210.02627 §3.3]
[fig. 1]. On a store of millions of passages the paper reports several GPU
hours for re-encoding and several CPU hours for re-indexing [§3.3]. A refresh
starts about every 750 updates on average, measured on a 250,000-passage base
[§3.3]. Gradients can be stale because the index lags the training, and the
paper gives no measure of how much lag is tolerable [§3.3].

The paper adds a side task. A sentence from the domain corpus is rebuilt from
passages retrieved from a store that does not contain it. A control token marks
these inputs [arxiv:2210.02627 §3.4] [eq. 3]. The loss weight between this task
and the QA task is not stated [§3.4].

Against the original design, which keeps the passage encoder frozen [§3.3],
exact match on QA-only training is 8.08 against 2.95 on COVID-19, 8.39 against
7.26 on news and 24.25 against 12.09 on conversations [arxiv:2210.02627 tab. 1].
With the reconstruction task added, the figures are 8.32 against 3.66, 14.08
against 8.62 and 25.95 against 14.21 [tab. 1]. These are author-reported single
runs. On open-domain SQuAD, the in-domain check, exact match is 40.02 against
28.12 [tab. 5]. The learning rate, batch size, optimiser and warmup are not
stated [§4.2].

### Model-written keywords, term-count ranking

This is the autonomous retrieval of [ChatDoctor](/papers/2303.14070), a LLaMA-7B
model fine-tuned on patient-doctor dialogue. The same model drives every step:

- **Keywords.** The model writes comma-separated search terms for the question
  [arxiv:2303.14070 fig. 4] [p. 4].
- **Lookup and ranking.** The terms are matched against a knowledge base: live
  Wikipedia pages, or a disease table built from MedlinePlus [arxiv:2303.14070
  p. 3] [p. 4]. Sections are ranked by the number of term hits. The paper calls
  this a term-matching system, not embedding search [arxiv:2303.14070 p. 4-5].
  Section size is not stated in the paper.
- **Reading.** The model reads the top N sections one at a time and keeps what
  helps the question [arxiv:2303.14070 fig. 5] [p. 4]. The paper uses N = 5
  [arxiv:2303.14070 p. 4].
- **Answer.** The model answers from all the kept notes in one prompt
  [arxiv:2303.14070 fig. 6] [p. 5].

The paper's qualitative examples show this retrieval answering recent terms
such as Mpox, which the baseline did not recognise. The examples were selected
by the authors [arxiv:2303.14070 fig. 7] [p. 5-7].

### Web search with evidence excerpts and reranking

This is the pipeline of [Sparrow](/papers/2209.14375), a dialogue agent built on
Chinchilla 70B. The model writes a search query, Google Search returns results,
and the page is cut to an excerpt of up to 500 characters [arxiv:2209.14375
§2.4] [app. C]. The excerpt joins the dialogue as a search-result turn and is
shown to raters as quoted evidence [arxiv:2209.14375 §2.4]. Per turn the agent
samples eight candidates, four without evidence and four with evidence from two
queries of two results each, and keeps the highest-scoring one [arxiv:2209.14375
§2.6].

On evidence-backed factual replies, the full pipeline scores 0.78 for
"supported and plausible", against 0.61 for the base model that always searches
[arxiv:2209.14375 §3.2]. The same model choosing when to search scores 0.70
[arxiv:2209.14375 §3.2].

### Web search only when sampled answers disagree

This is the text pipeline of [Med-Gemini](/papers/2404.18416), a set of Gemini
models tuned for medicine. The model samples several reasoning paths and
measures the Shannon entropy of the answer choices across them. If the entropy
is at or below a threshold, it returns the majority answer. Otherwise it writes
three queries aimed at the disagreement, and the results join the prompt for
the next round [arxiv:2404.18416 §2.1]. The threshold and the log base are not
stated. The search runs on the three text benchmarks only [arxiv:2404.18416
§2.1]. The MedQA run uses four rounds, and the NEJM CPC and GeneTuring runs use
one [arxiv:2404.18416 §3.1].

The same search step produces training data. Each training question gets one
chain without search results and one with them, and chains with a wrong final
answer are dropped before fine-tuning [arxiv:2404.18416 §2.1] [fig. 2].

On MedQA, accuracy rises from 87.2% to 91.1% over the search rounds, and the
paper credits self-training with a further 3.2 points [arxiv:2404.18416 §4.1]
[fig. 4]. Search adds 4.0 points to top-10 accuracy on NEJM CPC. In the
per-specialty top-10 table, search helps internal medicine and pediatrics but
not neurology or psychiatry [arxiv:2404.18416 §4.1] [tab. C4]. These are
author-reported on the authors' own setup.

### Evidence chains, a utility check and a feedback loop around a black-box LLM

This is the wrapper of [LLM-Augmenter](/papers/2302.12813), which keeps a
ChatGPT-style model fixed. A Knowledge Consolidator searches for evidence, links
entities to descriptions and keeps a short list of evidence chains
[arxiv:2302.12813 §2.3.1]. The prompt carries the query, history, evidence and
any feedback [§2.3.2]. A Utility module scores each candidate and writes
feedback, which goes into the next prompt if the score is too low [§2.4]. In the
main experiments the policy is a fixed rule that always consolidates first, and
the loop repeats until a candidate passes [§3.2] [§1]. A learned policy appears
only in a learning curve [§3.3] [fig. 3].

The dialog results are author-reported KF1 on the paper's own splits. On news
chat with BM25 and feedback, KF1 is 36.41 against 26.71 for ChatGPT alone
[arxiv:2302.12813 tab. 1]. On customer service it is 37.41 against 31.33
[arxiv:2302.12813 tab. 2]. Feedback adds about 3 points over no feedback on that
split [arxiv:2302.12813 tab. 4]. On OTT-QA the F1 goes from 0.59 for ChatGPT
to 2.38 with DPR evidence, 8.08 with the CORE consolidator, and 11.80 with
feedback added [arxiv:2302.12813 tab. 5]. Rows labelled gold use human-annotated
knowledge and are an oracle [§3.2].

### Past successes as in-context examples

This is the experience recall of [ExpeL](/papers/2308.10144), an agent that
keeps its weights fixed. Task descriptions are embedded with all-mpnet-base-v2,
and the top-k successful trajectories closest to the new task go into the prompt
[arxiv:2308.10144 §4.2.1]. The retrieved units are whole trajectories, not
passages. A list of natural-language insights, distilled from successes and
failures on training tasks, goes into the same prompt [arxiv:2308.10144 §4.3].

The paper reports the two parts separately [arxiv:2308.10144 §5.2.1]. On
HotpotQA, insights alone reach 36% and retrieved trajectories alone 31%, against
28% for ReAct. On ALFWorld the order flips: insights alone 50%, retrieved
trajectories alone 55%, against 40% for ReAct. The full method reaches 39% and
59% [arxiv:2308.10144 tab. 3]. Each HotpotQA trajectory uses about three times
the tokens of ReAct, 4,310 against 1,320 [arxiv:2308.10144 tab. 6].

### Stored edits routed to a second model

This is the memory of [SERAC](/papers/2206.06520), a model editor that leaves
the base model's weights unchanged. Each edit is stored as a text descriptor.
A scope classifier scores the input against the stored edits. If the best score
is at least 0.5, a counterfactual model answers from that edit and the input.
Otherwise the base model answers [arxiv:2206.06520 §3.1] [eq. 1]. The
comparison is with retrieve-and-prompt (RP), which keeps the scope classifier
but lets the base model answer [arxiv:2206.06520 §5.1]. SERAC has the higher
edit success on all four benchmarks. RP had the lower drawdown on FC, 0.010
against 0.051 [arxiv:2206.06520 tab. 3]. These numbers are author-reported on
the paper's own benchmarks.

### Live page retrieval inside a multi-agent workflow

This is the knowledge branch of [SurgRAW](/papers/2503.10265), a zero-shot
workflow for questions about robotic-surgery frames. For cognitive-inference
questions, the workflow retrieves MedlinePlus text for the question and gives it
to a chain-of-thought agent [arxiv:2503.10265 §3.3] [§3.4]. In the authors'
ablation on GPT-4o, retrieval raises the cognitive-inference average from 20.11
to 41.32 without chain-of-thought, and from 58.97 to 70.39 with it [arxiv:2503.10265
tab. 4]. The effect differs by task. With chain-of-thought, action prediction
rises from 49.66 to 70.48 and patient detail from 78.16 to 96.21, but surgical
outcome moves only from 41.51 to 42.87 [arxiv:2503.10265 tab. 4].

### Retrieved alignment mixed into a generative model's scores

This is the inference-time retrieval of [Tranception](/papers/2205.13760), a
protein sequence transformer. Retrieval runs once per wild-type protein. A
multiple sequence alignment is built and turned into a per-position amino-acid
profile [arxiv:2205.13760 §4.1] [app. B.5]. At scoring time the model's
next-residue log-probabilities are mixed with that profile, with weight 0.6,
chosen on a validation subset [arxiv:2205.13760 §4.2] [eq. 3] [app. B.5].
Positions the alignment does not cover keep the model's own term. A mutant is
scored as its log-likelihood ratio against the wild type, averaged over the
two reading directions [arxiv:2205.13760 §3.4] [eq. 2]. Building the alignment
is part of the method: the paper uses a Jackhmmer search with a bit-score
threshold and an evolutionary-coupling selection [arxiv:2205.13760 app. C].

Reported results are author-reported Spearman on the ProteinGym substitution
benchmark. With retrieval the score is 0.451. Without it, the same model
gets 0.406 [arxiv:2205.13760 §6.2]. The gap to EVE
over all assays is 0.003. On shallow alignments the paper reports 0.453
against EVE's 0.408. On deep alignments EVE leads, 0.507 against 0.488
[arxiv:2205.13760 §6.2].

### Name the entity from the image, then read a Wikipedia page

This is the With-KB pipeline of [InfoSeek](/papers/2302.11713), a visual
question-answering benchmark. A fine-tuned CLIP model scores a 100K-entry
Wikipedia knowledge base against the image and question, and keeps the top five
entities [arxiv:2302.11713 §4.2]. A Fusion-in-Decoder reader then takes the top
20 passages for each entity, 100 in all [arxiv:2302.11713 §4.2] [app. B].

The paper's comparison is by overall score. On the human split the pipeline
reaches 18.2 against 10.8 for the best end-to-end model. On the Wikidata split
the end-to-end model is ahead, 22.1 against 19.3 [arxiv:2302.11713 tab. 5].
Recognition is the limit: with the gold entity, the same reader reaches 45.6 on
the human split [arxiv:2302.11713 tab. 5]. The pipeline beats the end-to-end
model on the most popular entities and loses on the least popular
[arxiv:2302.11713 §5.2]. The released evaluation code has no CLIP entity recognizer or FiD reader, so
the pipeline cannot be re-run from it.

### Search as one step in a tree of agent actions

This is the retrieval action of [DrugMCTS](/papers/2507.07426), a drug
repositioning framework. A retrieval agent takes the top ten molecules by
Tanimoto similarity and the top ten by ChemBERTa cosine, merges them, and
collects the proteins known to bind any of them [arxiv:2507.07426 §2.3] [eq. 2]
[eq. 3]. The retrieval scores are deterministic, so the pool is the same on
every run [arxiv:2507.07426 §2.3]. Later actions summarise the query, filter the
candidates with the model, and pull pocket text and PubMed abstracts for the
reference proteins [arxiv:2507.07426 §2.3] [alg. 1]. Monte Carlo tree search
decides which sequence of actions to try, and a leaf is scored by how often the
model repeats its choice and how often it answers yes when asked whether the
molecule interacts with the target [arxiv:2507.07426 §2.2] [§2.4].

The paper's headline recall on DrugBank is 0.4466 with the ground-truth count
used as selection, against 0.1552 for GPT-4o-mini with no retrieval
[arxiv:2507.07426 tab. 1]. That protocol uses the ground-truth count, so compare
it with care [arxiv:2507.07426 §3.4]. The released code's default retrieval is
Tanimoto only. The paper's two-metric union is an option [code:
src/drugmcts/config.py#L35].

### Retrieval built into the architecture, trained with the model

This is the design of [RETRO](/papers/2112.04426). The input is cut into
64-token chunks. For each chunk the model looks up neighbours in a database of
up to about 2 trillion tokens, with keys from a frozen BERT encoder [arxiv:2112.04426
§2.2] [§2.3]. A small encoder turns each neighbour into features, and chunked
cross-attention in some decoder layers lets later chunks read them [arxiv:2112.04426
§2.4] [alg. 1]. Training uses two neighbours per chunk [arxiv:2112.04426 §4.1].
Training neighbours are looked up ahead of time. At sampling time the lookup
runs after each chunk [arxiv:2112.04426 §2.3] [§2.4].

On the 7.5B model, C4 bits per byte falls from 0.78 to 0.66 with retrieval on.
The same model with retrieval switched off scores 0.78 [arxiv:2112.04426 app.
F.1]. On Natural Questions with DPR passages it reaches 45.5 exact match, below
FiD's 51.4 [arxiv:2112.04426 tab. 5]. In a 247M ablation, removing retrieval
raises C4 bits per byte from 0.822 to 0.987 [arxiv:2112.04426 app. D]. A
pretrained model can get the lookup later by training only the new
cross-attention and encoder weights, under 10% of the weights for the 7B model
[arxiv:2112.04426 §4.2]. The paper says the store can change without retraining,
but does not test it [arxiv:2112.04426 §3.2]. The MassiveText index is about 93
TB [arxiv:2112.04426 §4.1].

### Frozen model, one retrieved passage prepended every few tokens

This is [In-Context Retrieval-Augmented Language Models](/papers/2302.00083).
The model is unchanged. The last 32 tokens of the text form a BM25 query, the
top passage goes before the prefix, and the query runs again every 4 tokens
[arxiv:2302.00083 §3.1] [§3.2] [§5.2] [§5.3]. Passages are 100-word chunks,
truncated to 256 tokens [arxiv:2302.00083 §4.3]. Two rerankers can choose among
the top 16 BM25 hits. One scores each passage by the LM's likelihood of the
recent tokens. The other is a trained RoBERTa classifier [arxiv:2302.00083 §6.1]
[§6.2]. The released code covers only the first, with an oracle option
[code: README.md#L98-L119].

With BM25, a 345M GPT-2 reaches 21.5 word perplexity on WikiText-103, against
26.3 without retrieval. The 1.5B model without retrieval reaches 20.0
[arxiv:2302.00083 tab. 1]. With the trained reranker the 345M model reaches
19.7 [tab. 1]. On Natural Questions, LLaMA-13B rises from 12.0 to 31.0 exact
match with two DPR passages [arxiv:2302.00083 tab. 4]. These are perplexity
results, and the paper does not measure generated text quality [arxiv:2302.00083
tab. 1].

### Interleaved retrieval and reasoning

This is [IRCoT](/papers/2212.10509). The model writes one reasoning sentence,
and that sentence becomes the next BM25 query. The paragraphs it retrieves join
the prompt before the next sentence. The loop ends when the reasoning contains
"answer is" or at a step limit. The first retrieval uses the question alone, and
the collected paragraphs are capped at 15 [arxiv:2212.10509 §3.1]. No training
is needed. The prompt holds annotated demonstrations, 15 questions per set,
drawn from 20 written [arxiv:2212.10509 §4.1].

With GPT-3 (`code-davinci-002`), answer F1 is 60.7 on HotpotQA. One-step
retrieval gives 53.6 and no retrieval 47.5. On 2WikiMultihopQA it is 68.0
against 54.8. On MuSiQue it is 36.5 against 29.4. On IIRC it is 49.9 against
49.8 [arxiv:2212.10509 tab. 4]. With Flan-T5-XXL (11B), HotpotQA reaches 59.1
against 49.7 [tab. 4]. In a manual check of 40 questions per set, the number of
chains with at least one false fact fell by 50% on HotpotQA [arxiv:2212.10509
§5]. Each retrieval step needs an extra model call, and the prompt grows with
each collected paragraph [arxiv:2212.10509 §Limitations].

### Model-chosen retrieval with graded passages and sentences

This is [Self-RAG](/papers/2310.11511), a Llama 2 model trained to emit
reflection tokens. A retrieve token chooses, for each segment, between no
retrieval, retrieval, and continuing with evidence already retrieved
[arxiv:2310.11511 tab. 1] [§3.1]. If retrieval is chosen, the model fetches K
passages and writes one continuation per passage. Each continuation is graded
with the relevance, support and usefulness tokens, and the highest-scoring
candidates are kept in a beam [arxiv:2310.11511 §3.3] [eq. 3] [eq. 4]
[alg. 1]. The score adds the LM probability to a weighted sum of the grades.
The weights and the retrieval threshold δ are inference-time settings. The
paper's threshold is 0.2 for most tasks and 0 for ALCE [§4.3]. Raising δ
means less retrieval [arxiv:2310.11511 fig. 3c].

The critic is trained on GPT-4 labels, and the generator is trained on the
labelled text with retrieved passage text excluded from the loss
[arxiv:2310.11511 §3.2.1] [§3.2.2]. Results are author-reported. The 13B model
scores 55.8 on PopQA, against 50.8 for Ret-ChatGPT, and 70.3 citation
precision on ASQA, against 65.1 for Ret-ChatGPT. ChatGPT stays ahead on
TriviaQA, 74.3 against 69.3 [arxiv:2310.11511 tab. 2]. In the 7B ablation, the
PopQA score is 45.5 with the full model, 42.6 without the critic, and 41.8 with
only the top passage [arxiv:2310.11511 fig. 3]. The paper reports no
wall-clock cost, and the decoding cost scales with K and B [§3.3].

### Web search ranked by publisher credibility, with quoted evidence

This is [MUSE](/papers/2403.11169), a fact-checking pipeline that writes replies
to social-media posts. An LLM writes three search queries. Results are grouped
into publisher tiers, derived from factuality and bias ratings, and searched from
the most credible tier down [arxiv:2403.11169 §Approach] [§Experiments] [code:
model/main.py#L792-L793]. Only pages published before the post are kept. Pages
that pass a relevance threshold go to an LLM, which quotes refuting and context
passages. The reply links each fact [arxiv:2403.11169 §Approach]. The thresholds
and the stop rule appear only in the code [code: model/main.py#L531] [code:
model/main.py#L831].

Expert raters scored replies 8.1 out of 10. The figures were 6.3 for
high-helpfulness Community Notes and 5.9 for GPT-4 without search [arxiv:2403.11169
fig. 2]. The authors report that retrieval adds a relative 25% to quality over
the version without it [arxiv:2403.11169 §Results]. Their cost estimate is about
0.2 USD and two minutes per post [arxiv:2403.11169 §Discussion].

### Retrieved API documentation in the prompt

This is the documentation variant of [Gorilla](/papers/2305.15334), a LLaMA-7B
model that writes API calls. It is instruction-tuned on pairs of a request and
the correct call, built from model-hub records [arxiv:2305.15334 §3.1]
[§3.2]. In retrieval mode a retriever returns the single most relevant API
document, which is appended to the request. Oracle retrieval, which returns the
correct document, is only an upper bound [arxiv:2305.15334 §4]. Training with
the documentation in the user turn is a separate choice, and the paper trains
both versions [arxiv:2305.15334 §3.2].

On its own APIBench, zero-shot Gorilla reaches 59.13 overall accuracy on
TorchHub, 71.68 on HuggingFace and 83.79 on TensorFlow Hub. Zero-shot GPT-4
reaches 38.70, 19.80 and 18.20 [arxiv:2305.15334 tab. 1]. The model trained
with the oracle document scores 0 overall without a retriever on every hub.
The model trained without documentation drops to 37.63, 11.28 and 34.30 with
BM25 at test time. With the GPT-based index it is slightly higher on TorchHub,
60.21, and lower on the other two [arxiv:2305.15334 tab. 2].

### Corpus-wide knowledge graph with community summaries

This is [GraphRAG](/papers/2404.16130), which indexes a whole corpus once and
then answers questions about all of it. An LLM extracts entities, relations and
claims into a graph. Hierarchical Leiden clustering groups the graph, and an LLM
writes a summary for each community at each level [arxiv:2404.16130 §3.1.2]
[§3.1.4] [§3.1.5]. At query time the summaries are packed into windows of about
8k tokens. Each window gives a partial answer with a helpfulness score, and the
partial answers are merged into one reply [arxiv:2404.16130 §3.1.6].

On two corpora of about one million tokens each, the authors judged the answers
against vector RAG. Comprehensiveness win rates were 72 to 83% on Podcast and
72 to 80% on News. Diversity win rates were 75 to 82% on Podcast and 62 to 71%
on News [arxiv:2404.16130 §5.1]. The judging used 125 questions per corpus
[§4.1]. Empowerment is mixed, and directness favours vector RAG, as the control
predicts [§5.1]. Root-level summaries need over 97% fewer context tokens than
map-reduce over the source text [§5.1] [tab. 2]. Indexing took 281 minutes on
the Podcast corpus [§4.1.3].

### Analyses and proposals that name retrieval

- **Rare facts.** In closed-book QA, accuracy rises with the number of
  pre-training documents that mention both the question's entity and the
  answer [arxiv:2211.08411 §3.1]. The authors fit a log-linear trend in model
  size on rare questions, with R² = 0.98, and read it as needing very large
  models to match humans [arxiv:2211.08411 §4.2]. A BM25 setting that adds the
  top three Wikipedia paragraphs beats closed-book GPT-Neo at every count, most
  of all for rare questions. The paper gives no numbers for this in the text
  [arxiv:2211.08411 §4.3].
- **Corpus pollution.** The retrieved text is only as trustworthy as the store.
  The authors add one GPT-3.5 fake passage per question to the corpus of
  open-domain QA systems [arxiv:2305.13661 §4.3]. Under the three adversarial
  settings, exact match falls 14% to 54% for DPR-based setups and 20% to 87%
  for BM25-based ones [arxiv:2305.13661 §4.4] [tab. 2]. The most damaging setting, Hack,
  asks for the false answer to be rephrased ten ways, aimed at the machine
  reader rather than people [arxiv:2305.13661 §3.2]. Defenses are preliminary. A detector trained on
  in-domain text reaches 91.4 to 99.8 AUROC, but an out-of-domain detector stays
  between 50.7 and 64.8 [arxiv:2305.13661 tab. 3]. Reader voting raises exact
  match by 11% to 23% relative to the baseline [arxiv:2305.13661 tab. 4]. Card:
  [Misinformation Pollution with Large Language Models](/papers/2305.13661).
- **Authority.** The legal survey proposes authority-grounded retrieval. A
  source must be checked for hierarchy, binding force, jurisdiction and valid
  time, and for whether it supports the claim. Conflicts stay visible
  [arxiv:2303.09136 §3.2]. The survey labels these requirements as design
  proposals, not a validated standard [arxiv:2303.09136 §1.2]. Card: [A Short
  Survey of Viewing Large Language Models in Legal Aspect](/papers/2303.09136).
- **Named only.** The hallucination survey lists retrieval-augmented
  knowledge-grounded dialogue [arxiv:2202.03629 §8.2.4] and RAG for LLMs
  [arxiv:2202.03629 §13.3] among its mitigation methods. Card: [Survey of
  Hallucination in Natural Language Generation](/papers/2202.03629). The agent
  review lists RAG among its ten mitigation mechanisms and gives it no pipeline
  detail [arxiv:2505.10468 §6.1]. Card: [AI Agents vs. Agentic AI: A Conceptual
  Taxonomy, Applications and Challenges](/papers/2505.10468). Neither gives
  enough detail to build a variant, so no pipeline is attributed to them.
- **Surveys that name RAG.** The LLM survey lists retrieval among its ways of
  augmenting a model, with no pipeline detail [arxiv:2402.06196 §4.3]. Card:
  [Large Language Models: A Survey](/papers/2402.06196).
- **Agent memory.** The memory survey describes retrieved textual memory: each
  entry is embedded when written, scored against the current query when read,
  and the top K are used [arxiv:2404.13501 §5.2.1]. It leaves K and the
  similarity function open, and gives Generative Agents' cosine score with
  recency and importance as one example [arxiv:2404.13501 §5.2.1]. It argues
  that textual memory is cheaper to write and parametric memory cheaper to read,
  without measuring either [arxiv:2404.13501 §5.2.3]. Card: [Agent-memory
  survey](/papers/2404.13501). The survey runs no experiments.
- **Taxonomy and small models.** The RAG survey sorts retrieval fusions and
  gives pseudocode for each, but runs no experiments of its own [arxiv:2407.13193
  §4] [alg. 3]. Card: [Retrieval-Augmented Generation for Natural Language
  Processing: A Survey](/papers/2407.13193). The small-model survey lists RAG as
  one way a small model can enhance an LLM [arxiv:2409.06857 §3.1.2]. Card:
  [Small-model survey](/papers/2409.06857).
- **Multi-agent chat.** AutoGen's retrieval chat app reports F1 23.40% and
  recall 62.60% without interactive retrieval. The interactive-retrieval result
  is only in a figure [arxiv:2308.08155 app. A2] [fig. 4b]. Card: [AutoGen:
  Enabling Next-Gen LLM Applications via Multi-Agent Conversation](/papers/2308.08155).

## When to use which

The table uses the constraints each claim depends on. Rows that rest on the
cards cite them. Rows marked as our inference or recommendation go beyond them.

| Situation | Use | Why |
|---|---|---|
| Answers must point to a source file, and your documents are plain text | Chunk, embed, top-k by cosine, show the source | This is the pipeline VirtualTA uses [arxiv:2309.10892 §3.2.3, §4.1.3]. Its effect is not measured. |
| Entries are named (articles, records) and users ask with the entries' own words | Keyword lookup with term-count ranking | This is ChatDoctor's pipeline [arxiv:2303.14070 fig. 4] [p. 4]. The paper does not compare it with embedding search, so which is more accurate is unknown. |
| Domain text with its own vocabulary, and you can train a retriever | Domain-tuned dense retriever | ChipNeMo's tuned retriever roughly doubles top-8 hit accuracy over untuned e5-small [arxiv:2311.00176 §3.6]. Training details are not stated, and its corpus and tests are internal [arxiv:2311.00176 §2.4]. |
| Per-query latency or model cost is tight | Prefer the cosine variant | One embedding call, one classifier call and one generation call per query [arxiv:2309.10892 §3.2.3]. The keyword variant needs N + 2 model calls, which is 7 with the paper's N = 5 [arxiv:2303.14070 p. 4-5]. |
| Users must be able to check each claim | Show the excerpt next to the answer | Sparrow showed evidence to raters [arxiv:2209.14375 §2.4]. Its measure is whether evidence supports a reply, which the paper says is not the same as correctness [arxiv:2209.14375 §3.2] [§4.1]. |
| Deciding when to search matters | Consider a choice step, not always-on retrieval | On Sparrow's supported-and-plausible metric, choosing when to search scored 0.70 against 0.61 for always searching [arxiv:2209.14375 §3.2]. This is one metric in one paper. |
| Answers from one model vary between samples, and extra search calls are costly | Search only in rounds where the sampled answers disagree, with an entropy gate | Med-Gemini does this, with three queries per uncertain question [arxiv:2404.18416 §2.1]. Its threshold is not stated, so calibrate it on your own questions. Our inference: the saving depends on how often samples agree, and the sections read here give no such rate. |
| The fact is rare in web text | Supply it at inference time, not through scale | Closed-book accuracy rises with document count [arxiv:2211.08411 §3.1], and the scale fit is an extrapolation [arxiv:2211.08411 §4.2]. Adding top-3 BM25 passages beats closed-book at all counts [arxiv:2211.08411 §4.3]. |
| Facts from a document set must reach a model, and you can run a retriever per query | Retrieve at answer time rather than continue training on the documents | Author-reported: on 910 current-events questions, Mistral 7B scores 0.875 with RAG against 0.504 after training on the original text [arxiv:2312.05934 tab. 2]. RAG on the base model beats training alone in 28 of 30 MMLU rows, the exceptions being Llama2 7B 0-shot on astronomy and college chemistry [arxiv:2312.05934 tab. 1]. The current-events questions were written from chunks in the store, which favours RAG [§4.3] [§5]. Our inference: the main-table K is not stated, so set K by testing it on your own questions. |
| Answers are relied on for a legal or regulated decision | Store status, jurisdiction and date per source, and check them before answering | The legal survey's proposal, not validated [arxiv:2303.09136 §3.2] [§1.2]. Our inference: a relevance rank alone does not cover these checks. |
| Some sources are recordings | Transcribe first, then chunk | Transcription is a separate step with a 25MB upload limit [arxiv:2309.10892 §5.1] |
| Source files are scanned PDFs | Add OCR before chunking | VirtualTA has no OCR, so scanned PDFs yield no text [arxiv:2309.10892 §5.1] |
| Records are short and structured (announcements, assignments) | Embed each record whole, without chunking | VirtualTA does this [arxiv:2309.10892 §3.2.2]. The paper does not address long records. |
| The corpus or embedding model is new to you | Calibrate the cut-off on your own queries before you rely on it | The 0.75 value is not justified in the paper [arxiv:2309.10892 §3.1.1] |
| Users must not get an answer that the sources do not support | Refuse or flag when no chunk passes, rather than answering as VirtualTA does | VirtualTA labels such answers as generated on their own [arxiv:2309.10892 §4.1.3]. Refusing is our inference from that behaviour, not something the paper tests. |
| You need to know whether retrieval works, not just whether answers read well | Log the retrieved passages and check them by hand before judging answers | Our recommendation. Neither the course chatbot nor the medical chat model reports retrieval accuracy [arxiv:2309.10892 §4]; ChatDoctor's one score is answer-level [arxiv:2303.14070 tab. 1]. |
| The model is a hosted black box that cannot be fine-tuned, and each answer must be checked against the evidence before it is sent | Consolidate evidence, score each candidate, and re-prompt with feedback when the score is low | Author-reported: customer-service KF1 is 37.41 against 31.33 for ChatGPT [arxiv:2302.12813 tab. 2], and OTT-QA F1 is 11.80 against 0.59 [arxiv:2302.12813 tab. 5]. The loop can call the LLM twice per response [arxiv:2302.12813 §6], and the threshold and stop rule are not stated [§3.2]. |
| An agent must improve across tasks without weight updates | Retrieve past successes, and consider distilled insights | On ALFWorld, retrieved trajectories alone reach 55% against 40% for ReAct [arxiv:2308.10144 §5.2.1]. Each HotpotQA trajectory uses about three times the tokens [arxiv:2308.10144 tab. 6]. |
| A fact in a deployed model must change without retraining | Store the edit and route inputs with a scope check | SERAC's edit success is above RP on all four benchmarks, with RP lower on FC drawdown [arxiv:2206.06520 tab. 3]. Author-reported on its own benchmarks. |
| Clinical use or a commercial product | Do not reuse ChatDoctor's released model | The authors call it a research prototype, not a clinical tool [arxiv:2303.14070 p. 11]. Its README prohibits commercial and clinical use of the model [code: README.md#L159]. |
| An agent needs facts it cannot know from its weights | Put retrieval behind a tool the agent calls, if your agent already has tool use | The agent review names RAG as a mitigation next to tool-augmented reasoning [arxiv:2505.10468 §6.1]. Making retrieval a tool call is our inference; the review does not specify it. |
| Your model generates sequences, and the family has few related sequences or mutants carry several changes | Mix a retrieved profile into the model's per-position scores | On ProteinGym, shallow alignments score 0.453 against EVE's 0.408, and five or more mutations 0.499 against 0.420 [arxiv:2205.13760 §6.2]. Deep alignments favour EVE, 0.507 against 0.488 [arxiv:2205.13760 §6.2]. Overall the margin is 0.003 [arxiv:2205.13760 §6.2]. |
| Images show named things (landmarks, species) and questions need outside facts | Recognise the entity, then read its Wikipedia page | On the human split this gives 18.2 against 10.8 for the best end-to-end model, but the Wikidata split favours end-to-end [arxiv:2302.11713 tab. 5]. Recognition is the bottleneck: the gold entity gives 45.6 [arxiv:2302.11713 tab. 5]. |
| A search over candidate options, where each option is scored by the model | Make retrieval one action in a tree search, with self-consistency and a yes/no check as the reward | The paper's headline is 0.4466 DrugBank recall with the ground-truth count, against 0.1552 for GPT-4o-mini [arxiv:2507.07426 tab. 1]. The protocol needs the ground-truth count, so report a fixed top-k too [arxiv:2507.07426 §3.4]. |
| Users or outside parties can add text to the corpus | Vet the sources, and add a reader-side defense before relying on answers | Injected fake passages cut exact match by up to 87% in the paper [arxiv:2305.13661 §4.4]. A detector trained in-domain helps, out-of-domain does not [arxiv:2305.13661 tab. 3]. Vetting sources is our recommendation; the paper does not test it. |
| The model is frozen or reachable only through an API, and the task is text prediction or open-domain QA | Prepend one BM25 passage, and retrieve every few tokens | With BM25, a 345M GPT-2 reaches 21.5 word perplexity against 26.3 without retrieval [arxiv:2302.00083 tab. 1]. These are perplexity numbers, not generation quality [arxiv:2302.00083 tab. 1]. |
| Questions need several hops, and the next search term is known only after the previous step | Interleave retrieval with reasoning sentences | With GPT-3, IRCoT gains 7.1 F1 over one-step retrieval on HotpotQA and 13.2 on 2WikiMultihopQA [arxiv:2212.10509 tab. 4]. Each step needs an extra model call [arxiv:2212.10509 §Limitations]. |
| You pretrain the model and can afford a very large store and index | Consider retrieval built into the architecture | RETRO's 7.5B model reaches 0.66 bits per byte on C4 against 0.78 without retrieval [arxiv:2112.04426 app. F.1]. The MassiveText index is about 93 TB [arxiv:2112.04426 §4.1]. Our inference: the storage cost limits this to large teams. |
| The model is black-box, with no weights and perhaps no logits | Use text concatenation; logits fusion only if you get logits | Only text concatenation and logits fusion work with black-box APIs [arxiv:2407.13193 §4.5]. Logits fusion runs the generator k+1 times [arxiv:2407.13193 §4.2]. |
| Replies must link sources from the open web, and publisher credibility matters | Rank web results by publisher tier, and quote evidence | Expert raters scored MUSE replies 8.1 against 5.9 for GPT-4 without search [arxiv:2403.11169 fig. 2]. The authors' study covers 232 responses [arxiv:2403.11169 §Results]. The relevance thresholds appear only in code [code: model/main.py#L531]. |
| You can fine-tune the model, and it should decide when to retrieve and grade its own citations | Self-RAG-style reflection tokens with an adaptive retrieval threshold | Author-reported: the 13B model reaches 70.3 ASQA citation precision against 65.1 for Ret-ChatGPT [arxiv:2310.11511 tab. 2]. The critic needs GPT-4 labels [arxiv:2310.11511 §3.2.1]. Each retrieved segment decodes one continuation per passage [§3.3], and no wall-clock cost is reported. |
| An agent must recall many stored records, and the full history no longer fits the prompt | Retrieved textual memory: embed on write, score on read, keep the top K | The memory survey describes this design and leaves K open [arxiv:2404.13501 §5.2.1]. It gives no measurements [arxiv:2404.13501 §5.2.3]. Our inference: choose K by testing it on your own agent tasks. |
| Tool documentation changes after training, and you can train the model | Train with the reference document in the prompt, and test with a retriever you have checked | Gorilla's model trained with documentation scores 0 zero-shot, and a weak test retriever lowers accuracy [arxiv:2305.15334 tab. 2]. Without a retriever, use the zero-shot model [arxiv:2305.15334 tab. 2]. The paper's documentation-change result is qualitative only [§4.2]. |
| Questions ask about a whole collection (themes, overall patterns), not one fact | Build a graph index with community summaries, and answer by map-reduce | Author-reported win rates over vector RAG on comprehensiveness and diversity, judged by an LLM [arxiv:2404.16130 §5.1]. Indexing took 281 minutes on the Podcast corpus [§4.1.3], and the paper does not measure fabrication [§6.1]. Our inference: the index is built once per corpus [§3.1], so frequent corpus changes mean rebuilding it; the paper does not discuss updates. |
| You must score whether an answer is supported by the retrieved context | Measure faithfulness as supported claims over all claims, and context precision over the top k chunks | The loss-and-metric survey defines both [arxiv:2307.02694 §11.2.4] [§11.2.6]. It runs no experiments and does not say how claims are extracted or how each is judged supported [arxiv:2307.02694 §11.2.6]. Our inference: an LLM judge or a human label has to fill that step. |
| Domain text is far from the retriever's training data, you can train the retriever with the generator, and you can afford repeated re-encoding | Train the retriever with the generator and refresh the index in the background | On news QA, exact match with the reconstruction task is 14.08 against 8.62 for the original design [arxiv:2210.02627 tab. 1]. Re-encoding and re-indexing cost hours on millions of passages [§3.3]. Our inference: the paper gives no store size below which the refresh cost is small. |

## Implementation sketch

A minimal version of the VirtualTA pipeline, then the keyword variant. Stages
and thresholds are from the cards; the functions `transcribe`, `chunk`,
`embed`, `generate` and `hits` are placeholders.

```pseudocode
# Index  [arxiv:2309.10892 §3.2.2]
def index(documents):
    store = []
    for doc in documents:
        # Recordings become text first; the ASR upload limit is 25MB  [§5.1]
        text = transcribe(doc) if doc.is_recording else doc.text
        # About 800 characters per chunk, sentences kept whole  [§3.2.2]
        for piece in chunk(text, target_chars=800):
            # 1,536-dimensional embedding  [§3.1.1]
            store.append((piece, embed(piece), doc.source))
    return store

# Answer  [arxiv:2309.10892 §3.1.1, §3.2.3]
def answer(query, store):
    q = embed(query)
    # Score every chunk by cosine similarity and keep the ten best  [§3.1.1]
    best = top_k(store, score=lambda c: cosine(q, c.vector), k=10)
    # Keep chunks above 0.75; the value is not justified in the paper  [§3.1.1]
    context = [c for c in best if c.score > 0.75]
    if not context:
        # The paper labels the answer as generated without course material  [§4.1.3]
        return generate(query, context=[]), []
    # Prompt with the context and an "I'm not sure" instruction  [§3.2.3]
    return generate(query, context), [c.source for c in context]

# Keyword variant  [arxiv:2303.14070 fig. 4-6] [p. 4-5]
def answer_by_keywords(query, entries, n=5):
    # The model writes the search terms  [fig. 4]
    raw = generate(keyword_prompt(query)).split(",")
    # Drop empty terms: an empty term matches every entry
    terms = [t.strip() for t in raw if t.strip()]
    # Rank entries by term hits and keep the top n  [p. 4-5]
    ranked = sorted(entries, key=lambda e: hits(e.text, terms), reverse=True)
    # The model reads each kept entry for what helps  [fig. 5] [p. 4]
    notes = [generate(select_prompt(query, e.text)) for e in ranked[:n]]
    # One final prompt over all notes  [fig. 6] [p. 5]
    return generate(final_prompt(query, notes))
```

Where variants differ: chunking and embedding change `index`, and a
domain-tuned retriever changes only `embed` [arxiv:2311.00176 §2.4]. A hybrid
or re-ranking step would replace the scoring line in `answer`. The keyword
variant replaces both halves: `index` keeps whole entries, and
`answer_by_keywords` ranks by term hits. The web-search variant replaces
`index` with a live query [arxiv:2209.14375 §2.4], and the past-success variant
returns whole trajectories instead of passages [arxiv:2308.10144 §4.2.1]. The
LLM-Augmenter wrapper keeps `generate` unchanged and adds a `utility` check
that returns feedback for the next prompt [arxiv:2302.12813 §2.4].
Self-RAG moves the retrieve decision into the decoding loop, generating one
continuation per retrieved passage and keeping the best B [arxiv:2310.11511
alg. 1].
RAG-end2end leaves `answer` as it is and moves the index refresh into a
background job [arxiv:2210.02627 §3.3].
RETRO moves retrieval into the forward pass, through chunk-wise cross-attention
[arxiv:2112.04426 §2.4]. In-Context RALM keeps the model and changes only its
input [arxiv:2302.00083 §3.1]. IRCoT loops over reasoning steps, with a
retrieval call at each step [arxiv:2212.10509 §3.1]. The
protein variant has no context prompt at all: it mixes a retrieved profile into
the model's per-position scores [arxiv:2205.13760 §4.2]. The
paper's source priority [arxiv:2309.10892 §3.2.1] has no mechanism to copy, so
it is left out. The API-documentation variant is the same answer step with
top-1 retrieval and the document appended to the request [arxiv:2305.15334
§3.2] [§4]. The graph variant replaces the top-k step with a map-reduce over
community summaries [arxiv:2404.16130 §3.1.6]. The Med-Gemini loop adds an
early exit: `answer_with_search` returns the majority answer once the entropy is
at or below a threshold, and otherwise adds search results to the next round
[arxiv:2404.18416 §2.1].

## Contradictions

No two cards report incompatible numbers for the same pipeline. Self-RAG
compares its own retrieval with top-1 and no-retrieval variants in an ablation
[arxiv:2310.11511 fig. 3], and no other card compares retrieval designs
directly. Seven points still pull in different directions, and none is resolved
here.

- **Relevance and authority.** The legal survey says retrieving relevant
  material does not establish that it is authoritative [arxiv:2303.09136 §3.2].
  The course chatbot, the medical chat model and the chip-design assistant rank
  passages by similarity or term hits [arxiv:2309.10892 §3.2.3] [arxiv:2303.14070
  p. 4] [arxiv:2311.00176 §3.6]. None reports checking source status or
  jurisdiction, so these papers neither confirm nor refute the survey's point.
- **Search on every query or on some.** On Sparrow's supported-and-plausible
  metric, always searching scored 0.61 and choosing when to search scored 0.70
  [arxiv:2209.14375 §3.2]. The course chatbot sends every query through
  retrieval [arxiv:2309.10892 §3.2.3]. SurgRAW retrieves only for
  cognitive-inference questions [arxiv:2503.10265 §3.3]. Med-Gemini searches
  only in rounds where its sampled answers disagree, a third rule [arxiv:2404.18416
  §2.1]. None of the cited comparisons tests that gate against the other two.
  The metrics and tasks differ, so this is a difference in design, not a tested
  conflict.
- **Insights or retrieved trajectories.** Within ExpeL, insights alone beat
  retrieved trajectories on HotpotQA, 36% against 31%. On ALFWorld the order
  flips, 50% against 55% [arxiv:2308.10144 §5.2.1].
- **Effect sizes are not comparable.** ChipNeMo's gain is on a 7-point rating
  [arxiv:2311.00176 app. Engineering Assistant Chatbot Human Evaluation],
  Sparrow's on a support rate [arxiv:2209.14375 §3.2], ExpeL's on success rate
  [arxiv:2308.10144 tab. 3] and SurgRAW's on accuracy [arxiv:2503.10265 tab. 4].
  Read each within its own paper.
- **Always-on retrieval.** Self-RAG argues that retrieving for every input
  can lower output quality, because passages may be off-topic
  [arxiv:2310.11511 §1]. Its own table shows the fixed-retrieval Llama 2 7B
  scoring 38.2 on PopQA, against 14.7 without retrieval [arxiv:2310.11511
  tab. 2]. In-Context RALM and IRCoT also report gains from retrieving often,
  at every few tokens or every reasoning sentence [arxiv:2302.00083 tab. 1]
  [arxiv:2212.10509 tab. 4]. LLM-Augmenter's main experiments also consolidate
  evidence on every query, and its learned policy is tested only in a learning
  curve [arxiv:2302.12813 §3.2] [fig. 3]. The Self-RAG claim is a motivation,
  not a test, and the setups differ, so none of these settles the question.
- **Retrieval frequency.** RETRO looks up neighbours once per 64-token chunk
  [arxiv:2112.04426 §2.2]. In-Context RALM finds a stride of 4 tokens best among
  the values it tried, and says that tying the stride to the query length hurts
  [arxiv:2302.00083 §5.2] [§3.2]. The models, objectives and metrics differ, so
  neither result tests the other.
- **Retrieval can lower accuracy.** Most cards report gains from retrieval.
  Gorilla reports that BM25 at test time lowers overall accuracy for the model
  trained without documentation on every hub [arxiv:2305.15334 tab. 2]. The
  paper says an imperfect test-time retriever can lower accuracy [§4]. The task
  is API choice rather than question answering, so this is a difference in
  setting, not a tested conflict.

Within IRCoT, the paper sets a limit of 8 reasoning steps, while the released
code defaults to 10 [arxiv:2212.10509 §3.1] [code:
commaqa/inference/ircot.py#L727].

Within the ChatDoctor paper, the text says the model reads five sections per
question while the released scripts read four [arxiv:2303.14070 p. 4]
[code: Autonomous_ChatDoctor_csv/csv_reader.py#L66]. Both values are stated
here; the paper's five is the one the paper describes.

## Papers

- [Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education](/papers/2309.10892): VirtualTA, a course chatbot with chunking, cosine top-k and a 0.75 cut-off; no quantitative evaluation.
- [ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge](/papers/2303.14070): a medical chat model with keyword-driven retrieval over Wikipedia or a disease table; one author-reported BERTScore comparison with ChatGPT.
- [ChipNeMo: Domain-Adapted LLMs for Chip Design](/papers/2311.00176): a domain-tuned e5-small retriever with top-8 passages; author-reported human ratings and EDA script results on internal benchmarks.
- [Improving alignment of dialogue agents via targeted human judgements](/papers/2209.14375): web-search evidence excerpts, eight-candidate reranking and RL; author-reported, with search entangled with RL.
- [ExpeL: LLM Agents Are Experiential Learners](/papers/2308.10144): retrieved past successes and distilled insights in the prompt of a frozen agent; author-reported on three text benchmarks.
- [Memory-Based Model Editing at Scale](/papers/2206.06520): SERAC, a stored-edit cache with a scope classifier and a counterfactual model; author-reported on its own benchmarks.
- [SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis](/papers/2503.10265): MedlinePlus retrieval for cognitive-inference questions inside a multi-agent workflow; author-reported ablation.
- [Large Language Models Struggle to Learn Long-Tail Knowledge](/papers/2211.08411): closed-book accuracy follows document counts; retrieval helps rare questions most; no numbers for retrieval in the text.
- [A Short Survey of Viewing Large Language Models in Legal Aspect](/papers/2303.09136): an authority-grounded framework for retrieval in legal AI; a proposal, not a validated standard.
- [Survey of Hallucination in Natural Language Generation](/papers/2202.03629): names RAG for LLMs and retrieval-augmented dialogue as mitigations; gives no pipeline detail.
- [AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges](/papers/2505.10468): a narrative review that lists RAG among ten mitigation mechanisms for agent systems; no experiments and no pipeline detail.
- [Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval](/papers/2205.13760): a protein language model that mixes a retrieved alignment profile into its scores; author-reported ProteinGym results, with a small overall margin over EVE.
- [Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?](/papers/2302.11713): InfoSeek, a visual QA benchmark, and a CLIP-then-FiD pipeline over Wikipedia; author-reported, on splits the main table does not name.
- [On the Risk of Misinformation Pollution with Large Language Models](/papers/2305.13661): LLM-written fake passages injected into open-domain QA corpora, and three preliminary defenses; author-reported on two QA sets.
- [DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree Search](/papers/2507.07426): retrieval as one action in a Monte Carlo tree search over agent steps; author-reported, with a ground-truth-count protocol.
- [Retrieval-Augmented Generation for Natural Language Processing: A Survey](/papers/2407.13193): a taxonomy of retrieval fusions with pseudocode and a qualitative comparison; no experiments of its own.
- [In-Context Retrieval-Augmented Language Models](/papers/2302.00083): one BM25 passage prepended to a frozen LM every few tokens; perplexity results on five corpora, author-reported.
- [Improving language models by retrieving from trillions of tokens](/papers/2112.04426): RETRO, chunked cross-attention over a trillion-token store, with a retrofitting option; author-reported.
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](/papers/2212.10509): IRCoT, retrieval and reasoning sentences interleaved; author-reported multi-hop QA gains with GPT-3 and Flan-T5.
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](/papers/2308.08155): a multi-agent framework with a retrieval chat app; its retrieval result is only in a figure.
- [Customized large language models can outperform Community Notes in correcting misinformation](/papers/2403.11169): MUSE, credibility-ranked web search with evidence quotes for fact-check replies; author-reported expert and reader studies.
- [Small-model survey](/papers/2409.06857): a survey that lists RAG among ways small models help LLMs; no numbers of its own.
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](/papers/2310.11511): a Llama 2 model trained to emit reflection tokens that decide when to retrieve and grade passages; author-reported, with a GPT-4-labelled critic.
- [Large Language Models: A Survey](/papers/2402.06196): a broad LLM survey that names retrieval as an augmentation method; no experiments.
- [Memory mechanisms in LLM-based agents: a survey](/papers/2404.13501): a survey of agent memory, including top-K retrieved textual memory; no experiments.
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](/papers/2302.12813): LLM-Augmenter, evidence consolidation and a utility-driven feedback loop around a black-box LLM; author-reported, with no code released.
- [Gorilla: Large Language Model Connected with Massive APIs](/papers/2305.15334): an API-calling model with optional retrieved documentation in the prompt; author-reported on its own APIBench.
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](/papers/2404.16130): GraphRAG, a graph index with community summaries answered map-reduce style; author-reported, judged by an LLM on two corpora.
- [Capabilities of Gemini Models in Medicine](/papers/2404.18416): Med-Gemini, text models that search the web only when sampled answers disagree, plus multimodal models; author-reported, with models not released.
- [Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering](/papers/2210.02627): RAG-end2end, joint training of the retriever with the generator and a background index refresh; author-reported on three domain QA sets.
- [Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs](/papers/2312.05934): fine-tuning against RAG for adding facts to 7B models, on MMLU subsets and 910 current-events questions; author-reported, RAG ahead in nearly all settings.
- [Loss Functions and Metrics in Deep Learning](/papers/2307.02694): a survey whose RAG section defines context precision and claim-level faithfulness; no experiments.
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](/papers/2401.01313): a taxonomy of 32 techniques, with retrieval placed before, during or after generation or end to end; no experiments.