A Survey on Retrieval-Augmented Text Generation

In plain words

A language model usually keeps everything it knows inside its weights. With retrieval-augmented generation, the model can also look up similar examples or documents while it writes, and build its output from them. This survey explains the general recipe: where the examples come from, how they are matched to the input, and how the model uses them.

It then goes through how the idea has been applied to chatbot replies, to translation and to other tasks, and ends with open problems. It is a literature review, not a new method, and it runs no experiments.

TL;DR

• A survey, not a method: it gives a general paradigm and a map of existing work §1 §2.
• The paradigm has three choices: where memory comes from §2.2, how it is scored §2.3, and how it enters the generator §2.4.
• Covered tasks: dialogue response generation §3, machine translation §4 and other generation tasks §5.
• It reports no experiments and no numbers. Implementers must choose all settings themselves.

Problem

Storing all needed knowledge in model parameters is the usual approach for text generation. The authors argue that an explicit, plug-and-play memory scales better, and that generating from a retrieved human-written reference can ease the task §1. The paper aims to review representative approaches across these tasks §1.

Method

The paper formalises the paradigm as an output y that depends on the input x and on a set z of retrieved pairs (x^r, y^r) drawn from a memory §2.1 eq. 1. Retrieval may be empty when the memory has no input side, for example with unsupervised sources §2.1. The three components are:

1. Retrieval source §2.2. The training corpus; an external dataset in the same format; or a large unlabelled corpus, which the survey illustrates with a cross-lingual retriever for translation §2.2.
2. Retrieval metric §2.3. Sparse-vector scores such as TF-IDF and BM25, which match keywords through an inverted index; dense-vector scores, usually inner products of encoder outputs §2.3; and task-specific metrics trained end-to-end with the generator §2.3.
3. Integration §2.4. Data augmentation, which concatenates retrieved material with the input; attention over separately encoded retrieved targets; and skeleton extraction, which keeps only a masked part of a retrieved response §2.4.

Per-task coverage, as organised by the survey:

• Dialogue response generation §3. Retrieval-based and generation-based chatbots are contrasted §3, Background. Shallow integration re-ranks the outputs of both §3, Shallow Integration. Deep integration feeds retrieved responses into the generator, sometimes via an extracted skeleton [§3, Deep Integration]. Knowledge-grounded generation uses knowledge bases and documents as the memory §3, Knowledge-Enhanced Generation.
• Machine translation §4. Translation memory in statistical MT appears in three forms: constrained decoding, phrase-table aggregation and parameter tuning §4.1. In neural MT the memory either shapes inference, by rewarding target words, or training, by augmenting inputs or adding encoders §4.2.
• Other tasks §5. Language modelling, summarisation, paraphrase, style transfer and data-to-text generation are listed, each with one or a few representative methods §5.
• Future directions §6: retrieval sensitivity, retrieval efficiency, local versus global optimisation, multi-modal retrieval, and diverse, controllable retrieval §6.

Figure 1 is the survey's overview figure, according to its caption fig. 1.

# Generic retrieval-augmented generation  [§2.1] [eq. 1]
# Input:  input x; memory as a list of (x_r, y_r) pairs
# Output: generated text y

def retrieval_augmented_generate(x, memory, scorer, k, generator):
    # 1. Score each memory pair against the input  [§2.3]
    # Scorer may be sparse (TF-IDF, BM25), dense or task-trained.
    scores = [scorer(x, pair) for pair in memory]
    # 2. Keep the k best pairs as z  [§2.1]; k is not stated.
    z = top_k(memory, scores, k)
    # 3. Feed x and z into the generator  [§2.4]
    # Options: concatenation, attention over z, or skeletons from z.
    return generator(x, z)

Hyperparameters

The paper is a survey and specifies no training or inference setup.

Results

The paper reports no experiments and no benchmark numbers. The table below lists the scope the survey covers, not results. Author-reported claims are qualitative and appear in the Problem and Pitfalls sections.

Paper vs code

No official code. The LaTeX source contains no code link, and a GitHub search for the title and method names found no repository from the authors.

Pitfalls

• No setup to reproduce. Choosing k, the metric and the integration method is left to the reader §2.1 §2.3 §2.4. Any reproduction needs its own experiments.
• Retrieval quality decides the outcome. The authors say these methods do well when retrieved examples closely match the query, and can do worse than a model with no retrieval when they do not §6, Retrieval Sensitivity.
• Cost grows with memory. A larger memory is more likely to hold a close match, but retrieval overhead slows inference. The survey suggests compressing memory §6, Retrieval Efficiency.
• Dialogue collapse. The generator can learn to ignore retrieved responses and behave like a plain sequence model, especially when retrieved and target responses disagree §3, Deep Integration. Earlier fixes filter training data by similarity, which shrinks the usable data §3, Deep Integration.
• One retrieved response in dialogue. Most dialogue methods use only one, and retrieval uses general relevance scores rather than task-specific ones [§3, Limitations].
• Surface-form bias of sparse metrics. Sparse scores favour similar wording and can miss passages that are semantically relevant §2.3.
• Training and inference mismatch. Jointly learned retrievers receive loss from only a few retrieved items during training, but rank over the whole memory at inference §6, Local vs. Global Optimization.
• Translation memory at every step. At inference, memory can be applied at each decoding step, which costs speed and can reduce fluency §4.2, Limitations.
• Abstract wording. The arXiv abstract for v2 says "important directions" where the LaTeX source says "promising directions" §6. The summary above follows neither word.

Lineage

• Builds on: not applicable; this is a survey. The survey cites example works for each task in its introduction §1: a retrieval dialogue model, a retrieval-guided MT model and a code-generation model.
• Compared against: not applicable.
• Superseded by: not stated in this card.