---
kind: concept
slug: hallucination-mitigation
title: Hallucination mitigation
summary: >-
  Ways to make generated text stay faithful to its source or to world knowledge: clean or augment the training data, change architecture, objective or decoding, edit activations, or check outputs after generation. For LLMs add retrieval, RLHF, model editing, debate and self-verification. Trade-off: faithfulness against fluency, diversity, cost and extra failure points.
tags: [hallucination, faithfulness, factuality, nlg, llm, retrieval, multimodal]
papers: ["2202.03629", "2201.08239", "2206.04624", "2302.00923", "2302.12813", "2303.14070", "2304.10428", "2305.14325", "2305.15334", "2306.03341", "2306.13549", "2307.10169", "2309.05922", "2309.10892", "2310.03744", "2310.10683", "2310.11511", "2311.16922", "2401.01313", "2403.11169", "2407.13193", "2411.11394", "2601.03267"]
related: [hallucination-detection, dataset-curation, chain-of-thought-prompting, vision-transformer, retrieval-augmented-generation, rlhf, language-model-pretraining, activation-steering, model-editing, multi-agent-systems, multimodal-language-model, automated-fact-checking, tool-use, decoding-strategies, self-refinement, llm-evaluation]
status: draft
updated_at: 2026-11-01
---

## Overview

A generation model hallucinates when its output is nonsensical or not faithful to the source it was given [arxiv:2202.03629 §2]. Two kinds are distinguished: *intrinsic* hallucinations contradict the source, *extrinsic* ones can be neither confirmed nor refuted from it, and may even be true [arxiv:2202.03629 §2.1]. *Faithfulness* (agreement with the source) is kept apart from *factuality* (agreement with world knowledge) [arxiv:2202.03629 §2.3]. What the "source" is depends on the task: the document for summarization, the table or meaning representation for data-to-text, dialogue history plus knowledge for dialogue, world knowledge for generative QA, and input plus training data for LLMs [arxiv:2202.03629 §2.2, §13.1].

Mitigation targets the causes. The survey groups them into [arxiv:2202.03629 §3]:

1. **Data:** references that say more than the source, from heuristic collection (62% of WikiBio first sentences contain facts missing from the infobox, as reported from Dhingra et al.), duplicated pretraining data, or tasks that reward diversity such as chit-chat [arxiv:2202.03629 §3.1].
2. **Training and inference:** weak encoder representations, attention to the wrong part of the input, diversity-oriented sampling (top-k sampling correlates with more hallucination, per Dziri et al.), exposure bias from teacher forcing, and pretrained knowledge overriding the given input [arxiv:2202.03629 §3.2].

Every method attacks one of these two groups, so the shared design question is: *which cause dominates in my setting, and what does the fix cost in fluency, diversity or compute?* You need a detection metric to measure progress; see [hallucination-detection](/concepts/hallucination-detection).

The LLM survey sorts the fixes by where they act: prompt-side methods (retrieval, self-refinement, prompt tuning) and model-side methods (decoding, knowledge graphs, faithfulness losses, fine-tuning) [arxiv:2401.01313 §2, §3]. Its taxonomy is a navigation aid, not a mechanism description (see Contradictions).

Twenty-three cards cover this concept. Six are surveys: [the NLG survey](/papers/2202.03629) catalogues the families below, [the LLM survey](/papers/2401.01313) sorts 32 LLM techniques into prompt-engineering and model-development families, with a per-technique table [arxiv:2401.01313 tab. 1], [the multimodal LLM survey](/papers/2306.13549) covers hallucination in vision-language models, [the RAG survey](/papers/2407.13193) maps the ways retrieved text enters a model, [the foundation-model survey](/papers/2309.05922) tabulates 28 works across text, image, video and audio, each tagged for detection or mitigation [arxiv:2309.05922 tab. 1], and [the challenges survey](/papers/2307.10169) lists hallucination among open LLM problems and names retrieval and decoding changes as its mitigations [arxiv:2307.10169 §2.8]. None of them benchmarks the families against each other, and the numbers they quote come from other papers [arxiv:2202.03629 §2] [arxiv:2306.13549 §1] [arxiv:2401.01313 §2, §5] [arxiv:2407.13193 §1] [arxiv:2309.05922 §1] [arxiv:2307.10169 §1]. The other seventeen are single systems, each with a narrow claim. The newest is [the GPT-5 system card](/papers/2601.03267), a closed model family whose hallucination and factuality results are author-reported, with no training recipe [arxiv:2601.03267 §2, §3.7]. A factuality benchmark with a decoding change and a continued-training change for open-ended LMs, measured with Wikipedia-based error metrics [FactualityPrompts](/papers/2206.04624). Gradient-ascent unlearning of bad answers, measured with a BERTScore rule on one hallucination set [LLM Unlearning](/papers/2310.10683). A retrieval-based course assistant that reports no measurements [Sajja et al., VirtualTA](/papers/2309.10892). A vision-language model that reports a qualitative resolution effect and one measured abstention gain [LLaVA-1.5](/papers/2310.03744). An LLM-based named entity recognizer whose self-verification pass gives a small measured F1 gain on one benchmark [GPT-NER](/papers/2304.10428). A medical chat model that adds a keyword-driven retrieval loop but measures answer similarity, not hallucination [ChatDoctor](/papers/2303.14070). A dialogue model trained to call tools, whose groundedness score counts only replies with external claims [LaMDA](/papers/2201.08239). A debate scheme among copies of one chat model, with author-reported gains on factual biographies [Du et al., multiagent debate](/papers/2305.14325). An inference-time edit to attention heads that raises TruthfulQA scores [ITI](/papers/2306.03341). A two-stage chain-of-thought model that adds image features [Multimodal-CoT](/papers/2302.00923). A misinformation-correction pipeline that writes source-linked replies, judged by expert fact-checkers [MUSE](/papers/2403.11169). A generator trained to emit retrieval and critique tokens, which decides when to fetch evidence and grades its own support [Self-RAG](/papers/2310.11511). A training-free decoding change for vision-language models that contrasts the output with a noised copy of the image [Visual contrastive decoding](/papers/2311.16922). A plug-in retrieval loop around a hosted model, [LLM-Augmenter](/papers/2302.12813), is described under retrieval below. A fine-tuned model that writes API calls, [Gorilla](/papers/2305.15334), is described there too. A pipeline that writes navigation instructions and regenerates any that disagree with their route, [InstruGen](/papers/2411.11394), is a data-side variant below.

## Variants

### Data-side: build, clean or augment the dataset

- **Build a faithful dataset** [arxiv:2202.03629 §5.1.1]. Annotators either write targets from scratch (risk: low diversity) or revise real sentences in three passes: trim phrases unsupported by the source, decontextualize, then fix the syntax.
- **Filter automatically** [arxiv:2202.03629 §5.1.2]. Score each source–reference pair with a hallucination metric, rank, and drop the lowest. The threshold is not stated. Because divergence is mostly word-level, a single score per instance throws away signal.
- **Filter with learned discriminators** (dialogue) [arxiv:2201.08239 §6.1]. LaMDA's discriminators score 2.5M pre-training turns for sensibleness, specificity, interestingness and safety, and 800K turns are kept for fine-tuning. These are dialogue-quality and safety labels rather than factuality labels, so this is a quality filter, not a faithfulness fix.
- **Correct the input instead of the target** (data-to-text) [arxiv:2202.03629 §5.1.2]. Parse a meaning representation from the reference and repair the input MR by slot matching, so the pair agrees again.
- **Augment the input** [arxiv:2202.03629 §5.1.3] with entities, extracted relation triples, pre-computed operation results, synthetic perturbations, retrieved knowledge or retrieved similar examples, so the model has the facts it would otherwise invent.
- **Regenerate on mismatch** (vision-and-language navigation) [arxiv:2411.11394 §3.2]. A large multimodal model writes an instruction for a route of labelled room and action triplets. The text is then parsed back into (room, action) pairs, and the instruction is regenerated if those pairs differ from the route labels. [InstruGen](/papers/2411.11394) adds regex clean-up after this check. Its ablation on the author's pipeline reports R2R val-unseen success rate rising from 67.42 to 69.32 with the check, and to 70.9 with clean-up [arxiv:2411.11394 tab. 1]. The paper reports no instruction-level error rate. The retry limit and match rule are not stated [arxiv:2411.11394 §3.2]. The check cannot catch errors in the route labels, which come from CLIP and an estimated action model [arxiv:2411.11394 §3.1].

For LLMs the data side becomes pretraining-corpus quality (up-sampling factual sources), small curated instruction sets (the survey cites LIMA's 1,000 examples and filtered versions of Alpaca's 52k) and clean reward-model data [arxiv:2202.03629 §13.3].

### Architecture and decoding changes

Encoder and attention changes include dual sequence + graph encoders, source-conditioned attention bias, sparse attention and inductive attention [arxiv:2202.03629 §5.2.1]. Decoder changes include multi-branch, uncertainty-aware, dual (sequential + tree) and constrained decoders, plus decoding schemes that lower sampling randomness as generation proceeds [arxiv:2202.03629 §5.2.1]. Uncertainty-aware beam search adds a weighted uncertainty penalty to the beam score; the weight is not stated [arxiv:2202.03629 §10.3, §12.1]. The survey names fluency and diversity as the price [arxiv:2202.03629 §5.2.1].

LLM decoding variants need access to the model internals [arxiv:2202.03629 §13.3]. A vision-language variant, [visual contrastive decoding](/papers/2311.16922), applies the same contrast idea to the image; see the vision-language section below:

- **DoLa** contrasts the next-token distributions of late and early layers.
- **CAD** (context-aware decoding) contrasts predictions made with and without the context.
- **ITI** (inference-time intervention) probes attention heads for truthfulness directions and shifts activations along them; see the next section.
- **Factual-nucleus sampling** is a top-p variant for open-ended text. The nucleus starts at p for each sentence, shrinks by a factor λ at every token, is reset at the start of each sentence, and never falls below a floor ω [arxiv:2206.04624 §5.1]. Its best reported setting on 1.3B (p 0.9, λ 0.9, ω 0.3) cuts Wikipedia-based named-entity error on factual prompts from 52.4% to 42.1% compared with top-p 0.9, but lowers the distinct-n-gram diversity from 0.88 to 0.55 [arxiv:2206.04624 tab. 4]. Greedy decoding gives 39.9% on the same metric with diversity 0.05 [arxiv:2206.04624 tab. 4].

### Inference-time activation editing

On LLaMA-7B the ITI authors report a gap of about 40 points between how well linear probes read the truth from hidden states and how often the model generates the true answer [arxiv:2306.03341 §1]. [Inference-Time Intervention](/papers/2306.03341) uses that gap. It fits a linear probe to each attention head on labelled true and false answers, keeps the K heads whose probes score best, and at every decoding step adds a scaled truthful direction to their outputs. The direction is the mean of the true activations minus the mean of the false ones (mass mean shift), which the authors find works best among the directions they compare [arxiv:2306.03341 §3.2] [arxiv:2306.03341 §3.3] [arxiv:2306.03341 tab. 3]. No weights change, but the method needs white-box access and labelled pairs. The paper says as few as about 40 pairs can suffice [arxiv:2306.03341 §2].

Author-reported TruthfulQA gains on true*informative are 32.5% to 65.1% for Alpaca and 51.5% to 74.0% for Vicuna [arxiv:2306.03341 tab. 2]. On LLaMA-7B with 2-fold validation the gain is 30.5% to 42.3% [arxiv:2306.03341 tab. 3]. Transfer without retuning to Natural Questions, TriviaQA and MMLU adds 4.7, 1.5 and 4.45 points; the authors call this "somewhat better", and the gains are small [arxiv:2306.03341 tab. 4] [arxiv:2306.03341 §5.3].

The cost shows in the same tables. For Vicuna, the KL divergence on a text sample rises from 0 to 1.41 and cross-entropy from 2.63 to 3.36 [arxiv:2306.03341 tab. 2]. The strength alpha trades truthfulness against informativeness, and higher values give more "I have no comment" answers [arxiv:2306.03341 §5.4] [fig. 6]. Alpha is measured in units of each head's activation spread, so it has to be retuned for every model [arxiv:2306.03341 app. B]. The paper's K=48 and alpha=15 come from a sweep on LLaMA-7B, but Table 3 lists alpha=20 as optimal for the same direction [arxiv:2306.03341 §4.3] [tab. 3].

### Training objectives

[arxiv:2202.03629 §5.2.2, fig. 1]

- **Planning / sketching:** generate a plan first, then the text. Two-stage pipelines can amplify errors from the first stage.
- **Reinforcement learning** with rewards derived from hallucination metrics, such as a slot-consistency reward, an NLI-based persona-consistency reward, or ROUGE combined with a cloze score.
- **Multi-task learning** with auxiliary tasks such as word alignment, entailment or rationale extraction.
- **Controllable generation:** treat the hallucination level as an attribute to control, via controlled re-sampling or control codes that are either supplied by hand or predicted automatically [arxiv:2202.03629 §5.2.2]. In dialogue, control codes are concatenated with the input so the model attends to how much the response relies on the knowledge evidence [arxiv:2202.03629 §8.2.4]. The same knob can also allow more hallucination where an application tolerates it [arxiv:2202.03629 §5.2.2].
- **Regularization and reconstruction losses.**
- **Factuality-aware continued training** on Wikipedia. Each sentence gets its article title as a prefix, and the loss starts at a pivot inside the sentence, so earlier tokens are not scored. The authors recommend the middle word as pivot [arxiv:2206.04624 §6.1, §6.2]. Wikipedia continued training with no changes does not reliably improve factuality on its own [arxiv:2206.04624 §6.3, tab. 5].
- **Contrastive learning** against synthetically hallucinated summaries [arxiv:2202.03629 §7.3.2].
- **NMT-specific:** perturbation augmentation, corpus filtering, minimum risk training, scheduled sampling against exposure bias, and token loss truncation or decoder hidden-state masking [arxiv:2202.03629 §11.3].

For LLMs: **RLHF**. As reported in the survey, InstructGPT halves the hallucination rate of GPT-3 on closed-domain tasks, 41% to 21% [arxiv:2202.03629 §13.3.2]. **Model editing**, e.g. Elastic Weight Removal, which rates each parameter's share in the unwanted behaviour with the Fisher information matrix and then interpolates parameters to remove it [arxiv:2202.03629 §13.3].

### Closed-model post-training: browsing and abstention

The [GPT-5 system card](/papers/2601.03267) says its models were trained to browse effectively for current information, and to reduce hallucination when they answer from internal knowledge [arxiv:2601.03267 §3.7]. The card gives no training recipe, data mix or reward for this, so only the measured behaviour and its evaluation setup can be used [arxiv:2601.03267 §2].

On SimpleQA without web access, the author-reported hallucination rate is 0.40 for the thinking model and 0.46 for o3, at accuracy 0.55 and 0.54 [arxiv:2601.03267 §3.7]. The mini variant is reported at accuracy 0.22 and hallucination rate 0.26, against 0.24 and 0.75 for o4-mini, and the authors say it abstains much more than o4-mini [arxiv:2601.03267 §3.7]. The paper also reports a drop of about 8x in hallucinations on challenging health conversations, from o3 to the thinking model [arxiv:2601.03267 §3.10]. On open-ended factuality it reports over five times fewer factual errors than o3, with and without browsing, but the graded values appear only in figures [arxiv:2601.03267 §3.7].

The paper's hallucination grader is o3 with browsing, applied to batches of ten claims, and the authors report 75% agreement with human graders [arxiv:2601.03267 §3.7]. Rates from a model grader are a different measure from exact-match or human-labelled rates, so compare them only within the same grader.

### Multi-agent debate

[Multiagent debate](/papers/2305.14325) runs several copies of one chat model. Each copy answers, then reads the others' answers and revises its own, for a few rounds. It uses only text prompts, so it needs no model internals [arxiv:2305.14325 §1] [arxiv:2305.14325 §2.1]. The paper does not define how the final answer is chosen from the copies [arxiv:2305.14325 §2.1].

With gpt-3.5-turbo, three agents and two rounds, the author-reported accuracies rise from 67.0% to 81.8% on arithmetic and from 77.0% to 85.0% on GSM8K [arxiv:2305.14325 tab. 1]. On factuality they rise from 66.0% to 73.8% on biographies and from 63.9% to 71.1% on MMLU, and chess move validity from 29.3% to 45.2% [arxiv:2305.14325 tab. 2]. On the same tables, single-agent self-reflection lowers GSM8K and MMLU accuracy against the plain single agent [arxiv:2305.14325 tab. 1] [tab. 2].

Three limits matter for a faithfulness use. The paper reports that debates often converge on one answer that can still be wrong, with the models confidently affirming it [arxiv:2305.14325 §5]. The biography judge is the same model family as the generator, so their blind spots may overlap [arxiv:2305.14325 §A.2]. Cost grows with the number of agents and rounds. Each agent makes one call per round, and the paper's round counting is not explicit: three agents over two rounds is six calls per question if the initial answers count as a round [arxiv:2305.14325 tab. 1].

### Unlearning undesirable answers

Gradient ascent on the answer tokens of bad question–answer pairs, with two regularisers: a random-mismatch loss that trains each bad prompt toward an unrelated answer, and a KL term that keeps the output on normal data close to the original model [arxiv:2310.10683 §4] [eq. 2]. It needs only negative examples, not good replies [arxiv:2310.10683 §2]. On a hallucination set built from wrong answers, the rate flagged by a BERTScore rule falls from 58.5% to 11% (gradient ascent) or 15% (with the mismatch term) on OPT-1.3B [arxiv:2310.10683 tab. 5]. On Llama 2 (7B) it falls from 49.5% to 13% or 11.5%. The authors state that hallucination falls less than harmful output or copyright leaks, and call it the harder task [arxiv:2310.10683 §8].

The cost is in the outputs. Gradient ascent alone makes normal answers nonsensical, which the authors show on normal prompts [arxiv:2310.10683 §3.2] [tab. 2]. Their table reports fluency as "not meaningful" for these rows [tab. 5]. The forget-set loss can keep rising while the output is still harmful, so stopping on the loss is unreliable; the authors measure outputs instead [arxiv:2310.10683 §3.2] [tab. 1]. Use this only where a blank or degenerate reply is acceptable.

### Retrieval augmentation

Give the model retrieved evidence at generation time: retrieval-augmented knowledge-grounded dialogue [arxiv:2202.03629 §8.2.4] and RAG for LLMs [arxiv:2202.03629 §13.3]. It falls under input augmentation [arxiv:2202.03629 §5.1.3] but needs a retriever at inference. It degrades when retrieval is poor [arxiv:2202.03629 §13.3].

[LLM-Augmenter](/papers/2302.12813) keeps the LLM as a black box and wraps it in plug-in parts. A knowledge consolidator retrieves evidence and links it into short evidence chains [arxiv:2302.12813 §2.3.1]. A utility module scores each candidate answer and writes feedback, which is added to the prompt for another round until the score passes; the threshold is not stated [arxiv:2302.12813 §2.4] [§3.2]. The main experiments use a fixed rule that always consolidates evidence, with ChatGPT as the model. A learned T5-Base policy appears only in a learning-curve experiment [arxiv:2302.12813 §3.2] [§3.3] [fig. 3]. With BM25 retrieval and feedback, Knowledge F1 is 36.41 against 26.71 for ChatGPT alone on news chat, and 37.41 against 31.33 on customer service [arxiv:2302.12813 tab. 1] [tab. 2]. These are author-reported, and the paper gives no model snapshot. The loop can call the LLM twice per response [arxiv:2302.12813 §6]. No official code is released, so reproduce it from the paper's description alone [arxiv:2302.12813 §2].

[Self-RAG](/papers/2310.11511) moves the retrieval decision into the model. It is trained on text labelled with four reflection tokens: Retrieve, IsRel (passage relevance), IsSup (support for the segment) and IsUse (usefulness of the answer) [arxiv:2310.11511 §3.1] [tab. 1]. At inference it retrieves only when the Retrieve token passes a threshold, writes one continuation per passage, and keeps the best candidates by LM probability plus weighted critique scores [arxiv:2310.11511 §3.3] [eq. 3] [eq. 4]. The threshold and the critique weights are inference-time knobs. Raising the support weight raises ASQA citation precision and lowers fluency [arxiv:2310.11511 fig. 3b]. Author-reported, the 13B model beats ChatGPT on PubHealth (74.5 vs 70.1) and PopQA (55.8 vs 29.3), and reaches 70.3 citation precision on ASQA against 65.1 for retrieval-augmented ChatGPT [arxiv:2310.11511 tab. 2]. In the 7B ablation, removing the critic lowers ASQA from 32.1 to 18.1 and retrieving only the top passage lowers it to 28.6 [arxiv:2310.11511 §5.2] [fig. 3]. The critic is distilled from GPT-4 labels [arxiv:2310.11511 §3.2.1], and the cost grows with passages and beam width. The paper reports no wall-clock figures [arxiv:2310.11511 §3.3].

The RAG survey splits the step where retrieved text meets the model into four fusion types: text placed in the prompt, mixing of the model's output distribution with one computed on each passage, injection of passage features into the network's hidden states, and per-document adapter weights merged at query time [arxiv:2407.13193 §4]. Its qualitative table rates prompt concatenation as black-box, memory- and latency-heavy, and suited to fast rollout and knowledge that changes often; it rates parametric injection as white-box and suited to fixed domain knowledge [arxiv:2407.13193 tab. 2]. The survey does not benchmark these against each other [arxiv:2407.13193 §4.5].

A concrete instance is VirtualTA, a course chatbot over LMS files and lecture transcripts [Sajja et al., arxiv:2309.10892 §3.2]. It embeds chunks of about 800 characters, keeps the ten nearest by cosine similarity, and then keeps only those above 0.75 [arxiv:2309.10892 §3.2.3]. If none pass, the answer is marked as generated without course material [arxiv:2309.10892 §4.1.3]. The paper also states that hallucination control is prompt-only, with "I'm not sure" instructions, and that this effect is not measured [arxiv:2309.10892 §3.1.1, §3.2.3]. The 0.75 cut-off is not justified in the paper, so calibrate it on your own embedding model and corpus.

ChatDoctor is a second instance, in medicine, where retrieval is one part of a larger fine-tune [arxiv:2303.14070 p. 2-5]. The model first writes search keywords for the patient's question, then the keywords are matched against Wikipedia pages or a disease table and sections are ranked by the number of keyword hits. The model reads the top sections one at a time, keeps the parts that help, and writes the answer from those notes [arxiv:2303.14070 p. 4-5]. The paper sets the number of sections read to five, while its released retrieval code reads four; this is recorded in its card [arxiv:2303.14070 p. 4]. The authors motivate retrieval by terms missing from the training data (their example is a recent disease name) and by answer specificity, and show these with hand-picked examples rather than counts [arxiv:2303.14070 p. 5-7]. They also say Wikipedia is not fully reliable and that more authoritative sources could replace it [arxiv:2303.14070 p. 3]. Keyword-count ranking is a simple retriever; ties and matching rules are not defined in the paper.

Tool calls learned in fine-tuning are a third route. LaMDA is fine-tuned to write a query for a retrieval system, a calculator or a translator, and then to write its reply from the returned snippets [arxiv:2201.08239 §6.2]. Its worked example caps the research loop at four steps and does not say what is returned when the cap is hit [arxiv:2201.08239 §6.2]. At 137B the author-reported groundedness is 73.2%, against 57.9% for the pre-trained model and 67.9% for the quality-safety fine-tune without tools [arxiv:2201.08239 app. F]. Citation accuracy is 65% [arxiv:2201.08239 §7]. The groundedness score counts only replies that contain claims about the external world [arxiv:2201.08239 §4.1], and LaMDA stays far behind crowdworkers with tools on groundedness, by the authors' own comparison [arxiv:2201.08239 §7].

[Gorilla](/papers/2305.15334) targets a narrower failure: invented API names, model repositories and arguments. Its LLaMA-7B model is instruction-tuned on about 1,645 hub APIs, each paired with ten synthetic requests [arxiv:2305.15334 §3.1, §3.2]. In its hallucination column, zero-shot Gorilla scores 6.98, 10.95 and 5.40 on Torch Hub, HuggingFace and TensorFlow Hub, against 36.55, 37.16 and 78.65 for zero-shot GPT-4 [arxiv:2305.15334 tab. 1]. The paper defines a hallucination as a call that matches no reference API, so this measures invented calls to a fixed API set, not factual errors in prose [arxiv:2305.15334 §3.3]. The same model can instead be trained with the reference documentation in its prompt and given a retriever at test time [arxiv:2305.15334 §3.2]. That variant depends on the retriever. A model trained without documents loses overall accuracy with BM25 at test time on every hub: 37.63, 11.28 and 34.30 against 59.13, 71.68 and 83.79 zero-shot [arxiv:2305.15334 tab. 2]. A model trained with the correct document scores 0 overall zero-shot [arxiv:2305.15334 tab. 2], so a deployment without a retriever needs the zero-shot model. These are author-reported results on the authors' own benchmark.

[MUSE](/papers/2403.11169) applies retrieval to misinformation. It asks an LLM for search queries, retrieves pages from publishers rated for factuality and bias, keeps pages whose text is close to the post, quotes refuting and context sentences, and writes a reply with source links [arxiv:2403.11169 §Approach]. Expert fact-checkers rated its replies 8.1 out of 10, against 6.3 for high-helpfulness Community Notes and 5.9 for GPT-4 without retrieval (n=232 responses each) [arxiv:2403.11169 fig. 2]. These are the authors' own studies, on English text and images from one platform [arxiv:2403.11169 §Discussion]. The paper reports calibrating its relevance thresholds on 100 page–post pairs but does not state the values it chose [arxiv:2403.11169 §Experiments]. Its text stop rule is two refuting pages, while the released code counts three refuting items, so a reproduction has to pick one [arxiv:2403.11169 §Approach]. Expert agreement was only fair for most criteria [arxiv:2403.11169 §Experiments].

The retrieval cards also show that the step's quality depends on a cut-off. VirtualTA and MUSE both drop pages below a similarity score. The VirtualTA cut-off of 0.75 is not justified in its paper, and MUSE does not state the values it calibrated. Calibrate the cut-off on your own corpus and say what happens when nothing passes.

### Post-processing: generate, then correct

A separate corrector edits or re-ranks the output [arxiv:2202.03629 §5.2.3, §7.3.3]: SpanFact, a corrector trained on heuristically corrupted data, re-ranking contrast candidates built by entity swaps, and HERMAN, which re-ranks by quantity consistency. In dialogue, a token-level critic flags suspect spans and an entity retriever fills replacements [arxiv:2202.03629 §8.2.4]. Little training data is needed, and the base model is untouched, but the edited text may be ungrammatical [arxiv:2202.03629 §5.2.3].

LLM counterparts [arxiv:2202.03629 §13.3]:

- **Chain-of-Verification:** draft an answer, plan verification questions, answer each independently of the draft, then revise.
- **Post-editing** systems such as PURR, REFEED and CRITIC; CRITIC checks and corrects its first output with external tools such as search engines and code interpreters.
- **Multi-LLM debate:** covered in its own section above [arxiv:2305.14325 §2.1].
- **Self-verification of extracted items** [arxiv:2304.10428 §4.3] [fig. 3]: after the model proposes candidate entities, a second prompt asks a yes/no question about each one, with demonstrations, and a "no" removes the span. The paper states the verification task for entity recognition only. The number of verification demonstrations is not stated.

On CoNLL2003 with the full test set, the verified rows reach 90.62 F1 (zero-shot verification) and 90.91 (few-shot), against 89.97 for the GPT-3 row without verification [arxiv:2304.10428 tab. 2]. That is a gain of 0.65 to 0.94 F1 on one benchmark, reported by the authors. Verification costs one extra model call per candidate span, sent one at a time [arxiv:2304.10428 §4.3].

### Vision-language: input resolution, grounded rationales and abstention prompts

The MLLM survey lists three kinds of object hallucination: existence, attribute and relationship errors. It names CHAIR, POPE and MME as evaluations, and groups mitigations by whether they act before, during or after generation [arxiv:2306.13549 §6.1] [arxiv:2306.13549 §6.2] [arxiv:2306.13549 §6.3]. It cites one study finding that input resolution matters more than encoder size or training data mix, and it states this as a general result [arxiv:2306.13549 §2.1].

LLaVA-1.5 is a vision-language model; the survey's vision-language section covers the captioning side of this problem [arxiv:2202.03629 §12]. The authors say, without a table, that hallucination in detailed image descriptions drops at 448² input. They attribute it to training annotations that contain detail the input resolution cannot show [arxiv:2310.03744 §5.2]. Their higher-resolution variant tiles the image into 224² crops and adds a downscaled global view [arxiv:2310.03744 §3.4]. In the 7B ablation, adding the global view gives +0.9 on GQA and +71 on MME [arxiv:2310.03744 §4.2].

[Visual contrastive decoding](/papers/2311.16922) needs no training. At each token it compares two next-token distributions, one from the image and one from a copy with Gaussian noise added, and samples from (1+α)·logits(image) − α·logits(noised image) with α=1 [arxiv:2311.16922 §3.3.1] [eq. 3]. Only tokens whose probability under the clean image is at least β=0.1 times the top probability are kept [arxiv:2311.16922 §3.3.2] [eq. 4]. The authors argue that a weaker image makes the model lean more on language priors, so contrasting the two readings suppresses tokens that come from those priors [arxiv:2311.16922 §3.2]. On POPE they report up to +5.8 accuracy and +7.4 F1 across three LVLMs [arxiv:2311.16922 §4.2]. As one example, LLaVA-1.5 on MSCOCO random rises from 83.29 to 87.73 accuracy [arxiv:2311.16922 tab. 1]. On the MME hallucination subset the total rises for all three models, most for InstructBLIP, from 380.33 to 447.67 [arxiv:2311.16922 tab. 2]. The Position subset is slightly lower with it for Qwen-VL and InstructBLIP [arxiv:2311.16922 tab. 2]. Each generated token needs a second forward pass on the noised image, and the paper gives no timing [arxiv:2311.16922 §3.3.1].

Fixed answer-format suffixes also teach the model when to abstain. With the instruction "respond with 'Unanswerable'" on VizWiz, correct unanswerable responses rise from 11.1% to 67.8% [arxiv:2310.03744 §4.3]. This measures abstention on unanswerable questions, not hallucination in general.

[Multimodal-CoT](/papers/2302.00923) grounds the reasoning itself in the picture. A fine-tuned T5 writes a rationale from the text and frozen ViT features, and a second model answers from the text plus that rationale [arxiv:2302.00923 §4.1]. In a text-only two-stage pilot, the authors read 56% of 50 sampled errors as rationales that mislead the answer step [arxiv:2302.00923 §3.2]. Picture features correct 60.7% of those mistakes [arxiv:2302.00923 §3.3] [fig. 3]. On ScienceQA test the Base model reaches 85.31% with picture features, against 78.57% without [arxiv:2302.00923 tab. 3]. The Large model reaches 90.45%, below the concurrent LLaVA-13B at 90.92% [arxiv:2302.00923 tab. 4]. The method needs annotated rationales. Generated rationales score lower on average, 87.76 against 90.45 [arxiv:2302.00923 tab. 7]. The paper reports single runs without seeds or variance [arxiv:2302.00923 tab. 4].

### Other families from the LLM survey

The LLM survey names several families that the sections above do not cover. The primary papers have no cards here, so only the survey's one-line descriptions are used, and each needs its own source before implementation [arxiv:2401.01313 tab. 1].

- **Knowledge graphs.** RHO grounds dialogue replies in linked entities and relation predicates, then re-ranks candidates with a reasoning model. FLEEK extracts facts, retrieves evidence from curated graphs and the open web, and proposes revisions; the survey says it aims at human graders [arxiv:2401.01313 §3.2].
- **Faithfulness-based losses.** THAM adds a regulariser built from mutual information between the response model and a hallucination model, aimed at copying in video-grounded dialogue. mFACT weights each training summary's loss by a faithfulness score, for cross-lingual summarisation [arxiv:2401.01313 §3.3].
- **Prompt tuning.** UPRISE learns a retriever that picks prompts for a new task. SynTra tunes a system message on a synthetic task and transfers it to summarisation [arxiv:2401.01313 §2.1.3].
- **Self-refinement prompts.** The survey lists ChatProtect, MixAlign, CoNLI and others, which prompt the model to critique and revise its own output [arxiv:2401.01313 §2.1.2]. Chain-of-Verification is the same idea, covered above; see [self-refinement](/concepts/self-refinement).

## When to use which

The survey does not compare these families head to head, so the table rests on the mechanisms and caveats it states. The system cards add evidence for their own settings only.

| Situation | Use | Why |
|---|---|---|
| You train the model and the references contain unsupported facts (scraped pairs, WikiBio-style data) | Data filtering or MR correction | Attacks source–reference divergence directly [arxiv:2202.03629 §3.1, §5.1.2]. Instance-level scores are coarse, so expect to lose some good data. |
| You train the model and data cannot be cleaned | Controllable generation with control codes, or RL with a faithfulness reward | Uses the noisy data while steering towards faithful output [arxiv:2202.03629 §5.2.2] |
| Black-box API, no weights or logits | Retrieval augmentation, Chain-of-Verification, post-editing, multi-LLM debate | Need only prompting and extra calls [arxiv:2202.03629 §13.3]. Cost: more calls per query. |
| Black-box model, several calls per question affordable, short answers that a consensus can check (arithmetic, factual questions) | Multi-agent debate with a few agents and rounds | Text prompts only. Gains were measured on 100-item subsets with one model snapshot [arxiv:2305.14325 tab. 1, tab. 2]. Confident wrong consensus is a stated failure [arxiv:2305.14325 §5]. |
| Open weights, labelled true and false answers, and no budget to retrain | ITI on a sparse set of heads, with the strength tuned on a held-out set | No retraining, but KL and cross-entropy rise and informativeness falls as the strength grows [arxiv:2306.03341 tab. 2, §5.4] |
| Open weights, no budget to retrain, and no labelled data | DoLa, CAD or factual-nucleus decoding at decode time | Training-free, but need access to layer outputs, a second forward pass without context, or the decoder [arxiv:2202.03629 §13.3] [arxiv:2206.04624 tab. 4] |
| Model ignores the provided context in favour of memorized facts | CAD, input augmentation, retrieval | Targets the parametric-knowledge-bias cause [arxiv:2202.03629 §3.2, §13.3] |
| Using top-k or other diversity sampling | Lower randomness in decoding, or uncertainty-aware beam search | Sampling randomness is a listed cause [arxiv:2202.03629 §3.2, §5.2.1]. Costs diversity. |
| Summarization or data-to-text, little tolerance for errors | Post-processing corrector on top of any of the above | Low data needs, model untouched [arxiv:2202.03629 §5.2.3]. Check grammaticality of the edits. |
| Open-domain chit-chat | Be careful when penalizing extrinsic content | Extrinsic additions can be true and useful; tolerance is higher in chit-chat [arxiv:2202.03629 §2.1, §2.2] |
| Numbers matter (dosages, dates) | Add a numeric check such as HERMAN-style quantity re-ranking | Few methods treat numerals specially [arxiv:2202.03629 §7.3.3] |
| Dialogue model that must back factual claims, with tool access and crowd labels available | Tool-query fine-tuning with a groundedness check | Groundedness counts only replies with external claims, so report that denominator [arxiv:2201.08239 §4.1, §6.2] |
| Model must call one of a fixed API set that changes over time, you can fine-tune an open model, and you can generate synthetic request–call pairs | Fine-tune on the API set, with the zero-shot model for deployments that have no retriever | Author-reported lower hallucination than zero-shot GPT-4 on their hub benchmark [arxiv:2305.15334 tab. 1]. Retrieval-trained models score 0 overall without a retriever, and test-time BM25 lowers accuracy [arxiv:2305.15334 tab. 2]. Measure any retriever on your own docs before use. |
| Domain chat where answers need terms newer than the training data, and you can index a trusted source | Keyword-driven retrieval over that source, with the model reading the top sections before answering | Covers terms missing from training data. The authors give examples, not measurements [arxiv:2303.14070 p. 4-7]. Wikipedia reliability is a stated concern [arxiv:2303.14070 p. 3]. Check the released code's licence before reuse; see the card's pitfalls [ChatDoctor](/papers/2303.14070). |
| Question-answering over your own documents, with a retriever already in place | Retrieval with a similarity cut-off, and say so when nothing passes | VirtualTA uses this pattern without measuring its effect [arxiv:2309.10892 §3.2.3, §4.1.3]. Calibrate the cut-off yourself. |
| Hosted LLM that answers from memory, cannot be fine-tuned, and needs answers grounded in a retrievable source | Plug-in evidence consolidation with a utility check and feedback re-prompts, as in LLM-Augmenter | Needs no model access. Author-reported Knowledge F1 gains over ChatGPT on two dialogue tasks, with BM25 retrieval [arxiv:2302.12813 tab. 1, tab. 2]. Stopping rule and threshold are not stated, and the loop may cost two calls per response [arxiv:2302.12813 §6]. |
| Public-facing corrections that must cite credible, current sources | Credibility-ranked retrieval with evidence quoting and source links | Expert ratings favour it over lay notes and plain GPT-4 in one study [arxiv:2403.11169 fig. 2]. English text and images only; thresholds must be recalibrated [arxiv:2403.11169 §Discussion, §Experiments]. |
| Vision-language model describing detail the input cannot resolve | Raise input resolution, or add a global view next to tiles | The authors report fewer detailed-description hallucinations at 448² but give no table [arxiv:2310.03744 §5.2, §3.4]. The MLLM survey reports resolution as the stronger lever in one cited study [arxiv:2306.13549 §2.1]. |
| Vision-language model whose rationales name content the image does not show, and annotated rationales exist | Ground the rationale in image features | Picture features fixed a share of rationale errors in the authors' pilot [arxiv:2302.00923 §3.2, §3.3]. Results are single runs [arxiv:2302.00923 tab. 4]. |
| Open-weight vision-language model that names objects absent from the image, no budget to retrain, and room for one extra forward pass per token | Visual contrastive decoding with the paper's defaults as a start, then tune β on your own POPE-style check | Training-free. Gains are author-reported on three LVLMs and POPE [arxiv:2311.16922 tab. 1, §4.1]. The default β is not the best value in the paper's own MME sweep [arxiv:2311.16922 app. B]. |
| You can fine-tune the generator, can label critique data (GPT-4 in the paper), and want the model to decide when to retrieve and to grade its own support | Self-RAG-style reflection tokens, with the retrieval threshold and critique weights tuned at inference | Author-reported gains over ChatGPT on some tasks, plus a citation-precision knob [arxiv:2310.11511 tab. 2, fig. 3b]. Cost is one continuation per passage per segment [arxiv:2310.11511 §3.3]. Check the card's Paper vs code section before reproducing: [Self-RAG](/papers/2310.11511). |
| Vision-language model asked questions the image cannot answer | Format suffix that allows an "Unanswerable" reply, used in training and evaluation | Abstention on VizWiz rose from 11.1% to 67.8% [arxiv:2310.03744 §4.3] |
| LLM extracts a list of items (entities, fields) and false positives are costly | Yes/no self-verification of each candidate | Drops unconfirmed spans. The gain was measured only for NER on CoNLL2003 [arxiv:2304.10428 tab. 2]. Costs one call per candidate, so budget for it [arxiv:2304.10428 §4.3]. |
| Open-ended LM text where named-entity or fact errors matter more than variety, and you can change the decoder | Factual-nucleus sampling, optionally with factuality-aware continued training on Wikipedia | Lowers entity error on the authors' benchmark. Diversity falls sharply, to 0.33 vs 0.90 for top-p at 530B [arxiv:2206.04624 tab. 5]. The training gain was measured at 530B and the decoding ablations at 1.3B [arxiv:2206.04624 §5.2, §6.3]. |
| You have bad answers but no good ones, and a blank or degenerate reply is acceptable | Gradient-ascent unlearning with a mismatch term and a KL term | Needs only negative examples [arxiv:2310.10683 §2, §4]. Expect nonsense outputs on the forgotten prompts and some on normal prompts without the extra terms [arxiv:2310.10683 §3.2]. |
| You generate training text from labelled inputs with a multimodal model, and the text must match the labels | Parse the generated text back and regenerate on a mismatch with the labels | A consistency check gives a measured gain in one navigation pipeline, but the paper reports no instruction error rate [arxiv:2411.11394 tab. 1, §3.2]. Retry limit and match rule are not stated, so cost per accepted sample is unknown. Label errors pass through unchanged [arxiv:2411.11394 §3.1]. |
| Choosing how retrieved text enters the model | Text concatenation for black-box, fast-changing knowledge; latent or parametric fusion for white-box models with fixed domain knowledge | The RAG survey's qualitative table; it gives no measurements [arxiv:2407.13193 tab. 2, §4.5]. Latent and parametric fusion need white-box access and training [arxiv:2407.13193 §4.5]. |
| Closed API model you cannot retrain, and a wrong answer costs more than a refusal | Pick the variant that abstains more, and measure both accuracy and hallucination rate on your own questions | The card's mini model abstains much more than o4-mini. It shows a lower hallucination rate (0.26 vs 0.75) and slightly lower accuracy (0.22 vs 0.24) [arxiv:2601.03267 §3.7]. Judge the trade-off on your own labels. |
| Closed API model you cannot retrain, with browsing available, and you need fewer errors on open-ended factual questions | Use the newer reasoning tier with browsing on, and measure it on your own question set | Author-reported gains are small on SimpleQA without browsing (0.40 vs 0.46) and large only in the health subset (about 8x) [arxiv:2601.03267 §3.7, §3.10]. The factuality figures appear only in plots. |
| You have not yet chosen a family and want the landscape before reading primary papers | Read the LLM survey's families as a map | Its table lists tasks and models, not results, and its metrics do not compare, so do not rank techniques by it [arxiv:2401.01313 tab. 1, §Pitfalls]. |

## Implementation sketch

A generic pipeline that combines the families. Each stage is optional; the survey treats them as independent.

```text
# Training time (only if you train the model)
for (src, ref) in train_set:
    s = faithfulness_score(src, ref)          # any detection metric
keep pairs with s above a chosen cutoff       # cutoff: not stated, tune it
optionally prepend control code <faithful>/<unfaithful> by score
train; optionally add RL reward = faithfulness metric, or auxiliary tasks

# Inference time
ctx   = src + retrieve(src)                   # retrieval / input augmentation
# or, Self-RAG: retrieve only if the Retrieve token passes a threshold  [arxiv:2310.11511 §3.3]
if no chunk in ctx passes the similarity cut-off:
    return "not in the source material"       # abstain instead of guessing (VirtualTA-style)
draft = decode(model, ctx, low_randomness)    # or DoLa / CAD if internals are available
# vision-language: contrast with a noised-image pass  [arxiv:2311.16922 §3.3]
# optional: shift kept attention heads along a probed truthful direction  [arxiv:2306.03341 §3.3]

# Post-processing
qs      = plan_verification_questions(draft)  # Chain-of-Verification style
answers = [answer(q, ctx) for q in qs]        # answer each without showing the draft
out     = revise(draft, answers)              # or a trained corrector / re-ranker
# or: N copies answer, read each other, revise for a few rounds  [arxiv:2305.14325 §2.1]
# for extracted items: keep a span only if a yes/no check confirms it  [arxiv:2304.10428 §4.3]
check out with a detection metric; reject or regenerate if it fails
```

Where variants differ: data methods change only stage one; architecture, decoding and activation-editing methods change `decode`; corrector and debate methods replace the last block with a trained model or extra calls rather than a single prompt.

## Contradictions

The cards measure different things, so most of their numbers cannot be set against each other. FactualityPrompts scores named-entity errors against Wikipedia and an NLI model, and its human check reports a correlation of 0.81 for the expert labels but 0.47 for majority vote [arxiv:2206.04624 tab. 2]. LLM Unlearning scores hallucination with a BERTScore rule against the wrong answer, and its fluency column is not meaningful for most unlearned rows [arxiv:2310.10683 §8] [tab. 5]. LaMDA's groundedness, ITI's TruthfulQA score, the debate paper's accuracy, and MUSE's expert ratings each use their own judge and denominator. MUSE's expert study reports weighted Cohen's kappa that is only fair for most criteria [arxiv:2403.11169 §Experiments].

The families themselves are classified inconsistently. The LLM survey files ITI under decoding [arxiv:2401.01313 §3.1], while this page places it under activation editing [arxiv:2306.03341 §3.3]. It files TWEAK, a decoding-time re-ranker, under supervised fine-tuning [arxiv:2401.01313 §3.4] [§3.1]. The same survey says its §3 avoids fine-tuning, yet §3.4 is about supervised fine-tuning [arxiv:2401.01313 §3] [§3.4]. Neither is resolved here; use the survey's families as navigation.

The debate paper's own caveat is that agreement among agents can be confidently wrong, and its method does not check correctness [arxiv:2305.14325 §5]. Agreement alone therefore does not establish truth.

The GPT-5 card's SimpleQA and open-ended factuality rates use a model as the grader, and the grader's agreement with humans is 75% [arxiv:2601.03267 §3.7]. Other cards on this page use exact-match, BERTScore or human-labelled measures, so the GPT-5 numbers cannot be set against them. Its rates for the thinking and mini models are not measured against the retrieval systems above, which were tested with different tasks. Neither is resolved here.

Test-time retrieval is reported as helpful in some settings and harmful in another. MUSE and LLM-Augmenter report gains from retrieved evidence [arxiv:2403.11169 §Results] [arxiv:2302.12813 tab. 1]. Gorilla's model trained without documents loses overall accuracy with BM25 retrieval on all three hubs [arxiv:2305.15334 tab. 2]. The tasks differ (matching API calls against a fixed set, and factual dialogue or misinformation replies), so these numbers are not comparable. Neither side is resolved here.

Within two papers, the reported setting and the paper's own evidence do not always match. VCD's default β=0.1 is not the value that scores best in its own MME sweep, where β=0.5 is highest on all three subsets [arxiv:2311.16922 app. B]. Self-RAG's paper and released code give different retrieval and critique rules, so a reproduction has to choose which one to follow [arxiv:2310.11511 app. A.3] [arxiv:2310.11511 §4.3]; the card's Paper vs code section lists them. Neither is resolved here.

The survey's RAG caveat ("degrades when retrieval is poor") is not tested by VirtualTA, which measures nothing [arxiv:2309.10892 §4]. MUSE's ablation shows that retrieval adds to quality against GPT-4 without it [arxiv:2403.11169 §Results], but it does not test poor retrieval either. ChatDoctor's retrieval is not tested for hallucination either: its BERTScore comparison with ChatGPT measures similarity to physician replies, so it cannot be set against the survey's faithfulness numbers [arxiv:2303.14070 tab. 1]. ChatDoctor's paper and its released code disagree on the number of retrieved sections (five vs four) [arxiv:2303.14070 p. 4]. LLaVA-1.5's resolution effect is qualitative, so it cannot be set against the survey's numbers.

Reasoning before answering is not uniformly helpful. The Multimodal-CoT pilot finds that writing the rationale first lowers accuracy for a 200M text-only model, from 81.63% to 69.32% [arxiv:2302.00923 tab. 2]. That is one model size and one task, so it does not test larger models.

Caveats from the survey itself: it calls consistency-based checks "more reliable" without citing evidence, and its §1–§12 cover work up to about 2022, while only §13 (LLMs) was updated in January 2024 [arxiv:2202.03629 §13]. Numbers it reports are second-hand.

## Papers

- [Survey of Hallucination in Natural Language Generation](/papers/2202.03629): taxonomy of causes, metrics and mitigations across summarization, dialogue, QA, data-to-text, MT, vision-language and LLMs.
- [A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models](/papers/2401.01313): taxonomy of 32 LLM techniques in prompt-engineering and model-development families, with a per-technique table; runs no experiments of its own.
- [Challenges and Applications of Large Language Models](/papers/2307.10169): opinionated survey of open LLM problems, with hallucination in its behavior group; runs no experiments of its own.
- [LaMDA: Language Models for Dialog Applications](/papers/2201.08239): dialogue models with quality and safety fine-tuning and tool queries for grounding; author-reported groundedness and safety at 137B, with data not released.
- [FactualityPrompts: Factuality Enhanced Language Models for Open-Ended Text Generation](/papers/2206.04624): factual-nucleus sampling and factuality-aware continued training on Wikipedia; author-reported named-entity error on FactualityPrompts, with a diversity cost.
- [Multimodal Chain-of-Thought Reasoning in Language Models](/papers/2302.00923): two-stage rationale-then-answer model with frozen image features; author-reported ScienceQA accuracy, single runs.
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](/papers/2302.12813): LLM-Augmenter, evidence consolidation and utility feedback around a black-box LLM; author-reported Knowledge F1 gains on dialogue tasks, no released code.
- [ChatDoctor](/papers/2303.14070): LLaMA-7B fine-tuned on patient-doctor dialogue, with a keyword-driven retrieval loop over Wikipedia or a disease table; author-reported BERTScore on iCliniq, no hallucination measurement.
- [GPT-NER: Named Entity Recognition via Large Language Models](/papers/2304.10428): LLM entity extraction with retrieved demonstrations and a yes/no self-verification pass; author-reported F1 on CoNLL2003 and OntoNotes5.0.
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](/papers/2305.14325): copies of one chat model revise answers over rounds; author-reported gains on arithmetic, GSM8K, biographies and MMLU.
- [Gorilla: Large Language Model Connected with Massive APIs](/papers/2305.15334): LLaMA-7B fine-tuned to write API calls, with an optional retrieved-documentation variant; author-reported hallucination and accuracy on APIBench.
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](/papers/2306.03341): steers attention heads along probed truthful directions at decoding time; author-reported TruthfulQA gains on Alpaca and Vicuna, with a KL cost.
- [A Survey on Multimodal Large Language Models](/papers/2306.13549): survey of MLLM design, training and hallucination types; runs no experiments of its own.
- [A Survey of Hallucination in "Large" Foundation Models](/papers/2309.05922): taxonomy of hallucination work across text, image, video and audio, with a 28-row table of detection and mitigation marks; runs no experiments of its own.
- [Large Language Model Unlearning](/papers/2310.10683): gradient ascent on bad answers with a mismatch term and a KL term; author-reported harmful, copyright and hallucination rates on OPT and Llama 2, with degenerate outputs.
- [Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education](/papers/2309.10892): retrieval-based course chatbot with a similarity cut-off and prompt-only abstention; no measurements.
- [Improved Baselines with Visual Instruction Tuning](/papers/2310.03744): LLaVA-1.5; input resolution and format prompts, with a qualitative hallucination claim and one abstention result.
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](/papers/2310.11511): one generator emits retrieval and critique tokens and guides decoding with them; author-reported factuality and citation precision with Llama 2 7B and 13B.
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](/papers/2311.16922): training-free decoding that contrasts the image with a noised copy; author-reported POPE and MME gains on three LVLMs.
- [Customized large language models can outperform Community Notes in correcting misinformation](/papers/2403.11169): MUSE, a credibility-ranked retrieval pipeline that writes source-linked corrections; author-reported expert ratings.
- [Retrieval-Augmented Generation for Natural Language Processing: A Survey](/papers/2407.13193): survey of RAG components and four retrieval fusion types; runs no experiments of its own.
- [InstruGen: Automatic Instruction Generation for Vision-and-Language Navigation Via Large Multimodal Models](/papers/2411.11394): multimodal-model instructions from house-tour routes, with a label-consistency check and regex clean-up; author-reported R2R and RxR success rates.
- [OpenAI GPT-5 System Card](/papers/2601.03267): system card for the GPT-5 family; author-reported SimpleQA hallucination rates, health and factuality results, with no training recipe and most factuality numbers only in figures.