---
kind: concept
slug: hallucination-detection
title: Hallucination detection
summary: >-
  Flag generated text that contradicts its source or cannot be checked against it. Families: n-gram overlap, IE tuple matching, QA round-trips, NLI entailment, trained classifiers, conditional-vs-unconditional LM scoring and, for LLMs, record lookup, token-uncertainty, sample-consistency or self-evaluation checks. Trade-off: model-based checks catch more but inherit their sub-models' errors.
tags: [hallucination, evaluation, faithfulness, factuality, nlg, llm]
papers: ["2202.03629", "2206.04624", "2209.12356", "2207.05221", "2307.15343", "2309.05922", "2401.01301", "2603.28005", "2104.13346", "2310.09219"]
related: [hallucination-mitigation, chain-of-thought-prompting, self-consistency, uncertainty-estimation, llm-as-a-judge, llm-self-evaluation, calibration, llm-evaluation, benchmark-validity, foundation-model, text-summarization, algorithmic-fairness]
status: draft
updated_at: 2026-11-01
---

## Overview

A generator hallucinates when its output is nonsensical or unfaithful to the source it was given [arxiv:2202.03629 §2]. Detection means scoring or flagging such output automatically, because the usual quality metrics (ROUGE, BLEU, METEOR) do not register it [arxiv:2202.03629 §1]. In MT, for example, hallucinated translations were found not to move BLEU [arxiv:2202.03629 §11.2].

Two distinctions shape every detector:

- **Intrinsic vs extrinsic.** Intrinsic output contradicts the source; extrinsic output can be neither confirmed nor refuted by it, and may still be true [arxiv:2202.03629 §2.1]. Most automatic metrics fold both into a single score [arxiv:2202.03629 §4.3].
- **Faithfulness vs factuality.** Faithfulness is agreement with the source, factuality is agreement with world knowledge [arxiv:2202.03629 §2.3]. A detector checks against a *reference set*, and what that set is depends on the task: the input document for summarization, the table for data-to-text, dialogue history and knowledge for dialogue, world knowledge for generative QA [arxiv:2202.03629 §2.2]. For LLMs the reference spans both the prompt and the training data, so their hallucinations lean extrinsic [arxiv:2202.03629 §13.1].

The shared structure of all variants: fix a reference, compare the output with it through some proxy (n-grams, tuples, answers, entailment, likelihood, agreement between samples), and report a score or a set of flagged spans.

Ten cards cover this page. The text survey catalogues the families, and every finding it reports about other work is the survey's account, so check the primary papers it cites before relying on a specific number. [A Survey of Hallucination in "Large" Foundation Models](/papers/2309.05922) extends the catalogue to image, video and audio models, and its table is the only one here that covers non-text detection. [Gender Biases in LLM-Generated Reference Letters](/papers/2310.09219) uses sentence-level NLI against a source biography as a measurement step in a bias audit, not as a detector under test. Three cards measure detectors on their own data: [Large Legal Fictions](/papers/2401.01301) for LLMs on US case law, [Factuality Enhanced Language Models](/papers/2206.04624) for entity and entailment checks on open-ended text, and [Med-HALT](/papers/2307.15343) for medical LLMs. [News Summarization and Evaluation in the Era of GPT-3](/papers/2209.12356) asks whether automatic metrics agree with human judgments for prompted rather than fine-tuned systems. [FRANK](/papers/2104.13346) labels each sentence of about 2250 CNN/DM and XSum summaries as factual or not, and tags each error with a seven-category typology, so a metric can be scored on each kind of error [arxiv:2104.13346 §2–§3]. [Language Models (Mostly) Know What They Know](/papers/2207.05221) tests whether a model can grade its own sampled answers and learn to predict whether it knows an answer. [Atomic and Holistic LLM Judges for Reference-Grounded Support Labels](/papers/2603.28005) compares four judge prompts that label an answer as supported, partly supported or unsupported against a reference supplied in the input.

## Variants

All families are catalogued in [Survey of Hallucination in Natural Language Generation](/papers/2202.03629). The legal benchmark measures three LLM variants below: record lookup, sample contradiction, and confidence from sample agreement.

### Statistical (n-gram) metrics

These measure overlap or contradiction between the output and the source and/or references [arxiv:2202.03629 §4.1].

- **PARENT** computes n-gram entailment against both the source table and the reference text, combined into an F-score. **PARENT-T** drops the reference and uses only the table [arxiv:2202.03629 §4.1, §10.2].
- **Knowledge F1** is unigram F1 between a dialogue response and the gold knowledge that annotators used, so it needs datasets that record that knowledge. **Rare F1** restricts the count to infrequent words [arxiv:2202.03629 §8.2.3].
- **BVSS** (bag-of-vectors sentence similarity) is used for MT adequacy [arxiv:2202.03629 §11.2.1].

Cheap and model-free, but purely lexical: they miss paraphrase and semantic variation [arxiv:2202.03629 §4.1].

### Information-extraction based

Extract (subject, relation, object) tuples from the output and the reference and flag mismatches; a lighter variant compares only the named entities found by an NER model, which assumes a hallucination changes the entity set [arxiv:2202.03629 §4.2.1, §7.2.1].

### QA based (FEQA, QAGS, QuestEval)

A question-generation model writes question/answer pairs from the output, a QA model answers the same questions from the source, and the score is the agreement between the two answer sets [arxiv:2202.03629 §4.2.2]. For summarization these are reported to correlate substantially better with human faithfulness judgments than n-gram baselines, though the survey gives no numbers [arxiv:2202.03629 §7.2.1]. Data-QuestEval adapts the idea to data-to-text, with QG/QA models trained on synthetic (data, text) pairs and answers compared via BERTScore [arxiv:2202.03629 §10.2].

### NLI based

Score the output by the probability that the source entails it [arxiv:2202.03629 §4.2.3]. Refinements: NLI training data with long premises, sentence-level application (SummaC_Conv), and entailment over dependency arcs to localize the error [arxiv:2202.03629 §4.2.3, §7.2.1]. For data-to-text, running NLI in both directions separates omissions (input ⇒ text) from hallucinations (text ⇒ input) [arxiv:2202.03629 §10.2].

### Named-entity and entailment checks against Wikipedia (FactualityPrompts)

For open-ended text there is no reference answer, so the FactualityPrompts paper uses the Wikipedia article that FEVER annotates for each prompt's claim. The sentence-level reference is the article sentence closest to the output, found by TF-IDF or by sentence embeddings [arxiv:2206.04624 §3.2]. Two metrics follow:

- **NE error:** the share of the output's named entities that do not occur in the reference article [arxiv:2206.04624 §3.3].
- **Entailment ratio:** an MNLI-fine-tuned RoBERTa classifies each (reference sentence, generated claim) pair, and only an entailed label counts as factual [arxiv:2206.04624 §3.3].

On 200 labelled continuations, expert labels correlated with the entailment ratio at 0.81 and with NE error at −0.77. Majority vote over crowd labels gave 0.47 and −0.46 [arxiv:2206.04624 §3.4] [tab. 2]. The reference limits both metrics: a correct claim missing from the article is scored as an error [arxiv:2206.04624 §3.2]. The same paper also proposes decoding and training changes, covered on [hallucination mitigation](/concepts/hallucination-mitigation).

### Trained faithfulness classifiers

Train a task-specific classifier on synthetic data with inserted hallucinations or on human-labelled outputs [arxiv:2202.03629 §4.2.4]. FactCC learns consistency classification jointly with extracting the supporting span and the inconsistent span [arxiv:2202.03629 §7.2.2]. In MT, a token-level detector is trained on synthetic data produced by LM infilling, with a 0/1 label per token [arxiv:2202.03629 §11.2].

### LM based (conditional vs unconditional)

Train one LM on targets alone and one conditioned on the source, force-decode the output with both, and mark a token as hallucinated when the unconditional LM assigns it the lower loss. The score is the share of hallucinated tokens in the output [arxiv:2202.03629 §4.2.5].

### Task-specific detectors

- **Task-oriented dialogue:** slot error rate $\text{SER} = (p+q)/N$ over $N$ slots in the dialogue act, with $p$ missing and $q$ redundant slots; tree accuracy checks the output's structure against the input meaning representation [arxiv:2202.03629 §8.3.1].
- **Data-to-text:** table-record coverage and the ratio of hallucinated entities, using entity recognition plus heuristic alignment [arxiv:2202.03629 §10.2].
- **MT:** an auxiliary evaluation decoder; entropy of averaged attention (hallucinating models focus attention on a single point), compared on clean vs perturbed inputs; SimAlign word alignment, where an unaligned target token counts as hallucinated; Coverage Difference Ratio for under-translation; and detection of "natural" hallucinations from low unique-bigram counts and repeated targets [arxiv:2202.03629 §11.2].
- **Image captioning:** CHAIR, at object level ($\text{CHAIR}_i$) and caption level ($\text{CHAIR}_s$), using the 80 MSCOCO segmentation categories with exact word matching [arxiv:2202.03629 §12.1]. It needs a predefined object list [arxiv:2202.03629 §12.3].

### LLM detectors

- **Reference-dependent (benchmarks):** multiple-choice or cloze QA, long-tail question sets selected by frequency, popularity, recency or domain, adversarial sets such as TruthfulQA (817 questions in 38 categories), and automated pipelines that generate a query, retrieve evidence and verify with NLI or an LLM [arxiv:2202.03629 §9.3, §13.2.1].
- **Reference-free, uncertainty-based:** pool token probabilities to word, sentence or passage level (average, max or min, normalized product, max or mean entropy). Requires logits [arxiv:2202.03629 §13.2.2].
- **Reference-free, consistency-based:** sample several completions and measure how much they agree, using BLEU, BERTScore, n-gram or NER overlap, NLI or an LLM judge. Works black-box [arxiv:2202.03629 §13.2.2].
- **Record lookup (legal benchmark):** when a database holds the ground truth, such as court, citation or year in case metadata, compare the model's answer with the record at temperature 0. Non-matching answers that are not abstentions count as hallucinations [arxiv:2401.01301 §4.2] [eq. 1]. For GPT-4 on the court-identity task, zero-shot rates were 0.000 for the Supreme Court and 0.829 for District Courts [arxiv:2401.01301 tab. 2].
- **Sample contradiction (legal benchmark):** draw two temperature-1 answers and have a GPT-4 judge label whether they contradict. Any contradiction shows that at least one answer is wrong, so the rate is a lower bound, not an estimate [arxiv:2401.01301 §4.3] [eq. 2]. Agreement shows nothing about correctness. The paper adapts the self-consistency check of Mündler et al. [arxiv:2401.01301 §4.3]; see also [self-consistency](/concepts/self-consistency).
- **Confidence from sample agreement (legal benchmark):** take the share of samples that match the greedy answer as the confidence and measure its calibration against correctness with expected calibration error. The table note gives ten samples per query, though the paper's code uses eight for PaLM 2 [arxiv:2401.01301 app. D.1] [tab. 6]. Treat the result as a diagnostic: the paper found the scores overconfident for every model (see Known failure modes).
- **Self-evaluation, P(True) (Kadavath et al.):** the model writes an answer at temperature 1, then is asked whether that answer is True or False. The score is the probability it puts on "True" [arxiv:2207.05221 §3.2] [§4.1]. The comparison variant generates five samples for the question, judges one, and shows the other four in the prompt. The paper reports this as the stronger form with few-shot prompting [arxiv:2207.05221 §4.2] [fig. 11]. Reading the score needs token probabilities, so it has the same access requirement as the uncertainty methods.
- **Trained P(IK) probe (Kadavath et al.):** a value head on the model reads the last input token and is trained to predict whether the model knows the answer. Its labels come from 30 sampled answers per question, each marked correct or not [arxiv:2207.05221 §5] [app. A.6]. Trained on TriviaQA, it reaches AUROC 0.864 on held-out TriviaQA [arxiv:2207.05221 tab. 1]. It needs the weights and a training run. The models are the authors' own, and no weights or code are linked [arxiv:2207.05221 §1.2] [§7].

### Medical benchmark (Med-HALT)

Med-HALT is a set of seven test sets for medical LLMs, built from licensing and entrance exam items and from PubMed records [arxiv:2307.15343 §1.1] [§2]. Three reasoning tests probe a multiple-choice item:

- **False confidence:** the prompt includes one suggested answer, which may be wrong, and the model must judge it [arxiv:2307.15343 §3.1].
- **None of the above:** the correct option is replaced by a "none of the above" choice [arxiv:2307.15343 §3.1].
- **Fake questions:** the question is invented, and the model should recognise that it cannot be answered [arxiv:2307.15343 §3.1].

Four memory tests give one PubMed field and ask for another, such as the title from a PubMed ID, and allow the answer that the paper does not exist [arxiv:2307.15343 §3.2]. The paper defines a score of +1 for a correct answer and −0.25 for a wrong one, averaged over items [arxiv:2307.15343 §5.3] [eq. 1]. Author-reported: Llama-2 70B has the highest false-confidence accuracy (42.21%), and Falcon 40B reaches 99.89% on fake questions [arxiv:2307.15343 tab. 2]. The paper's headline that open models beat the OpenAI models on every task does not hold in its own tables. For example, GPT-3.5 leads Falcon 40B on false-confidence accuracy, 34.15% against 18.66% [arxiv:2307.15343 tab. 2].

### Multimodal detectors (foundation-model survey)

The foundation-model survey tabulates 28 works and marks 15 as detection methods: 14 text and one image work [arxiv:2309.05922 tab. 1]. Among the text ones, the table lists SelfCheckGPT, HaluEval, Chain of Knowledge, the work on the internal state of an LLM, and a citation check for LLMs, among others [arxiv:2309.05922 tab. 1]. These are named as the survey names them. Their methods and results are not described on this page, and the survey gives no numbers for them.

- **Image (VQA):** the one image detector in the table is listed as a VQA detector trained on M-HalDetect, a dataset the survey says has 16,000 fine-grained VQA annotations [arxiv:2309.05922 tab. 1] [§3]. CHAIR, above, appears in the table as the metric of one object-hallucination image work and the dataset of another; neither is marked as detection [arxiv:2309.05922 tab. 1].
- **Video and audio:** the table marks no video or audio work as detection. All six entries (four video, two audio) are marked mitigation-only, and none is a detector [arxiv:2309.05922 tab. 1]. This page has no tested detector for those modalities.

The survey's "detect" and "mitigate" marks come with no stated criteria. They are the authors' judgement, not measured results [arxiv:2309.05922 tab. 1].

### Reference-grounded LLM judges (support labels)

The reference is in the prompt and the judge returns a three-way support label. The paper compares four single-call designs [arxiv:2603.28005 §3.2]:

- **Candidate-side atomic (C):** the judge splits the answer into claims and checks each one against the reference.
- **Holistic rubric (H):** a four-point checklist (correctness, completeness, unsupported detail, style), told to skip claim-by-claim splitting.
- **Reference-side atomic (R):** the judge splits the reference into claims and checks whether the answer covers each one.
- **Bidirectional (B):** both decompositions in one call, returning six claim lists.

On the constructed ASQA labels, H beat C for all four judges, by 7.25 to 15.5 points, and used fewer tokens [arxiv:2603.28005 §4.1] [tab. 2]. R was the most accurate ASQA design for each judge it ran with, at 0.975 for Opus-4.6, and cost 28-29% more tokens than H [arxiv:2603.28005 §4.2] [tab. 2]. On TruthfulQA-style labels (a supported reference against a wrong answer), C was at least as accurate as H for every judge and slightly cheaper per call [arxiv:2603.28005 §4.1] [tab. 2]. These are author-reported results on labels the authors built, so they show which design fits which label, not which design detects hallucination in general.

### Sentence-level NLI in a bias audit (reference letters)

The check takes the source biography as the premise and each sentence of the generated letter as the hypothesis. A sentence whose top NLI label is not entailment is flagged as hallucinated. The NLI model is RoBERTa-large, fine-tuned on SNLI, MNLI, FEVER-NLI and ANLI R1 to R3 [arxiv:2310.09219 §3.3] [§4.4.1]. The paper does not use the flags to detect errors in the letters. It uses them to split each letter into a hallucinated part and a rest, then compares the style scores of the two parts [arxiv:2310.09219 §3.3].

Two results come from this split. Hallucinated sentences show the same male-favouring tone gaps as whole letters. For ChatGPT, the tests on the hallucinated part were significant for male formality and positivity and for female agency, and for Alpaca on female formality, male positivity and female agency [arxiv:2310.09219 tab. 6]. The paper calls this bias amplification [arxiv:2310.09219 §3.3]. The card records no accuracy check on the NLI flags, so the flags are an operational definition, not a validated detector.

### Meta-evaluation benchmark (FRANK)

FRANK is a labelled benchmark for scoring factuality metrics, not a detector. Crowd workers marked each sentence of about 2250 summaries from nine systems as factual or not, and gave each wrong sentence one or more labels from a typology of seven error types: predicate, entity, circumstance, coreference, discourse link, out-of-article content and grammar [arxiv:2104.13346 §2] [tab. 1]. Summary-level scores are means of the sentence labels [arxiv:2104.13346 §5.1].

Agreement is measured as a partial correlation with the human score, controlling for the summarising system [arxiv:2104.13346 §5.2]. Author-reported all-data results are weak. The best Pearson is 0.27 for BERTScore P and the best Spearman is 0.30 for FactCC [arxiv:2104.13346 tab. 2]. The two QA metrics, FEQA and QAGS, are near zero on all data [tab. 2]. Ablating one error group at a time shows which errors a metric's correlation depends on [arxiv:2104.13346 §5.4]. The paper's text describes a control for both system and dataset. The [card](/papers/2104.13346) records where the released script differs, so check the control before reusing a number.

### Human evaluation

Either rating on a scale or comparison against baselines or references. Annotation vocabularies differ widely between papers, and only a few rate intrinsic and extrinsic hallucination separately [arxiv:2202.03629 §4.3]. Automatic metrics are meta-evaluated against such judgments in FRANK and TRUE [arxiv:2202.03629 §4].

The GPT-3 summarization study uses a best-and-worst preference design: three annotators pick the most and least preferred of three systems per article, and majority vote decides [arxiv:2209.12356 §3.1] [§3.2]. Annotators were given no definition of quality [arxiv:2209.12356 §3.1]. The design measures overall preference, not hallucination alone, so it is a check on the metrics rather than a detector in itself.

## When to use which

The survey compares families qualitatively and runs no head-to-head experiments. The legal benchmark measures its variants but does not compare them with each other on the same outputs. The table below therefore states fit and known weaknesses, not measured winners.

| Situation | Use | Why |
|---|---|---|
| Structured source (table, MR), cheap check needed | PARENT / PARENT-T, record coverage, SER | Source fields are explicit, so lexical or slot matching is meaningful [arxiv:2202.03629 §10.2, §8.3.1] |
| Knowledge-grounded dialogue with labelled knowledge | Knowledge F1 | Needs the gold knowledge recorded during data collection [arxiv:2202.03629 §8.2.3] |
| Summarization, paraphrase-heavy outputs | QA-based or sentence-level NLI | Lexical metrics miss semantic variation; QA metrics reported to track human judgments better [arxiv:2202.03629 §4.1, §7.2.1] |
| Need the position of the error | Dependency-arc NLI, token-level classifiers, LM-based scoring, alignment (MT) | QA-based and sentence-level NLI scores cannot localize spans [arxiv:2202.03629 §4.2.3, §4.2.5, §11.2] |
| Labelled or synthesizable in-domain data | Trained classifier (e.g. FactCC-style) | Task-specific training with synthetic hallucinations [arxiv:2202.03629 §4.2.4, §7.2.2] |
| Separating omissions from additions | Two-direction NLI | Each direction tests one failure [arxiv:2202.03629 §10.2] |
| Object hallucination in captions | CHAIR | Needs a fixed object vocabulary [arxiv:2202.03629 §12.1, §12.3] |
| LLM with logit access, latency-sensitive | Uncertainty-based pooling | One pass, no extra samples [arxiv:2202.03629 §13.2.2] |
| LLM behind an API (no logits) | Consistency across samples | Black-box; costs several extra generations per query [arxiv:2202.03629 §13.2.2] |
| Open-domain factual claims, no source document | Retrieve-then-verify pipeline | Reference is world knowledge, so evidence must be fetched [arxiv:2202.03629 §13.2.1] |
| Answer maps to a field in an authoritative database | Record lookup | Exact comparison with the record; no judge model and no extra samples [arxiv:2401.01301 §4.2] |
| LLM behind an API, no record or source, budget for extra calls | Sample contradiction with a judge | Finds some errors without a reference, but only as a floor, and the judge must be validated [arxiv:2401.01301 §4.3] [app. C] |
| LLM with logit access, one extra call per answer, short factual answers | P(True) on one sample, with a few-shot prompt | Adds one prompt per answer; zero-shot P(True) is poorly calibrated [arxiv:2207.05221 §4.1] [app. A.5] |
| LLM with logit access, budget for five samples per question | P(True) with comparison samples | Reported as the stronger form with few-shot prompting [arxiv:2207.05221 §4.2] [fig. 11] |
| Own model weights and labelled samples in the target domain | Trained P(IK) probe | AUROC 0.864 in-distribution on TriviaQA, but transfer is partial, so retrain per domain [arxiv:2207.05221 tab. 1] [§5.2] |
| Named entities in open-ended text, reference article available | NE error and entailment ratio | Needs the reference article plus NER and NLI models; validated on a small labelled set [arxiv:2206.04624 §3.3] [§3.4] |
| Medical LLM, probing false confidence or invented sources | Med-HALT-style tests (suggested wrong answer, fake questions, PubMed recall) | Each test targets one failure, but check the scoring code and the dataset licence first [arxiv:2307.15343 §3.1] [§3.2] |
| Grouped audit of invented details, source text available, detector accuracy not needed | Sentence-level NLI against the source, then compare style scores of flagged and unflagged sentences | Shows where a generator's additions carry a group gap; the flags were not validated, so check them on labelled data before reporting rates [arxiv:2310.09219 §3.3] [tab. 6] |
| Image, video or audio model, need a detector for a non-text output | Survey as a map only; text or image detectors from the table | Only one image work is marked as detection and none for video or audio, so no tested option exists here [arxiv:2309.05922 tab. 1] |
| Reference supplied, label must catch omissions (completeness) | Holistic rubric (H) or reference-side atomic (R) | Candidate-side claims do not list what the answer leaves out [arxiv:2603.28005 §1]; R is most accurate on ASQA at 28-29% more tokens than H [§4.2] [tab. 2] |
| Reference supplied, label targets wrong statements, cost per call matters | Candidate-side atomic (C) | On TruthfulQA-style labels C was at least as accurate as H and slightly cheaper for every judge [arxiv:2603.28005 §4.1] [tab. 2] |
| Judging a prompted (not fine-tuned) summariser | Human best-and-worst study alongside metrics | Automatic metrics ranked the prompted outputs lower than the human raters did [arxiv:2209.12356 §3.2] [§4] |
| Choosing a summary factuality metric to adopt | Re-score it on sentence-level labels by error type, not only on the all-data correlation | All-data correlations are weak, and they change with the dataset, e.g. FactCC Pearson 0.36 on CNN/DM and 0.07 on XSum [arxiv:2104.13346 tab. 2] [§5.4] |

## Implementation sketch

Common core, with the comparison step swapped per family:

```text
detect(output, reference):
    units = split(output)                     # tokens, sentences, tuples or claims
    for u in units:
        s[u] = compare(u, reference)          # family-specific, see below
    flagged = {u : s[u] crosses threshold}    # threshold: tune on labelled data
    return aggregate(s), flagged

compare variants:
    n-gram:       overlap / entailment of n-grams with reference
    IE:           tuples(u) vs tuples(reference); mismatch => flag
    QA:           q,a = QG(u); a' = QA(q, reference); sim(a, a')
    NLI:          P(entail | premise=reference, hypothesis=u)
    NE:           entities(u) not in reference article => flag
    LM-based:     loss_uncond(u) < loss_cond(u | source) => hallucinated token
    record:       u != record[field]                     # abstentions excluded
    uncertainty:  pool(token probs of u)                 # needs logits
    consistency:  agreement(u, resample(prompt) x k)     # black-box, k extra calls
    contradiction: judge(u, s) for s in resample(prompt) # any True => flag
```

Fix the reference set before choosing `compare` [arxiv:2202.03629 §2.2]. Log which sub-models (NER, QG, QA, NLI) were used, since scores from different sub-models are not comparable and their errors carry into the metric [arxiv:2202.03629 §4.2].

Known failure modes to guard against:

- An off-the-shelf NLI model can mark an output that adds an unsupported detail as entailed or neutral, and it transfers poorly to long premises; apply it per sentence or use long-premise NLI data [arxiv:2202.03629 §4.2.3, §7.2.1].
- Penalizing everything unsupported also penalizes correct extrinsic content, which matters most for chit-chat [arxiv:2202.03629 §2.1, §2.2].
- Fact-verification models are reported to be vulnerable to adversarial inputs and to break on negations, numbers and comparatives, the small edits that flip whether a claim is true [arxiv:2202.03629 §6.1]. Test detectors on those cases, since a wrong number is among the most harmful errors [arxiv:2202.03629 §6.2].
- **Agreement is not correctness.** A model that is consistently wrong passes a sample-agreement check, so sample-based rates are floors [arxiv:2401.01301 §4.3].
- **Judge decides the contradiction result.** On 100 queries, GPT-4 agreed with two human coders at κ 0.87 and 0.77, and the coders agreed with each other at κ 0.79. The judge's temperature and few-shot count are not stated in the text [arxiv:2401.01301 app. C]. Validate any judge the same way on your own data.
- **Agreement confidence is overconfident.** Pooled ECE was 0.057 for PaLM 2, 0.099 for GPT-3.5, 0.190 for GPT-4 and 0.421 for Llama 2, and every model's error was positive [arxiv:2401.01301 §5.3]. Platt scaling lowered these to 0.033–0.051, but the scaled curves stayed poor between 0.2 and 0.4 [arxiv:2401.01301 app. D.2]. Use agreement as a relative signal, and calibrate on labelled data before trusting a threshold.
- **Premises are accepted.** Detectors that check only the answer can pass answers that accept a false premise. On the false-dissent premise, the legal benchmark's zero-shot hallucination rate for PaLM 2 was 0.990 on Supreme Court cases, while Llama 2 scored 0.000 [arxiv:2401.01301 tab. 5].
- **Refusals distort rates.** The legal benchmark counts abstentions as non-hallucinations [arxiv:2401.01301 §4.2], and in the false-premise test a refusal counts as correct. A low rate can therefore hide over-refusal. Report the abstention rate next to the detection rate [arxiv:2401.01301 §5.2].
- **Self-judged correctness is prompt-sensitive.** The P(True) prompt was the first one the authors tried and was not tuned. Zero-shot P(True) is poorly calibrated and sits near 50% for typical samples [arxiv:2207.05221 §4.1] [app. A.5]. Use a few-shot prompt and check the score on labelled data from your own domain.
- **The P(IK) probe transfers only partly.** Trained on TriviaQA alone, its AUROC was 0.606 on LAMBADA and 0.624 on GSM8k [arxiv:2207.05221 tab. 1]. Calibration on other tasks is poor and often underconfident [fig. 13]. Changing the few-shot format also changes the score, because the training labels were produced under the original format [arxiv:2207.05221 app. A.6] [app. A.7].
- **Context can raise the score.** Prepending a relevant Wikipedia article raised P(IK) on an obscure question from 18% to 78% at 52B [arxiv:2207.05221 §5.3]. The paper reads this as the signal tracking what the model can find in its context, not only what it memorised. It did not test that reading directly [§5.3], so a high P(IK) on a retrieved-context prompt does not prove the model knows the answer.
- **Named-entity checks depend on the reference.** A correct claim absent from the reference article is flagged, and the NE and entailment metrics were validated on only 200 labelled continuations [arxiv:2206.04624 §3.2] [§3.4]. Re-validate on your own domain before setting a threshold.
- **Truncated premises produce false flags.** In the bias audit, the source biography is the NLI premise and the released code cuts each pair to 256 tokens. Facts past the cut cannot be entailed, so they are flagged. The paper does not state this limit, and its hallucination results depend on it. See the [card](/papers/2310.09219), Paper vs code. Check the premise length against your model's limit before reading flag rates.
- **Empty inputs are scored as content.** A letter with no flagged sentence has an empty hallucinated part, and the released code still sends it to the style classifiers, which score a stray ".". Skip empty parts before any scoring, or the statistics include spurious observations. The [card](/papers/2310.09219) lists the code lines.
- **Keyword scoring inflates fake-question results.** The paper gives no matching rule for fake questions. The public code accepts any answer containing a term from a fixed list and never compares it to the label, so the 99.89% fake-question accuracy is an upper bound [arxiv:2307.15343 tab. 2]. See the [card](/papers/2307.15343) for the code details.
- **Benchmark data may not be usable commercially.** The project page restricts the Med-HALT dataset to research and non-commercial use. The paper's CC BY licence covers the paper, not the data [arxiv:2307.15343]. Check the data licence before building on the items.
- **Coverage labels are not factuality labels.** The ASQA and QAMPARI partial rows are built from the reference's own structure, so an answer can contain only accurate claims and still carry a partial label [arxiv:2603.28005 §3.1]. The paper calls these construction labels and says its accuracies measure agreement with a strict completeness standard [§3.1] [§8]. Do not read them as factuality accuracy.
- **Candidate-side decomposition misses omissions.** Splitting the answer checks what it says, not what it leaves out [arxiv:2603.28005 §1]. Where the label rewards completeness, check the reference side or a holistic rubric [§4.1].
- **Metrics validated on fine-tuned outputs.** FactCC and DAE were trained with reference summaries as positives [arxiv:2209.12356 §4.2]. Read their factuality scores on prompted outputs with that in mind.
- **Metric agreement is weak and dataset-dependent.** On FRANK the best all-data correlations are 0.27 (Pearson) and 0.30 (Spearman), and FactCC's Pearson falls from 0.36 on CNN/DM to 0.07 on XSum [arxiv:2104.13346 tab. 2]. Check each metric on labels from your own domain before setting a threshold.

## Contradictions

No direct conflict between the survey and the legal benchmark. The survey calls consistency-based LLM detection more reliable than uncertainty-based detection, without citing evidence [arxiv:2202.03629 §13.2.2]. The legal benchmark tests only consistency-based signals, and reports that agreement-based confidence is overconfident for every model it studied [arxiv:2401.01301 §5.3]. Those results concern different things (detector reliability and confidence calibration), so neither settles the comparison.

Two further results point in different directions, and this page does not resolve either:

- **QA metrics versus n-gram baselines.** The survey reports that QA-based faithfulness metrics correlate substantially better with human judgments than n-gram baselines, without giving numbers [arxiv:2202.03629 §7.2.1]. On FRANK's all-data partial correlations, FEQA (Pearson 0.00) and QAGS (Pearson 0.06) fall below BLEU (0.10), METEOR (0.14) and ROUGE-1 (0.14) [arxiv:2104.13346 tab. 2]. The two sources use different data and metric versions, and this page does not resolve the gap.

- **Which judge design wins on ASQA.** On the constructed labels, H beats C for all three primary judges. On one author's 60 rows, which the paper says had no inter-annotator agreement, C beats H for all three judges (54 vs 52, 54 vs 53, 55 vs 53 correct), and R is last [arxiv:2603.28005 §6.1] [tab. 4]. The paper's own ranking reverses with the label standard, so decide what the label should mean before choosing a design [§7].

- **Which human labels to trust.** In FactualityPrompts, the entailment ratio correlated at 0.81 with expert labels but at 0.47 with crowd majority vote [arxiv:2206.04624 tab. 2]. The paper reports both and does not say which is the right reference, so pick the labelling source for your use and validate it.
- **Preference versus metrics.** On CNN, annotators chose the prompted GPT-3 summary as best for 58% of articles against 36% for the fine-tuned BRIO [arxiv:2209.12356 tab. 3]. The reference-based and reference-free metrics scored the GPT-3 summaries lower [arxiv:2209.12356 §4]. The authors conclude that these metrics cannot reliably judge prompted systems. The study measures preference and metric agreement, not factual error alone, so it does not settle whether the metrics detect hallucination.
- **Confidence calibration.** Kadavath et al. report that models are well calibrated on letter-labelled multiple-choice questions, and that calibration improves with size and with few-shot prompting [arxiv:2207.05221 §2]. Dahl et al. report overconfident confidence scores on legal questions, with GPT-4 at ECE 0.190 [arxiv:2401.01301 §5.3]. The two use different confidence sources (option probabilities against sample agreement) and different tasks, so their results cannot be combined into one claim about model calibration.

Within the survey, the $\text{CHAIR}_i$ formula's denominator (objects in the ground truth) does not match the prose description (objects mentioned in the generated caption); consult Rohrbach et al.'s original definition [arxiv:2202.03629 §12.1].

## Papers

- [Survey of Hallucination in Natural Language Generation](/papers/2202.03629): taxonomy (intrinsic/extrinsic, faithfulness/factuality) and catalogue of detection metrics across summarization, dialogue, QA, data-to-text, MT, vision-language and LLMs.
- [A Survey of Hallucination in "Large" Foundation Models](/papers/2309.05922): tabulates 28 works across text, image, video and audio, each tagged for detection and mitigation. Runs no experiments and reports no new results.
- [Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models](/papers/2401.01301): measures record lookup, sample contradiction and sample-agreement confidence for four LLMs on US federal case questions, with a GPT-4 judge and PaLM 2 among the models studied.
- [Factuality Enhanced Language Models for Open-Ended Text Generation](/papers/2206.04624): NE error and MNLI entailment against Wikipedia on FactualityPrompts. The paper also proposes decoding and training changes.
- [News Summarization and Evaluation in the Era of GPT-3](/papers/2209.12356): human best-and-worst study and metric comparison for prompted news summaries.
- [Med-HALT: Medical Domain Hallucination Test for Large Language Models](/papers/2307.15343): medical benchmark with reasoning tests (false confidence, none of the above, fake questions) and PubMed memory tests.
- [Language Models (Mostly) Know What They Know](/papers/2207.05221): P(True) self-evaluation of sampled answers, and a trained P(IK) value head, tested on TriviaQA and on other tasks.
- [Atomic and Holistic LLM Judges for Reference-Grounded Support Labels: A Prompt-Controlled Comparison](/papers/2603.28005): four single-call judge designs for three-way support labels against a reference, on constructed labels from TruthfulQA, ASQA and QAMPARI, with one author's labels as a check.
- [Gender Biases in LLM-Generated Reference Letters](/papers/2310.09219): measures gender gaps in ChatGPT and Alpaca reference letters; flags unsupported sentences with a biography-premise NLI check and tests the style scores of the flagged part.
- [Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics](/papers/2104.13346): sentence-level factuality labels with a seven-category error typology for about 2250 CNN/DM and XSum summaries, used to score n-gram, embedding and factuality metrics against human judgement.