A Survey of Hallucination in "Large" Foundation Models

In plain words

Big AI models trained on huge piles of unlabeled data can write text, make pictures, and answer questions about images or sound. Sometimes they state things that are false or invent details, and they say them just as confidently as true things. That is called hallucination.

This survey collects the research on that problem across four kinds of input: text, images, video and audio. It sorts the existing work by whether it tries to detect hallucinations or to reduce them, and by which tasks, datasets and scores it uses. It also asks whether hallucination is always bad, for example when a model is used to brainstorm, and it lists directions for future work.

It does not build a model or run tests of its own. Read it as a map of the field and a starting list of papers to open next.

TL;DR

A survey of hallucination in large foundation models, covering text, image, video and audio. It defines hallucination as content that departs from factual reality or includes fabricated information §1.2. Its main output is one table of 28 works, each tagged for detection, mitigation, task, dataset and evaluation metric tab. 1. It runs no experiments and reports no new results.

Problem

Foundation models are trained on large unlabeled datasets and then used for many tasks §1.1. The survey states that hallucination is a serious risk for them, especially in journalism, healthcare and law, where factual errors are costly §1.2. It names possible causes: biased training data, no access to current information, and limited understanding of context §1.2. Before this survey, the authors note, the hallucination surveys they cite focused on text and on language models, and did not cover other modalities §1.3.

Method

This is a literature survey. It has no algorithm, equations or training procedure, so there is nothing to implement. What an implementer can take from it is the taxonomy and the table of works. The schema below restates how the survey organises each work; it is not an algorithm from the paper.

# Survey schema, one row per work  [§1.3.1] [§1.3.2] [tab. 1]
# Input:  works, papers on hallucination in foundation models
# Output: rows, one per work, with five axes

def tabulate(works):
    # Start an empty table  [tab. 1]
    rows = []
    for work in works:
        # Assign one of four modalities  [§1.3.2] [fig. 1]
        modality = modality_of(work)
        # Record the five axes for each work  [§1.3.1] [tab. 1]
        row = {
            "modality": modality,
            "detect": is_detection(work),      # yes or no, authors' judgement
            "mitigate": is_mitigation(work),   # yes or no, authors' judgement
            "tasks": tasks_of(work),
            "datasets": datasets_of(work),
            "metric": metric_of(work),
        }
        rows.append(row)
    return rows

• Definition. Hallucination in a foundation model is content that strays from factual reality or includes fabricated information §1.2.
• Modalities. Large foundation models are split into four groups: text, image, video and audio §1.3.2 fig. 1.
• Axes. For each work the survey records five things: detection, mitigation, tasks, datasets and evaluation metrics §1.3.1 tab. 1.
• Text. Covers LLMs §2.1, multilingual LLMs §2.2, and domain-specific LLMs in medicine and law §2.3. The example of a hallucinated answer is shown in fig. 3.
• Image. Covers object hallucination in vision-language models and a contrastive-learning setting §3 fig. 4.
• Video. Covers video captioning, video chat, video inpainting and human insertion into scenes §4 fig. 5.
• Audio. Covers music captioning and audio generation §5.
• Not always harmful. Argues that hallucinated output can help creative work §6.
• Future directions. Automated evaluation metrics, human-AI collaboration, adversarial testing and fine-tuning §7.1; curated knowledge sources for detection and mitigation §7.2.

Works covered tab. 1. Condensed from the source table; "yes" and "no" are the paper's own detect and mitigate marks. Titles are shortened to the first-author or method name, and the bibliography keys are the source's.

Hyperparameters

Not applicable. The survey trains and evaluates no models §1–§7.

Results

Author-reported. The survey runs no experiments, so these are figures it quotes from the works it cites. They are not verified here.

The survey's own tallies are counts of the table, not measurements: 28 works in total, 19 text, 3 image, 4 video and 2 audio tab. 1.

Paper vs code

No official code. The contributions list links github.com/vr25/hallucination-foundation-model-survey and says it will be updated §1.3.1. That repository is a resource list of papers, not an implementation, so it is not recorded as code and was not cloned. The raw metadata names no project page. The GitHub search step was skipped because the paper has no method to implement, and the one repository it links is a paper index.

Pitfalls

• Nothing to implement. The card can support a reading list and a taxonomy, not a method. Do not expect code, equations or hyperparameters.
• Figure cross-reference error. The text in §2 points to the image figure for the LLM example. The LLM example is fig. 3; fig. 4 is the image example.
• Figure 1 and Table 1 disagree. Figure 1 lists wu2023hallucination for images and yu2023deficiency for video. Neither appears in Table 1, because those rows are commented out in the source. Table 1 lists dai2022plausible, which Figure 1 does not fig. 1 tab. 1.
• Broken metric citation. In two rows the metric "attributable to identified sources" is cited to castaldo2007severe tab. 1. In the bibliography that key is a paper about sepsis, so the source of that metric is unverified.
• Ambiguous authorship of the Hallucinator. §3 says "we introduce the Hallucinator", yet the same paragraph cites wu2023hallucination for the Siamese setting it builds on. The card treats it as the cited work's content, not as this survey's method.
• Detection and mitigation are not discussed. The section subheadings for detection techniques, mitigation techniques and evaluation metrics are commented out in the source (§2 to §5). Those axes exist only as table columns tab. 1.
• Detect and mitigate marks have no stated criteria. The paper does not say what counts as "detect" or "mitigate". The marks are the authors' judgement.
• Numbers come from cited works. The figures in Results are quoted from other papers. Check the original before citing them.
• Authors' own claims. "First comprehensive survey across all major modalities" §1.3 is the authors' claim and is not checked here.
• §6 is short. The point that hallucination can help creative work rests on one cited work (goodhallucinate) and a paragraph of discussion.
• Version. This card covers arXiv v1 (2023-09-12). The authors say they will update the linked resources, so later versions may differ.

Lineage

• Related surveys named by the paper: ji2023survey (hallucination in natural language generation) and zhang2023siren (hallucination in LLMs) §1.3. The paper gives these as author-year keys with no arXiv ids.
• Compared against: not applicable.
• Superseded by: not known from this card.