Automated fact-checking

Overview

The problem is that a claim often needs checking against evidence the reader cannot see, and a bare verdict is hard to trust. Crowd notes on X were, in the authors' February 2023 snapshot of the 2403 paper, absent for 88% of suspicious posts and without a highly rated note in the first hour for 93% 2403.11169 §Introduction. Scientific claims raise a different difficulty: checking them needs domain experts and a corpus of specialist literature, and general fact-checking datasets are built on news and Wikipedia 2004.14974 §1 2004.14974 §2.

The shared idea is to let a model do the research step by step: find candidate evidence, pick the passages that matter, and produce a verdict or a reply that links them 2403.11169 §Approach 2004.14974 §5. The shared invariant is that every factual statement in the output is traceable to a source that the system actually retrieved. Both papers evaluate with human judgements or expert labels rather than lexical overlap. The first paper argues that ROUGE cannot judge whether a correction is factual, complete and sourced 2403.11169 §Introduction.

Task framing

Before choosing a route, fix the task. A 2018 survey sorts fact-checking work by three choices: the input (a claim sentence, a knowledge-base triple or a whole document), the output (a binary label, an ordinal or multiclass label, a score, or a label with supporting sentences) and the evidence type (none, metadata, knowledge graph, text, previously checked claims or social-network signals) 1806.07687 §3.1 1806.07687 §3.2 1806.07687 §3.3. Evidence is the axis that decides what a system can output and how it must be evaluated 1806.07687 §1. The survey reports no experiments of its own, so it gives no performance figures 1806.07687 §4. Its FEVER-style pipeline (retrieve documents, select sentences, classify against them) has the same shape as the scientific route below 1806.07687 §5.

A caution from the same survey: a method that assumes the evidence is given, such as an entailment model, does not solve the retrieval step on its own 1806.07687 §5. The survey also notes that most surveyed datasets are small or tied to one source, which limits training 1806.07687 §4. The survey dates from 2018 and predates the LLM routes below, so its method descriptions are historical and cannot say which later approaches were adopted. Card: Automated Fact Checking: Task formulations, methods and future directions.

Variants

There are two published routes, plus the image path of the first, and a measure of the human reader that sits beside them.

Public posts: retrieval-grounded reply

The pipeline is: queries, a search per publisher tier, a time filter, a relevance filter, evidence extraction, and a reply [arxiv:2403.11169 §Approach]. Publisher tiers (High, Medium, Low) come from Media Bias/Fact Check factuality and bias ratings, and the tiers are searched in that order 2403.11169 §Experiments. Only pages published before the post are used, which simulates a correction written at posting time 2403.11169 §Experiments. Author-reported: 12 expert fact-checkers rated replies 8.1/10 overall, against 6.3 for high-helpfulness Community Notes and 5.9 for GPT-4 without retrieval 2403.11169 §Results. Card: Customized large language models can outperform Community Notes in correcting misinformation.

Image posts: text description plus image matching

Each image is first turned into text: a caption, celebrity labels and OCR are combined by the LLM into an "informative image description" 2403.11169 §Approach 2403.11169 §Experiments. The search then also uses reverse image search, and candidate pages are scored on text and on image similarity 2403.11169 §Approach. Removing retrieval or the full system scored 25% and 33% higher, in relative terms, than the versions without retrieval and without vision 2403.11169 §Results.

The released code differs from the paper on this path, notably in how the image captions are revised and how pages are kept. See the card's Paper vs code section before reusing any threshold.

Scientific claims: retrieve, select rationale, label

The task takes a claim and a corpus of abstracts. The system returns the abstracts that support or refute the claim, and for each one a label from SUPPORTS, REFUTES or NOT_ENOUGH_INFO plus the sentences that justify it 2004.14974 §4. The benchmark, SciFact, has 1,409 expert-written claims checked against 5,183 abstracts, split 809 / 300 / 300 for train, dev and test 2004.14974 §3 2004.14974 tab. 2.

The baseline, VeriSci, runs three stages: TF-IDF retrieval of the top three abstracts, RoBERTa-large selection of rationale sentences, and RoBERTa-large label prediction from the selected sentences and the claim 2004.14974 §5 2004.14974 app. A.1. Author-reported open-setting test F1 is 39.5 for sentence selection with label and 46.5 for abstract label with rationale 2004.14974 tab. 4. Pre-training on FEVER before SciFact raised dev label accuracy from 75.7 to 81.9 2004.14974 tab. 3, and the authors report relative gains of 47% and 28% over the FEVER-only model on those two metrics 2004.14974 §6.2. The same table shows that an oracle rationale selector reaches 63.2 on the sentence metric in the open setting, so the authors locate much of the gap in the earlier stages 2004.14974 tab. 4 2004.14974 §6.2.

Card: Fact or Fiction: Verifying Scientific Claims.

Human readers: measuring verification habits

This is not an automated system. It is a 13-item, 5-point self-report scale of "critical thinking in AI use", with subscales for verification, motivation to understand AI and reflection on responsible AI 2512.12413 tab. 1. It fits when the question is how well people check AI output, for example to measure the human side of a deployment. Total Cronbach's α was .88 in the authors' first validation sample (N = 376) 2512.12413 §4.1.2.

The link to checking behaviour rests on one study. In Study 6 (N = 191), participants judged 12 statements through a GPT-4o chatbot, and the scale predicted their veracity accuracy with b = 6.91 and R² = .06, a small share of variance, although the authors' discussion calls it a moderate effect 2512.12413 §7.2 2512.12413 §8. In Study 3 the scale had no reliable link with a cognitive reflection test 2512.12413 §4.2.2. All results are author-reported and self-reported. Card: Understanding Critical Thinking in Generative Artificial Intelligence Use.

When to use which

Reader effect: in a survey of 988 U.S. participants, the correct belief that a post was misleading rose from 4.5 to 4.9, a 10% gain, as reported by the authors 2403.11169 §Results. Source disclosure (human or AI) had no significant effect on trust or sharing 2403.11169 §Results.

Before reusing the scientific numbers, note the spread of the evaluation. Bootstrap standard deviations on the 300-claim test set are 2 to 4 F1 points 2004.14974 tab. 4, and the COVID-19 case study is one annotator judging 36 claims, 23 of them plausible 2004.14974 §6.3.

Implementation sketch

Language-agnostic pseudocode of the text-only core of the first route. Steps 4 and 5 are where the variants differ. The paper's stop rule (two refuting pages) is used here, and the code's item count is noted in the card.

The scientific route follows the three stages in the Variants section. Its pseudocode is on the card, under Method.

# Retrieval-grounded correction, text-only post
# Input:  post text, post time
# Output: reply with source links

def correct_post(post):
    # 1. LLM writes search queries; no query if the post is uninformative
    #    [arxiv:2403.11169 §Approach]
    queries = llm_queries(post)
    refuting_pages = 0
    evidence = []
    # 2. Search publisher tiers from most to least credible
    #    [arxiv:2403.11169 §Experiments]
    for tier in ["High", "Medium", "Low"]:
        pages = search_and_crawl(queries, tier)
        # 3. Keep only pages dated before the post, similar to its text
        #    [arxiv:2403.11169 §Experiments]
        pages = [p for p in pages if p.date < post.time
                 and text_similarity(post, p) >= threshold]
        for page in pages:
            # 4. LLM quotes explicit refutations and context passages
            #    [arxiv:2403.11169 §Approach]
            refutes, context = llm_extract_evidence(post, page)
            evidence += refutes + context
            if refutes:
                refuting_pages += 1
        # 5. Stop after two refuting pages, the paper's rule
        #    [arxiv:2403.11169 §Approach]
        if refuting_pages >= 2:
            break
    # 6. Reply with cited facts; ranking by tier is the paper's intent
    #    [arxiv:2403.11169 §Approach]
    return llm_reply(post, evidence)

Contradictions

No conflicting numbers between the papers. The 2018 survey reports no results of its own, only dataset sizes 1806.07687 §4, and the 2512 paper is a self-report scale, not a checker. The other two results measure different things: the first compares expert ratings of replies with Community Notes and GPT-4 on public posts, and the second reports F1 on a claim-abstract benchmark. The numbers should not be compared across the two routes. Within the first paper, its text and its released code differ on the stop rule, on how evidence is ordered, and on the image path. These differences are listed on the card, which should be read before any reuse. The second paper's text and code also differ, on retrieval details and on label training; see its Paper vs code section.

Papers

• Automated Fact Checking: Task formulations, methods and future directions: a survey that unifies task definitions by input, output and evidence type, and reviews datasets and methods; it reports no experiments of its own.
• Customized large language models can outperform Community Notes in correcting misinformation: an LLM with tiered web retrieval and image matching writes sourced replies to posts, and is rated above crowd notes and plain GPT-4 by experts (author-reported).
• Fact or Fiction: Verifying Scientific Claims: SciFact benchmark and the VeriSci baseline, which retrieves abstracts, selects rationale sentences and predicts a label for each (author-reported open-setting F1 of 39.5 and 46.5).
• Understanding Critical Thinking in Generative Artificial Intelligence Use: Development, Validation, and Correlates of the Critical Thinking in AI Use Scale: a 13-item self-report scale of how people verify and reflect on AI output, validated over six studies; it is a human measure, not a fact-checking system (author-reported).