Customized large language models can outperform Community Notes in correcting misinformation

In plain words

False claims spread on social media faster than people can check them. A volunteer-written note under a post can help, but such notes often show up hours later, and many posts never get one. This paper asks whether a language model, given live web search, can write a useful correction fast enough to matter.

The system works like a careful research assistant. It reads the post, turns it into a few search questions, and looks up pages. It prefers pages from news and science outlets that are rated as accurate and not strongly slanted. From those pages it pulls the sentences that contradict the post, and sentences that add context. Then it writes a short reply that says what is wrong or misleading and links the sources. Posts with pictures get a text description of the image first, including any names and any words visible in it.

The authors compare these replies with human volunteer notes and with the same language model used without search. Expert fact-checkers preferred the system's replies, and in a survey of readers the corrections made people better at spotting the misleading post. All the results come from the authors' own studies.

TL;DR

MUSE is a training-free pipeline around a prompted LLM. It writes search queries, filters web pages by publisher rating and text or image similarity, extracts refuting and contextual evidence, and writes a reply with source links §Approach fig. 1. Author-reported: 12 fact-checking experts rated its replies 8.1/10 overall against 6.3 for high-helpfulness Community Notes and 5.9 for GPT-4 alone (n=232 responses each) fig. 2 §Results. A survey of 988 U.S. participants found a 10% gain in correct belief that a post was misleading §Results. Read the Paper vs code section before reusing any threshold or stop rule.

Problem

Misinformation is hard to correct on social media. Posts can be multimodal, claims can be partly true, and the corrections must be timely, evidence-based and politically even-handed §Introduction. X's Community Notes is a widely used crowd system with good explanations, but in the authors' February 2023 snapshot 88% of suspicious posts had no note and 93% had no highly rated note in the first hour §Introduction. The authors also argue that static benchmarks can be contaminated by training data and that lexical metrics such as ROUGE cannot judge whether a correction is factual, complete and cites credible sources §Introduction. The authors name these limits: English text and images only, no video, one platform, and dependence on relevant, credible evidence being available online §Discussion.

Method

MUSE has three modules: a prompted LLM for queries and replies, a web retriever with credibility-aware ranking, and a multimodal integrator for image posts §Approach. There are no numbered equations in the paper, and the sections are unnumbered, so citations below name the section. The steps for a text-only post are:

1. Queries. The LLM writes search queries from the post. The paper says "different queries"; the code asks for three numbered queries, and answers "NONE" when the post is uninformative §Approach prompt_query_generation_unimodal.txt [supp. fig. prompt-query-generation].
2. Search by publisher tier. Each query is sent to a search engine. Results are grouped into three publisher tiers, High, Medium and Low, derived from Media Bias/Fact Check factuality and bias ratings. The tiers are searched in that order §Approach §Experiments main.py.
3. Pre-filter. Only the top results per query are kept, then their pages are crawled main.py main.py.
4. Time filter. Only pages published before the post's time are kept. This simulates a correction written right after the post appeared main.py §Experiments.
5. Relevance filter. Each page is scored against the post text with a sentence-embedding model, and pages above a threshold are kept §Experiments main.py.
6. Evidence extraction. The LLM is given one page at a time and quotes at most two paragraphs that explicitly refute the post and at most two that implicitly refute it, meaning context that shows the fuller picture §Approach prompt_evidence_extraction.txt.
7. Stop rule. The search stops once enough refutation is found. The paper says two pages that refute the claim; the code counts three refuting items §Approach main.py.
8. Reply. The LLM writes a short reply with the refuting and context facts, each with its URL and date. Facts are meant to be ranked by publisher tier and by relevance §Approach [supp. fig. prompt-response-generation]. If no page refutes the post, the code asks for a reply with no facts main.py.

For a post with images, the paper turns each image into text first. Image captioning output, celebrity labels and OCR are combined into an "informative image description" by the LLM §Approach §Experiments [supp. fig. prompt-image2text]. The web search then also uses reverse image search, and each candidate page is scored on text and on image similarity §Approach fig. 1. The code keeps at most two pages per tier, the best text match and the best image match main.py. A second, text-based search runs if fewer than three refutations are found main.py.

Variants used in the experiments: GPT-4 alone (no retrieval, no vision), a version without vision (MUSE minus vision), and a version without retrieval (MUSE minus retrieval). A Llama 3 70B version and a GPT-5.5 version are reported in the discussion §Experiments §Discussion.

Pseudocode for the text-only path. The paper has no algorithm box, so this is reconstructed from the code and the prose:

# MUSE, text-only post (reconstructed from code, no paper algorithm)
# Input:  post (text, author, time); no images
# Output: reply text with source links
# [§Approach] [code: model/main.py#L755-L1040]

def correct_text_post(post, top_k):
    # 1. Write 3 search queries; NONE if the post is uninformative
    #    [code: model/main.py#L178-L241]
    queries = llm_queries(post)

    refute_evidence = []
    context_evidence = []
    # 2. Search publisher tiers from most to least credible
    #    [code: model/main.py#L792-L793]
    for tier in ["High", "Medium", "Low"]:
        # 3. Keep the top_k results per query  [code: model/main.py#L799]
        links = search_top_k(queries, tier, top_k)
        # 4. Crawl pages; keep English pages with 100+ words and a date,
        #    published strictly before the post  [code: model/main.py#L445]
        pages = crawl_and_filter_by_time(links, post.time)
        # 5. Keep up to 3 pages whose text score is at least 90
        #    [code: model/main.py#L572-L596]
        pages = filter_by_text_score(post.text, pages, 3, 90)
        for page in pages:
            # 6. Quote up to 2 explicit and 2 implicit refutations
            #    [code: model/main.py#L599-L706]
            refutes, contexts = extract_evidence(post, page)
            refute_evidence += refutes
            context_evidence += contexts
        # 7. Stop after 3 refuting items  [code: model/main.py#L831-L833]
        if len(refute_evidence) >= 3:
            break

    # 8. No refutation: retry once with top_k = 10
    #    [code: model/main.py#L835-L847]
    if not refute_evidence and top_k == 3:
        return correct_text_post(post, 10)

    # 9. Reply with facts only when refutations exist; context is then
    #    included, otherwise the reply uses no facts
    #    [code: model/main.py#L709-L748]
    if refute_evidence:
        return llm_reply(post, refute_evidence + context_evidence)
    return llm_reply(post, [])

Pseudocode for the image-page filter, as implemented in the code:

# MUSE multimodal page filter  [code: model/main.py#L477-L554]
# Input:  candidate pages of one tier; post text; post images
# Output: at most two pages: best text match and best image match

def multimodal_filter(post, pages):
    # 1. Text score per page: dot product of text embeddings
    #    [code: model/main.py#L500-L521]
    text_scores = []
    for page in pages:
        text_scores.append(dot(embed(post.text), embed(page.text)))
    # 2. Image score per page: DINO cosine between each post image and
    #    the page's lead image, averaged over the post images
    #    [code: model/main.py#L479-L526]
    image_scores = []
    for page in pages:
        image_scores.append(mean_cosine(post.images, page.lead_image))
    kept = []
    # 3. Keep the best text match if its score is at least 95
    #    [code: model/main.py#L531-L543]
    best = argmax(text_scores)
    if text_scores[best] >= 95:
        kept.append(pages[best])
    # 4. Keep the best image match if its score is at least 0.7
    #    [code: model/main.py#L545-L547]
    best = argmax(image_scores)
    if image_scores[best] >= 0.7:
        kept.append(pages[best])
    return kept

Hyperparameters

The paper states few numeric settings. Values from the code are marked as such. Where the paper and code disagree, the Paper vs code section explains which one this card follows.

Results

Author-reported. Setup: 247 X posts (February 2021 to February 2023) with a high- and an average-helpfulness Community Notes response; 12 expert fact-checkers, 10 in the second phase, rating 13 criteria, with overall quality on a 10-point scale; GPT-4 as the primary model; a Prolific study of 988 U.S. participants §Experiments fig. 2. Baselines: high- and average-helpfulness Community Notes written by laypeople, GPT-4 without retrieval or vision §Experiments.

Expert agreement, measured with weighted Cohen's κ, is in §Experiments. Reference reachability reaches 0.79, four criteria (overall quality among them) fall between 0.41 and 0.51, and the remaining scored criteria fall between 0.28 and 0.40. Llama 3 70B was reported as qualitatively comparable to GPT-4 §Discussion [supp. fig. impact-foundation-llm].

Paper vs code

The paper's acknowledgements link the repository as the study's code and data, so it is treated as the official one §Acknowledgements. It is Apache-2.0 and was cloned at 94662ccf546abbbece52c6b608f730fb2c4e0301. The code is a single script, model/main.py, with prompts and lists under model/data/ main.py. Differences:

• Stop rule. The paper stops after two refuting pages §Approach. The code stops after three refuting evidence items, not pages main.py main.py.
• Evidence order is lost. The paper and the reply prompt both say evidence should be ranked by publisher tier and relevance, and facts at the top should be used first §Approach [supp. fig. prompt-response-generation]. The code collects evidence into Python sets and passes them on as list(set(...)). The order is then arbitrary and can change between runs main.py main.py.
• Multimodal filter. The paper describes relevance scoring that removes irrelevant pages §Approach. The code keeps at most two pages per tier, the single best text match and the single best image match main.py.
• Image captions. The paper says BLIP-2, Rekognition and Textract outputs are turned into descriptions by the LLM §Experiments. The script does not run those models. It reads precomputed JSON files main.py main.py main.py. It also runs a GPT-4 step that revises the caption against the post and against the OCR text, which the paper does not describe main.py. When there is no OCR and no celebrity label, the code uses a fixed "A photo of" prefix without the LLM main.py.
• Celebrity labels. A label is kept only if the name appears in the post text or in the OCR output main.py. The paper does not describe this filter.
• Image relevance. The paper says "visual content" §Experiments. The code compares each post image with the page's lead image only main.py main.py.
• Fallback search. The paper does not describe the code's fallback to a text search and a reverse-image search for image posts with too few refutations main.py.
• Retry with more results. The code repeats the unimodal search with top 10 results when nothing refutes the post main.py. The paper's "exhausting all credible pages" is not the same rule.
• Search engines. The paper says "Google Search". The code uses the Google Programmable Search JSON API for text main.py and SerpApi's Google reverse-image engine for images main.py.
• Time filter. The paper's temporal matching uses pages published up to 30 minutes before each Community Notes response §Experiments. That matching is not in main.py. The script keeps only pages published strictly before the post main.py.
• Llama model ID. The client is created for meta-llama/Meta-Llama-3.1-70B-Instruct but the request names meta-llama/Meta-Llama-3-70B-Instruct main.py main.py. The Llama function is not called by the script's __main__.
• Query prompt. The code's query prompt asks for exactly three queries. The paper says "different queries" §Approach prompt_query_generation_unimodal.txt.
• Context dropped. When there are no refutations, the code passes no facts to the reply prompt, so context-only evidence is discarded main.py.
• Data shipped. The repository releases tweet IDs only, as the README says, so the script needs the tweet content and image URLs obtained separately README.md. It also needs an OpenAI key, a SerpApi key and a Google Custom Search key, read from a keys file main.py. The eval folder ships the Community Notes data and the expert ratings README.md.
• Figures and supplement. The paper's figure and supplement labels such as "Supp. Fig." are cited here by caption title rather than by number, because the supplement numbering was not verified against a compiled PDF.

No official code.

Pitfalls

• Rule choices hidden in code. Thresholds of 90, 95 and 0.7 and the top-3 and top-10 limits are only in the code main.py main.py main.py main.py. The paper describes calibrating its relevance thresholds on 100 pairs but gives no resulting values §Experiments. The text scores are raw dot products, not normalised similarities, so a threshold of 90 or 95 means something only for the embedding scale it was tuned on. Check that scale before reusing these values.
• Set-based evidence. Because evidence goes through sets, two runs on the same post can use facts in different orders, so the reply prompt's request to use the facts listed first cannot be met main.py.
• Refutation counted as items. A refutation count of 3 can come from one long page or from three pages. The paper's "two pages" rule is not reproduced by the code main.py.
• LLM failure is silent. After 20 failed calls the helper returns an empty string, and the evidence parser then returns no evidence, with no error main.py main.py. A failed run looks like a post with no refutation.
• Parser fragility. The evidence parser keeps only lines that start with a dash and skips any line containing the word "NONE" anywhere, including inside a quoted sentence main.py main.py. A quote that contains "none" is dropped.
• Model alias. The code asks for gpt-4, a moving alias, not a dated snapshot main.py. The paper's contamination analysis relies on GPT-4's assumed September 2021 cutoff §Results. Pin a snapshot before any reproduction, and rerun the contamination split if the model changes.
• Temperature and sampling. All OpenAI calls use temperature 0 in the code main.py, but the paper does not say so. Expect output drift between model versions even at temperature 0.
• Cost is not in the code. The cost estimate (about 0.2 USD per post) and the two-minute latency are the authors' figures §Discussion; the repository has no cost accounting. The code makes many LLM calls per post: captions, queries, one evidence call per page, and the reply. It also crawls many pages in a Firefox WebDriver with a 10-second page-load timeout main.py.
• Crawl quality. Pages must have at least 100 words and a publication date, and duplicate text is dropped. Pages without a parsed date are removed silently main.py.
• Image relevance is one page. The image score uses the page's single lead image. If the image cannot be fetched or decoded, the score is -1 and the page cannot pass the image test main.py main.py.
• Results are self-reported. The expert study used 12 experts, of whom 10 completed the second phase, and 247 posts from X's Community Notes. Agreement was only fair for most criteria §Experiments.
• Information voids. The authors say they did not systematically evaluate detection of information voids or abstention, so the system's behaviour on breaking news with little evidence is not measured §Discussion.
• Scope. English text and images only, no video, and one platform §Discussion. The paper's expert and reader results do not show performance on other languages or platforms.

Lineage

• Builds on: X Community Notes, the crowd fact-checking system that supplies the baselines and the evaluation posts §Introduction §Experiments. No card exists for it.
• Uses: the MBFC ratings for publisher tiers §Experiments; msmarco-distilbert-base-tas-b and dino-vitb8 for relevance §Experiments; BLIP-2 for captions §Experiments.
• Compared against: GPT-4 alone, MUSE without retrieval, MUSE without vision, high- and average-helpfulness lay responses, GPT-5.5 and Llama 3 70B §Experiments §Discussion.
• Superseded by: not known from content in this wiki.