Paper · 2021-02-27 · cs.CL, cs.SI
A Survey on Stance Detection for Mis- and Disinformation Identification
Survey of stance detection for mis- and disinformation detection. Frames stance as a fact-checking step or as a component of rumour and media pipelines, tabulates 15 datasets and the methods tried on them, and lists open challenges. Runs no experiments; the scores it tabulates come from other papers.
Stance detectionAutomated fact-checkingMulti-task learningmisinformationstance-detectionsurvey
AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Online, false claims spread quickly, and people who check them want to know what others think about each claim. Stance detection asks a simple question about a text: does it agree with a claim, disagree with it, or take neither side? This survey asks how that question helps catch fake news and rumours.
It describes two ways to use it. In the first, stance is the core of fact-checking: the question is what the evidence says about the claim. In the second, stance is one part of a larger pipeline: the question is how the crowd or the media react to a rumour, and that reaction helps decide whether the rumour is true.
The survey also collects the datasets and methods used so far, and lists open problems. Examples are small and unevenly balanced datasets, the need for more languages, and the difficulty of explaining a model's decisions.
TL;DR
A survey, first published 2021-02-27 and last revised 2022-05-08 (v3), on stance detection as a tool for mis- and disinformation detection. It separates two formulations: stance as a fact-checking step, and stance as a component of rumour and media pipelines. It tabulates 15 datasets and the approaches tried on them. It runs no experiments of its own, and its results table repeats scores from other papers that use different metrics, so use it as a map of the field and a list of open problems rather than as a benchmark.
Problem
Stance detection is central to finding false content online, yet the authors say no survey had linked the different stance formulations to the detection of mis- and disinformation §1. The task itself is not standardised: label sets, targets, sources and evidence formats differ between datasets §2 §3 tab. 1, so results from different datasets are hard to compare §4. The survey assumes that the target of a stance may be stated in the text or only implied by its context §3.
Method
This is a survey, so the "method" below is the scope and taxonomy the paper sets out, with the open issues it lists. The paper contains no equations and no algorithms.
Definitions §2
- Stance is a speaker's standpoint towards a proposition, and it can be shaped by social and cultural factors, not only personal opinion §2.
- Stance detection, in the survey's working definition, is a classification of a (text, target) pair into a stance label. The label set is Favor, Against or Neither; a Neutral label is sometimes added, and the target may or may not appear in the text §2.
- Stance detection is distinct from biased language detection, emotion recognition, perspective identification (the target is always explicit), sarcasm detection and sentiment analysis §2.
- Mis- and disinformation both mean false information. Disinformation adds an intent to harm §1.
Two formulations §3 fig. 1
- (a) Fact-checking as stance. The stance of the evidence towards a claim is used as the veracity signal §3.1 fig. 1a.
- (b) Stance as a pipeline component. The stance of the crowd, the media or other sources towards a rumour feeds a decision about its veracity. The rumour setting uses the labels Support, Deny, Query and Comment §3.2. The survey also covers stance use in misconception checks, media profiling and user profiling app. B, and in systems that combine stance with other evidence app. C fig. 1b.
Sketch of formulation (a). The paper gives no code or aggregation rule; the aggregation step is marked as not stated §3.1:
# Formulation (a): stance of the evidence as the veracity signal [§3.1]
# Input: claim (text); evidence (list of texts)
# Output: veracity label (FEVER-style: SUPPORTED, REFUTED, NOT ENOUGH INFO)
def verify_claim(claim, evidence):
# Predict the stance of each evidence text towards the claim [§3.1]
stances = [stance_model(claim, text) for text in evidence]
# How the stances become one label is not stated in the survey [§3.1]
return aggregate(stances)Taxonomy of stance inputs §3 fig. 2
The survey organises the field in four levels: (i) the source of the data; (ii) the inputs, the target that stance is about and the context that expresses it; (iii) meta-level categories of the input; and (iv) the textual object type, such as a tweet or a headline §3 fig. 2. When the target is implicit, the task looks much like sentiment analysis §3, footnote. Example scenarios (a headline with a news body, tweets with an implicit topic, a multi-hop claim check, and threaded discussions) are illustrated in tab. 3 and discussed in app. A.
Datasets tab. 1
The survey lists 15 datasets: 10 English and 5 non-English. Table 1 gives the source, target, context, evidence type, size and task of each. Sizes are in thousands, rounded to the hundreds tab. 1.
| Dataset | Source(s) | Target | Context | Evidence | Instances | Task |
|---|---|---|---|---|---|---|
| Rumour Has It | Topic | Tweet | Multiple | 10K | Rumours | |
| PHEME | Claim | Tweet | Thread | 4.5K | Rumours | |
| Emergent | News | Headline | Article (body summarised) | Multiple | 2.6K | Rumours |
| FNC-1 | News | Headline | Article | Single | 75K | Fake news |
| RumourEval '17 | Implicit | Tweet | Thread | 7.1K | Rumours | |
| FEVER | Wikipedia | Claim | Facts | Multiple | 185K | Fact-checking |
| Snopes | Snopes | Claim | Snippets | Multiple | 19.5K | Fact-checking |
| RumourEval '19 | Twitter, Reddit | Implicit | Post | Thread | 8.5K | Rumours |
| COVIDLies | Claim | Tweet | Single | 6.8K | Misconceptions | |
| TabFact | Wikipedia | Statement | WikiTable | Multiple | 118K | Fact-checking |
| Arabic FC | News | Claim | Document | Single | 3K | Fact-checking |
| DAST (Danish) | Submission | Comment | Thread | 3K | Rumour | |
| Croatian | News | Title | Comment | Single | 0.9K | Claim verifiability |
| ANS (Arabic) | News | Claim | Title | Single | 3.8K | Claim verification |
| AraStance (Arabic) | News | Claim | Title | Single | 4K | Claim verification |
Evidence types, as the survey defines them: Single is one document or fact; Multiple is several pieces of evidence; Thread is a conversation of posts tab. 1.
Method families §4
- Feature-based systems on FNC-1. Competitors used hand-made features: words, embeddings and sentiment lexica. In one analysis, lexical features mattered most, topic-model features came next, and sentiment did not help §4. Models struggled with the Disagree class (up to 18 F1) and found Unrelated easy (over 99 F1) §4.
- Memory networks. A model that learns a similarity matrix and a stance filter, and extracts snippets as partial explanations. It also performed poorly on Agree and Disagree because the memory was trained without gold snippets §4.
- Pre-trained Transformers. Transfer from pre-trained Transformers gave large gains. On FNC-1, RoBERTa raised Disagree from 18 to 58 F1 and Agree from 50 to 70 F1; the most important hyperparameter was the learning rate, and freezing layers did not help §4. Cross-lingual results include mBERT at 76.7 F1 on the Arabic ANS dataset §4, and language-specific pre-training was described as pivotal for AraStance §4.
- Evidence retrieval and reasoning (FEVER). A retrieval stage selects Wikipedia sentences, and a classifier then judges the claim. Top systems used LSTM-based inference models. A neural semantic matching network ranked first with a FEVER score of 64.2, using page-view and WordNet features with contextual embeddings §4. Later systems used graph neural networks over evidence, reaching 67.1 and then about 70 FEVER score, with a better retriever and graph propagation §4. A retriever with co-reference pre-training was also reported §4.
- Fact-checking without retrieval. Pre-trained language models used as checkers, through masked-token scoring or the perplexity of the claim given the document. They need no retrieval step, but are prone to pattern biases and perform below supervised approaches, though few-shot training helps §4.
- Threaded stance. Tree-structured models over conversation threads, branch-wise LSTMs, and Tree and Hierarchical Transformers for rumour debunking. Pre-training on stance data, and per-class pre-training data built from related tasks, improved RumourEval results §4. Thread depth and temporal patterns matter: denying posts appear later, and the share of querying posts rises over time while their absolute number falls §4.
- Multi-dataset learning. Multi-task learning over several stance and related datasets; cross-domain learning over 16 stance datasets with a domain adaptation via MoLE; and pattern-exploiting training across 12 languages, which the paper reports as useful in low-resource and full-resource settings §4 §5. Cross-domain learning used a domain-adaptation architecture named MoLE, which applies adaptation at several stages of the model §4.
Open issues §5
- Dataset size and imbalance. Most stance datasets are small and skewed. Mitigations are sampling, class weighting, auxiliary-task data and training on several datasets §5.
- Data mixing. Label inventories differ, so combining datasets needs label embeddings or other ways to relate labels §5.
- Multilinguality. Few non-English datasets exist, and most are small and not cross-lingual. Cultural context matters, so the paper does not argue for a fully language-agnostic setup §5.
- Context. Stance depends on user history, thread structure and media features such as funding and bias §5.
- Multimodal content. Deepfakes and memes need image and text together §5.
- Shades of truth and label semantics. Fact-checking often uses more than two labels, while stance labels rarely express strength §5.
- Explainability. Partial explanations come from memory networks, attention and topics, but each has limits; the paper flags attention weights as possibly misleading §5.
- Integration. The paper argues for a tighter link between stance and fact-checking pipelines, and points to credibility weighting of users and sources §5.
Hyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| Experiments run by the survey | none | §1–§5 | Survey only; no training settings are given |
| Learning rate (fine-tuning) | not stated | §4 | Named as the most important hyperparameter for the cited FNC-1 transformer work; no value given |
| Stance datasets' class weights | not stated | §5 | Weighting is mentioned as a mitigation; no scheme is given |
Results
Author-reported. The survey runs no experiments. tab. 2 lists, for each dataset, the best score it found in the literature, with the paper that reports it. The table notes that the scores use different metrics (macro F1, FEVER score, accuracy) and different datasets, so they cannot be compared directly §4 (State of the Art paragraph). Some papers round to integers; the survey marks those with "?" and this card writes them as "decimals not stated".
| Paper (as cited in tab. 2) | Dataset | Score | Metric |
|---|---|---|---|
| Hardalov et al. 2021 | Rumour Has It | 71.2 | F1 macro |
| Kumar and Carley 2019 | PHEME | 53.2 | F1 macro |
| Hardalov et al. 2021 | Emergent | 86.2 | F1 macro |
| Guderlei and Assenmacher 2020 | FNC-1 | 78.2 | F1 macro |
| Yu et al. 2020 | RumourEval '17 | 50.9 | F1 macro |
| Dominiks (2021), FEVER leaderboard | FEVER | 76.8 | FEVER score |
| Wang et al. 2020 | Snopes | 78.3 | F1 macro |
| Yang et al. 2019 | RumourEval '19 | 61.9 | F1 macro |
| Weinzierl et al. 2021 | COVIDLies | 74.3 | F1 macro |
| Liu et al. 2022 | TabFact | 84.2 | Accuracy |
| AraStance 2021 | Arabic FC | 52 (decimals not stated) | F1 macro |
| Lillie et al. 2019 | DAST | 42.1 | F1 macro |
| Bosnjak and Karan 2019 | Croatian | 25.8 | F1 macro |
| AraStance 2021 | ANS | 90 (decimals not stated) | F1 macro |
| AraStance 2021 | AraStance | 78 (decimals not stated) | F1 macro |
Source: tab. 2. The Dominiks row is taken from a public leaderboard, not from a paper tab. 2 footnote.
Paper vs code
No official code.
What was searched: the paper, its abstract and the arXiv metadata name no code repository; the only links are to datasets and to the FEVER leaderboard tab. 2. A GitHub repository search for "stance detection misinformation survey" returned nothing. A search within the copenlu organisation (the Augenstein lab) returned code for other papers, TESTED and wiki-stance, which this survey does not link and which are not used here.
Pitfalls
- Table 2 is not one benchmark. The scores use different datasets, metrics and splits. The paper says so itself §4 (State of the Art paragraph). Do not rank methods by this table.
- Arabic FC and AraStance are swapped between text and table. The text says AraStance reaches 52 F1 and Arabic FC 78 F1 §4. The table gives 52 to Arabic FC and 78 to AraStance tab. 2. The paper does not resolve this; check the AraStance paper before quoting either number.
- RumourEval '19 has two different figures. The text gives 57.9 F1 for the best stance result on RumourEval '19 §4. The table gives 61.9 for the same paper and dataset tab. 2. The difference is not explained.
- Rounded scores. Three entries are integers in the source (Arabic FC, ANS, AraStance); their decimals are not stated tab. 2.
- The leaderboard row is not a peer-reviewed result. Dominiks (2021) is taken from the FEVER leaderboard tab. 2 footnote.
- Label sets differ by dataset. FNC-1 uses positive, negative, discuss and unrelated §3.1; the general definition uses Favor, Against and Neither §2; FEVER uses SUPPORTED, REFUTED and NOT ENOUGH INFO §3.1; rumour work uses Support, Deny, Query and Comment §3.2. Do not merge them without a mapping. The paper says FEVER differs from stance detection because it scores evidence against a claim, not an author's stance §3.1.
- Dataset-size claim is loose. The paper says most stance datasets have at most a few thousand examples §5, but tab. 1 lists 0.9K to 185K, and the larger ones are fact-checking sets rather than stance sets.
- Summaries add noise. The Emergent dataset uses summarised article bodies tab. 1; the paper notes that automatic summaries can be noisy and that human summaries are costly app. A.
- Selection method not stated. The text does not say how the surveyed papers were found or chosen. Treat coverage as the authors' choice.
- Weighting threads is not trivial. Class weighting does not carry over simply to threaded stance §5 footnote.
- Attention is weak as an explanation. The paper cites work saying that attention weights can mislead, so attention maps should not be read as reasons §5.
- Figures are described only in words. Figure 1 contrasts the two formulations and Figure 2 shows the stance-type taxonomy fig. 1 fig. 2. Figure content is not reproduced here.
Lineage
- Builds on: none stated.
- Compared against: the surveys cited in §1: Zubiaga et al. 2018 on stance in rumour verification; a survey of stance detection on social media, cited as ALDAYEL2021102597; a holistic stance survey by Kucuk (2020), cited as kucuk-2020-stance-survey; and fact-checking surveys. The paper says none of them surveys the link between stance and false-content detection §1.
- Superseded by: not known from content already on this wiki.
Source and license
Adapted from “A Survey on Stance Detection for Mis- and Disinformation Identification” by Momchil Hardalov, Arnav Arora, Preslav Nakov, Isabelle Augenstein (arXiv:2103.00242), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.