wikillm

Paper · 2023-10-16 · cs.CL

Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks

SheepDog fine-tunes RoBERTa to judge news by content, not style. Training adds GPT-3.5 rewrites of each article in several tones, a KL consistency loss across them, cross-entropy, and a multi-label head for four LLM-elicited rationales. Author-reported macro-F1 on LLM-restyled test sets: 80.99, 74.45, 85.63 (PolitiFact, GossipCop, LUN), against best baselines of 77.60, 71.60, 66.34.

Text classificationConsistency regularizationData augmentationAdversarial robustnessfake-newsrobustnessdata-augmentationllm

arXivPDFCodeMarkdown16 min read

AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Fake news detectors learn habits of writing. Real stories often read calm and factual, while fake ones often read loud and sensational, so a detector can pick up the shortcut "sensational means fake." That shortcut breaks when someone asks a language model to rewrite a fake story in the calm voice of a trusted newspaper. The paper shows that detectors built this way lose a lot of accuracy in that case.

The proposed detector, SheepDog, tries to look past the costume. During training, each news article is rewritten by a language model into other tones, and the detector is trained to give the same verdict for the original and for each rewrite. It also learns to say which of four common problems with fake news apply to an article, such as false information or biased opinion. That is a second training signal, and it can also serve as a human-readable hint for the verdict.

The method is a fine-tuned language-model classifier with a few extra training terms, so it can be added to an existing text classifier. The results here are the authors' own, measured on three public datasets.

TL;DR

  • Text-only detectors trained on ordinary articles lose F1 when a language model restyles the same articles. The largest drop in the paper is 38.33 points, for zero-shot LLaMA2-13B on LUN; the largest drop for a trained detector is 31.83, for KPT on LUN tab. 1.
  • SheepDog is RoBERTa-base with three losses: a KL consistency term across LLM tone rewrites, cross-entropy on the labels, and a multi-label BCE on four content rationales. The three are summed with equal weights §5.4 eq. 7.
  • Author-reported: on the adversarial set A, SheepDog reaches 80.99, 74.45 and 85.63 macro-F1 on PolitiFact, GossipCop and LUN, against best baselines of 77.60, 71.60 and 66.34 tab. 3.
  • The released code covers training of the RoBERTa model only. It has several details that differ from the paper, listed under Paper vs code.

Problem

Given the text of a news article, predict whether it is real or fake §3. The paper studies a text-only setting. Its concern is style: a malicious author can ask an LLM to rewrite a fake article in the style of a reputable outlet, and a detector trained on style-consistent data can fail on it §4.1 §4.2. The paper states two observations: text-based detectors are vulnerable to this, and LLMs used zero-shot are weaker fake news detectors than fine-tuned detectors §4.2.

Datasets are PolitiFact, GossipCop (both from FakeNewsNet) and LUN §6.1.1 tab. 2.

Method

SheepDog has one shared language-model encoder with two linear heads and three training losses §5 fig. 2. The reported test accuracy uses only the veracity head on the original article sheepdog.py.

Step 1: Reframing the training articles §5.1 app. B.2

  • For each labelled training article p, the method produces two rewrites with an LLM: a reliable-style rewrite p_R and an unreliable-style rewrite p_F §5.1.
  • The prompt asks the LLM to rewrite the article in one named tone, with the article appended after the instruction §5.1. The appendix gives the four exact prompts app. B.2.
  • Reliable tones are "objective and professional" and "neutral". Unreliable tones are "emotionally triggering" and "sensational" §5.1 app. B.2.
  • Each training article gets one reliable and one unreliable rewrite per pass, chosen at random §5.1. The released code redraws the choice every epoch (see Pitfalls).
  • The generator is GPT-3.5, with temperature 0.7 and at most 512 response tokens app. B.2. The model snapshot is not stated.
  • A check on content: the authors ask GPT-3.5 for a central claim of each article and whether the rewrite entails it. Entailment holds in 86.20% to 89.22% of pairs on GossipCop and LUN app. C tab. 12.

Step 2: Encoder and heads §5.2 eq. 1 eq. 2 eq. 5

  • A pretrained language model M (RoBERTa in the paper's experiments) gives one vector per text: h = M(p), h_R = M(p_R), h_F = M(p_F) eq. 1. The paper calls the vector a representation of dimension d, and does not say which pooled output is used.
  • The veracity head is a single layer. It gives two logits per text (real, fake) eq. 2 §6.1.3.
  • The attribution head is a second single layer. It gives four scores per text, one per rationale eq. 5 §5.3.

Step 3: Style alignment loss §5.2 eq. 3

The veracity distribution of the original article should match that of each rewrite. The paper averages two KL-divergence terms, one for p_R and one for p_F. It writes the loss as MEAN(L1(y_R, y), L1(y_F, y)) with L1 the KL divergence, and does not state which argument is the target or whether gradients flow through the original's distribution (see Paper vs code).

Step 4: News loss §5.2 eq. 4

Cross-entropy on the veracity logits of the original article, using its gold label.

Step 5: Content-focused attributions §5.3 app. B.3

  • An LLM is asked, for each fake training article, which of four problems it has, with the instruction to answer "No problems" if none apply app. B.3. The four rationales are: lack of credible sources (A), inconsistencies with reputable sources (B), false or misleading information (C), and biased opinion (D) app. B.3. The letters are the paper's rationale labels, not the test-set letters A to D.
  • The answer becomes a 4-dimensional binary pseudo-label. Real articles and their rewrites get all-zero labels §5.3.
  • The paper says the pseudo-labels for the rewrites are obtained the same way. The released data stores separate attribution files for each reframing tone pair load_data.py.
  • Temperature is 0 for the attribution queries app. B.3.
  • Loss: binary cross-entropy on the sigmoid of each score, averaged over the three texts eq. 6.

Step 6: Total objective §5.4 eq. 7

The loss is the sum of the style, news and attribution losses, all with weight one. The paper trains the whole model end to end §5.4.

Algorithm

The paper gives no algorithm box. The pseudocode below follows the equations and the released training loop.

Pseudocode
# SheepDog loss for one batch  [§5.4] [eq. 7]
# Shapes: logits (B, 2), attribution scores (B, 4); B = batch size.

def sheepdog_loss(p, p_R, p_F, y, s, s_R, s_F):
    # One shared encoder for the article and its two rewrites. [eq. 1]
    h = encode(p)                            # (B, d)
    h_R = encode(p_R)                        # (B, d)
    h_F = encode(p_F)                        # (B, d)

    # Veracity logits, two per text. [eq. 2]
    z = veracity_head(h)                     # (B, 2)
    z_R = veracity_head(h_R)                 # (B, 2)
    z_F = veracity_head(h_F)                 # (B, 2)

    # Attribution scores, four per text. [eq. 5]
    a = attribution_head(h)                  # (B, 4)
    a_R = attribution_head(h_R)              # (B, 4)
    a_F = attribution_head(h_F)              # (B, 4)

    # Cross-entropy on the original article only. [eq. 4]
    l_news = cross_entropy(z, y)

    # Original's probabilities as the KL target. [code: src/sheepdog.py#L185]
    p_orig = softmax(z)                      # (B, 2)

    # Style alignment: average KL over the two rewrites. [eq. 3]
    # [code: src/sheepdog.py#L192-L193]
    l_style = 0.5 * (
        kl_divergence(p_orig, log_softmax(z_R))
        + kl_divergence(p_orig, log_softmax(z_F))
    )

    # Multi-label BCE on sigmoid scores, averaged over three texts. [eq. 6]
    l_attr = (
        bce(sigmoid(a), s)
        + bce(sigmoid(a_R), s_R)
        + bce(sigmoid(a_F), s_F)
    ) / 3

    # Equal-weight sum of the three terms. [eq. 7]
    return l_style + l_news + l_attr

Training procedure

  • Batch size 4, 5 epochs, learning rate 2e-5, maximum length 512 tokens §6.1.3.
  • Each epoch, the rewrites for the training set are reloaded and the random choice between tones is redrawn sheepdog.py load_data.py.
  • Ten runs with different seeds are averaged §6.1.3.

Hyperparameters

NameValueWhereNotes
EncoderRoBERTa-base, pretrained weights§6.1.3code loads roberta-base sheepdog.py
Framework versionsPyTorch 1.10.0, Transformers 4.13.0§6.1.3README pins Python 3.7 README.md
Max sequence length512§6.1.3sheepdog.py
Batch size4§6.1.3default in code sheepdog.py
Epochs5§6.1.3default in code sheepdog.py
Learning rate2e-5§6.1.3sheepdog.py
Optimizernot statednonecode uses AdamW with default weight decay sheepdog.py
Learning-rate schedulenot statednonecode: linear decay to zero over 10,000 steps, no warmup sheepdog.py
Dropout on pooled outputnot statednonecode: 0.5 before both heads sheepdog.py
Head typesingle layer each§6.1.3code uses one linear layer per head sheepdog.py
Veracity head outputs2eq. 2real and fake
Attribution head outputs4§5.3four rationales
Loss weights1, 1, 1eq. 7code sheepdog.py
Style lossmean of two KL termseq. 3code uses 0.5 each sheepdog.py
Attribution lossmean of three BCE termseq. 6code divides by 3 sheepdog.py
Reframing generatorGPT-3.5, temperature 0.7, max 512 tokensapp. B.2model snapshot not stated
Reframing tonesreliable: objective and professional, neutral; unreliable: emotionally triggering, sensational§5.1 app. B.2
Attribution generatorGPT-3.5, temperature 0app. B.3model snapshot not stated
Attribution rationales4 (A to D, see Method)app. B.3
Runs10, averaged§6.1.3code seeds 0 to 9 sheepdog.py
Metricsaccuracy; macro-F1§6.1.3code uses macro averaging sheepdog.py
Model selectionnot statednonecode evaluates the final epoch only, with no validation split sheepdog.py
Dataset split80/20; temporal (last 20% as test) for PolitiFact and GossipCop; random for LUN§6.1.1
Adversarial test setsA to D: real news in National Enquirer or The Sun style; fake news in CNN or New York Times style§4.1 §6.2 tab. 4generated with GPT-3.5
Baseline LLMsgpt-3.5-turbo-0301 (temp 0); gpt-3.5-turbo-instruct; Llama-2-13b-chat-hf (greedy)app. B.1
LLaMA2 fine-tuning (appendix)QLoRA, 1 epoch, batch 16, learning rate 1e-4app. A
Two-layer head variant (ablation)hidden size 64§6.5not in released code

Results

All numbers are author-reported. Macro-F1 is in percent. Setup: 13 baselines §6.1.2, three datasets tab. 2, 10 runs §6.1.3. Significance markers are the authors' Wilcoxon signed-rank tests at p < .01 tab. 3.

Headline numbers:

  • Adversarial set A (macro-F1): SheepDog 80.99 on PolitiFact, 74.45 on GossipCop and 85.63 on LUN. The best baseline on the same set is UDA (77.60), KPT (71.60) and SAFE (66.34) tab. 3.
  • Average gain over the best baseline across all four adversarial sets: 2.59 points on PolitiFact, 2.77 on GossipCop and 15.70 on LUN §6.2. These averages match the per-set gaps in tab. 3.
  • Unperturbed test sets (accuracy / macro-F1): SheepDog 88.44 / 88.39 (PolitiFact), 75.77 / 75.75 (GossipCop), 93.05 / 93.04 (LUN). The best baseline F1 is 87.76 (DualEmo), 75.36 (DualEmo) and 84.06 (KPT) tab. 5.

Table 1. Vulnerability of existing detectors (macro-F1; "drop" is the original-to-restyled gap, in points) tab. 1

MethodPolitiFact originalPolitiFact dropGossipCop originalGossipCop dropLUN originalLUN drop
dEFEND\c82.5912.1570.744.3480.9219.16
SAFE\v79.858.7470.642.9379.4613.12
SentGCN80.7713.8269.295.5979.6616.65
DualEmo87.7615.3475.365.8981.5224.97
BERT84.9912.6874.505.5280.9624.61
RoBERTa87.4011.2374.053.0582.1229.65
DeBERTa86.3011.7373.802.8583.6730.34
UDA87.7410.1474.224.5482.9420.71
PET85.5111.0274.633.0883.6631.08
KPT87.7013.2674.232.6384.0631.83
GPT-3.5 (zero-shot)69.6127.4856.3016.7179.9720.34
InstructGPT (zero-shot)64.5920.6950.389.1368.1611.39
LLaMA2-13B (zero-shot)63.1529.9153.5427.7570.9738.33

Table 2. Dataset statistics tab. 2

DatasetArticlesRealFake
PolitiFact450225225
GossipCop7,9163,9583,958
LUN7,5003,7503,750

Table 3. Macro-F1 on the four adversarial test sets A to D tab. 3 (* marks the paper's significance; set definitions are in Table 4 below.)

GroupMethodPF APF BPF CPF DGC AGC BGC CGC DLUN ALUN BLUN CLUN D
G1dEFEND\c70.4469.7773.6772.9866.4066.5568.9369.0761.7662.2872.9572.50
G1SAFE\v71.1170.8075.5575.2467.7167.0568.3167.6566.3467.0872.4073.16
G1SentGCN66.9562.5069.5465.0863.7063.0763.6163.0163.0162.5076.1175.56
G1DualEmo72.4271.2377.0775.8069.4768.5071.6970.7156.5554.7868.5366.80
G2BERT72.3171.3777.2376.2468.9868.1771.9571.1156.3554.6168.5066.74
G2RoBERTa76.1774.9578.2877.0571.0070.4772.5672.0252.4753.6268.3169.46
G2DeBERTa74.5774.3680.6080.3570.9571.1572.5172.7153.3355.4567.1669.27
G2UDA77.6075.5779.2177.1769.6869.3372.1671.8062.2361.8068.2567.80
G2PET74.4970.7575.4971.7671.5570.8573.7473.0252.5853.3063.7164.33
G2KPT74.4473.3277.7376.6071.6071.0173.6973.1052.2353.6265.7167.15
G3GPT-3.542.1343.4456.6158.1739.5938.6748.4447.3859.6361.2465.7467.43
G3InstructGPT43.9043.9054.2154.2141.2540.1844.2643.1256.7757.1558.9359.32
G3LLaMA2-13B33.2434.4853.6455.4525.7926.0637.0737.4032.6433.0050.8151.33
OursSheepDog80.99*79.89*82.36*81.2474.45*74.38*75.95*75.88*85.63*86.06*87.89*88.32*

G1: text-based detectors. G2: fine-tuned language models. G3: LLMs.

Table 4. Adversarial set definitions tab. 4 §4.1

  • Set A: real news in National Enquirer style, fake news in CNN style.
  • Set B: real news in National Enquirer style, fake news in New York Times style.
  • Set C: real news in The Sun style, fake news in CNN style.
  • Set D: real news in The Sun style, fake news in New York Times style.

Table 5. Unperturbed test sets, accuracy / macro-F1 tab. 5

MethodPF Acc.PF F1GC Acc.GC F1LUN Acc.LUN F1
dEFEND\c82.6782.5970.8570.7481.3380.92
SAFE\v79.8979.8570.7170.6479.9379.46
SentGCN81.1180.7769.3869.2980.0779.66
DualEmo87.7887.7675.5175.3681.7881.52
BERT85.2284.9974.6074.5081.1380.96
RoBERTa88.0087.4074.1474.0582.5382.12
DeBERTa86.3386.3073.8673.8084.0183.67
UDA87.7787.7474.2874.2283.0282.94
PET85.5685.5174.7574.6384.0083.66
KPT87.7887.7074.3874.2384.4084.06
GPT-3.571.1169.6161.4956.3080.6779.97
InstructGPT67.7864.5958.3350.3870.8768.16
LLaMA2-13B65.5663.1555.7453.5472.4770.97
SheepDog88.4488.3975.7775.7593.05*93.04*

Table 6. SheepDog on different encoders, macro-F1, adversarial set A tab. 6

MethodPolitiFactGossipCopLUN
RoBERTa76.1771.0052.47
SheepDog-RoBERTa80.99*74.45*85.63*
BERT72.3168.9853.97
SheepDog-BERT81.37*73.54*80.36*
DeBERTa74.5770.9553.33
SheepDog-DeBERTa81.10*73.89*82.58*

Note: the BERT row for LUN here (53.97) differs from the BERT row for set A in Table 3 (56.35). See Paper vs code.

Table 7. SheepDog on LLM backbones, macro-F1, adversarial set A tab. 7

MethodPolitiFactGossipCopLUN
SheepDog (RoBERTa)80.99*74.45*85.63*
SheepDog-LLaMA280.82*74.04*81.87*
RoBERTa76.1771.0052.47

Table 8. Ablation, macro-F1, adversarial set A tab. 8

VariantPolitiFactGossipCopLUN
SheepDog80.9974.4585.63
with 2-layer heads79.8374.0384.75
without reframings and style loss (-R)76.7170.9853.27
without attributions (-A)80.7373.7484.83
RoBERTa76.1771.0052.47

Table 9. Robustness to reframing prompts, macro-F1, adversarial set A tab. 9

Variant (reliable / unreliable tone pair)PolitiFactGossipCopLUN
Best baseline77.6071.6066.34
SheepDog (default: samples among all four tones)80.9974.4585.63
R1: objective and professional / emotionally triggering79.0274.2277.42
R2: objective and professional / sensational79.9374.1686.18
R3: neutral / emotionally triggering80.3673.5576.77
R4: neutral / sensational79.7174.0185.55

Note: the default row samples between the two reliable and the two unreliable tones; each "R" row fixes one tone pair §6.6.

Table 10. LLM in-context learning, macro-F1, set A tab. 10

MethodPolitiFactGossipCopLUN
GPT-3.5 zero-shot42.1339.5959.63
+ 2 in-context examples38.5738.8857.51
+ 2 in-context examples, with reframings41.7641.8360.29
+ 4 in-context examples42.3541.4760.25

Table 11. LLaMA2-13B fine-tuning with QLoRA, macro-F1, set A tab. 11

MethodPolitiFactGossipCopLUN
LLaMA2-13B zero-shot33.2425.7932.64
+ fine-tuning36.9647.8334.31
+ fine-tuning with reframings44.4456.0642.22

Table 12. Claim entailment between originals and objective-tone rewrites, % tab. 12

Datasetoriginal entails rewriterewrite entails original
GossipCop86.2089.17
LUN89.2287.53

Tables 13 and 14 are worked examples, not results. Figure 1 is a motivating example, figure 2 an overview of the method, and figure 3 a case study on one LUN article fig. 1 fig. 2 fig. 3. The case study reports that RoBERTa labels the original article fake, but labels two restyled versions real, while SheepDog keeps the fake label, and its top attribution is false or misleading information §6.7.

Paper vs code

Code: https://github.com/jiayingwu19/SheepDog at commit 3223e35d5ba514d8d2e9cd6a91fd644a0205d8ca. The repository has no LICENSE file, so nothing is quoted verbatim. The repository is linked from the paper's footnote and is under the first author's account. The repository contains the training script, its data files, and archived logs.

Matches:

Differences and gaps:

  • Learning-rate schedule. Not stated in the paper. The code uses linear decay over a fixed 10,000 steps with no warmup sheepdog.py.
  • Optimizer. Not stated in the paper. The code uses AdamW sheepdog.py.
  • Dropout. Not stated in the paper. The code applies dropout 0.5 to the pooled encoder output before both heads sheepdog.py sheepdog.py.
  • Encoder output. The paper calls the encoder output h without saying which pooling is used. The code uses the RoBERTa pooler output sheepdog.py.
  • Style-loss direction. The paper leaves the order of the KL arguments open. The code computes KL from the original's probabilities to each rewrite's, with the original's probabilities not detached, so gradients flow through both sides sheepdog.py.
  • Rewrite selection. The paper says one reliable and one unreliable rewrite are chosen at random. The code redraws, every epoch, which half of the articles use the neutral and sensational rewrites in place of the objective and emotionally triggering ones load_data.py sheepdog.py.
  • Evaluation set. The paper reports all four adversarial sets. The code loads only set A by default, and says other sets need a code change load_data.py.
  • Model selection. The paper does not say. The code reports the final epoch with no validation split sheepdog.py.
  • Missing from the repository: the two-layer-head ablation, the -R and -A ablations, the BERT, DeBERTa and LLaMA2 variants, the LLM baselines, the in-context and fine-tuning experiments, the claim-entailment check, and the case-study inference. Only RoBERTa training is included.
  • Paper-internal inconsistency. The BERT row for LUN in Table 6 (53.97) does not match the BERT row for set A in Table 3 (56.35). The other rows of Table 6 match Table 3 for set A.
  • Archived logs. logs/logs_archive_all4_adv/ contains ten-run averages for SheepDog that match the paper's SheepDog numbers. In log order, the PolitiFact averages are 88.39 (original) and 80.99, 79.89, 82.36 and 81.24 (A to D). GossipCop: 75.75 and 74.45, 74.38, 75.95, 75.88. LUN: 93.04 and 85.63, 86.06, 87.89, 88.32. The block labels for A to D are not in the logs, so this mapping is inferred from log order. The log format (for example, per-class F1 lines) does not match the current src/sheepdog.py, so the logs come from an earlier version of the script.

Pitfalls

  • The 38% headline. The largest drop in Table 1 is for zero-shot LLaMA2-13B on LUN (38.33). The largest drop for a trained detector is 31.83 (KPT, LUN). Table 1's "A" column is a drop in points, while "A" in Table 3 is a test-set name.
  • Adversarial and rewrite data cannot be regenerated exactly. The restyled test sets and reframings were produced by GPT-3.5, and the model snapshot is not stated §6.2 app. B.2. The released .pkl files are the only exact copies .
  • Attention-mask bug. The dataset class sets the second rewrite's attention mask from the first rewrite's encoding sheepdog.py. Where the two rewrites tokenize to different lengths, the second rewrite is encoded with the wrong mask. The paper does not discuss it, and its effect is not measured here.
  • Learning-rate schedule fixed at 10,000 steps. The decay length does not depend on the dataset size. Using Table 2 and the 80/20 split, PolitiFact has about 360 training articles, or about 90 steps per epoch (about 450 in total). The rate would then end near 1.9e-5. LUN has about 6,000 training articles, or about 1,500 steps per epoch (about 7,500 in total), ending near 5e-6. These sizes are derived from the paper's split, not checked against the .pkl files.
  • Head shape. The paper says "MLP" with a single layer. The code uses one linear layer per head with dropout before them, so the pooled output passes through no hidden nonlinearity sheepdog.py.
  • Attribution pseudo-labels. The paper says real articles and their rewrites get all-zero labels. The released label files were not inspected for this, so check them before relying on it.
  • Pickle files. The data is stored as Python pickles load_data.py. pickle.load runs code embedded in the file, so load only files from a source you trust, such as the authors' repository at the commit above.
  • Final-epoch evaluation. The reported test numbers come from the last epoch with no early stopping or validation set sheepdog.py.
  • Pinned versions. The README pins Python 3.7, PyTorch 1.10 and Transformers 4.13. The code passes pad_to_max_length=True to the tokenizer sheepdog.py. This argument may fail on newer Transformers versions. Not tested here.
  • CUDA only. The device is hard-coded to cuda sheepdog.py.

Lineage

  • Builds on: RoBERTa (arXiv:1907.11692, no card yet), used as the encoder. The LLM rewriting and attribution steps have no cards linked here.
  • Compared against: the 13 baselines in Table 3 (dEFEND, SAFE, SentGCN, DualEmo, BERT, RoBERTa, DeBERTa, UDA, PET, KPT, GPT-3.5, InstructGPT, LLaMA2-13B). No cards in this wiki for them yet.
  • Superseded by: not known from content in this wiki.

Source and license

Adapted from “Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attacks” by Jiaying Wu, Jiafeng Guo, Bryan Hooi (arXiv:2310.10830), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/jiayingwu19/SheepDog at commit 3223e35d5ba5 (none (no LICENSE file in repository)). Any quoted code is shown under that license.