BAE: BERT-based Adversarial Examples for Text Classification

In plain words

A text classifier, for example one that decides whether a review is positive, can be fooled by changing a few words in a way a person would hardly notice. Earlier attacks often swapped words for synonyms without checking the sentence around them, which produced odd phrasing that people can spot.

BAE uses a language model trained to fill in blanks. It hides one word at a time, asks the model what fits there, and keeps only candidates that keep the meaning close to the original. It can also add a new word next to a word instead of replacing it. It goes through the words that matter most to the classifier first and stops as soon as the label flips. It only needs the classifier's output probabilities, not its internals.

TL;DR

BAE is a black-box attack on text classifiers. It scores each word by how much deleting it lowers the probability of the correct label, then masks words in that order and fills each mask with BERT's masked-LM predictions. Candidates are filtered by sentence similarity (USE) and, for replacements, by part of speech. The paper reports larger accuracy drops than TextFooler on most settings, but its own numbers show BERT is the exception for replacement-only attacks. Results are author-reported on seven datasets (sentiment, subjectivity and question type).

Problem

Synonym-replacement attacks such as TextFooler pick substitutes from word embeddings, not from the sentence context, so they can change meaning or produce unnatural words §1. The paper wants replacements that fit the grammar and context of the sentence §1 §2.

The threat model is black-box and soft-label: the attacker can query the classifier for output probabilities on a given input and has no access to parameters, gradients or training data §2. The goal is an adversarial text that the classifier labels differently from the true label, and that stays grammatical and semantically similar to the original §2.

Method

BAE works at the token level with two operations §2:

• Replace (R): replace a token with a new one.
• Insert (I): insert a new token to the left or right of a token.

Both are done by placing a mask token and letting a pre-trained BERT base-uncased masked LM predict what goes there §2 app. A. The paper presents four attack modes §2:

• BAE-R: replace each token.
• BAE-I: insert a token to the left or right of each token.
• BAE-R/I: for each token, either replace it or insert next to it.
• BAE-R+I: replace the token, then insert next to it.

The steps are as follows.

1. Token importance. Each token's importance is the drop in the probability of the correct label when that token is deleted, following the approach of TextFooler and Ren et al. §2. The paper does not state whether the importance is recomputed after edits; alg. 1 computes it once on the original sentence.
2. Mask and predict. Visit tokens in descending importance. Mask the current position in the adversarial text and take the top K BERT-MLM predictions, with K = 50 §2 app. A alg. 1.
3. Filter. Keep candidates whose sentence-level similarity to the input, measured with the Universal Sentence Encoder, passes a threshold of 0.8 app. A. For R only, also keep candidates with the same part of speech as the original token §2 app. A. The paper also filters stop words (NLTK list) for R and I, and for sentiment tasks filters antonyms using synonym embeddings app. A.
4. Choose. If one or more candidates flip the prediction, pick the one whose full adversarial sentence is most similar to the original input. If none flip it, pick the candidate that lowers the probability of the true label most §2 alg. 1.
5. Stop. Stop when the label flips (success), or when every token has been processed (failure) §2 alg. 1.

Algorithm 1 is the full R procedure. Its pseudocode, in our words:

# BAE-R attack  [alg. 1]
# Input: tokens (list of n words), true label, classifier returning probs
# Output: adversarial token list, or None on failure

def bae_replace_attack(tokens, true_label, classifier):
    # Start the adversarial copy from the original sentence  [alg. 1]
    adv_tokens = list(tokens)
    # Rank positions by how much deleting each token lowers P(true_label)
    # [§2] [alg. 1]
    positions = rank_by_deletion_importance(tokens, true_label, classifier)
    for position in positions:
        # Mask the current position in the adversarial text  [alg. 1]
        masked = adv_tokens with mask at position
        # Take the 50 most likely fillers from BERT-MLM  [§2] [app. A]
        fillers = top_k_mlm_predictions(masked, k=50)
        # Keep fillers that keep the sentence close and the POS the same
        # [§2] [app. A]
        fillers = filter_by_use_and_pos(fillers, tokens, adv_tokens, position)
        # Build one candidate sentence per filler  [alg. 1]
        candidates = [replace(adv_tokens, position, f) for f in fillers]
        flipping = [c for c in candidates if classifier(c) != true_label]
        if flipping:
            # Return the flipping candidate closest to the original  [§2]
            return most_similar_to(flipping, tokens)
        # Otherwise keep the candidate that lowers P(true_label) most
        # and continue to the next position  [§2] [alg. 1]
        adv_tokens = most_damaging(candidates, true_label, classifier)
    # Every position tried without a flip  [§2] [alg. 1]
    return None

The pseudocode is our reading of alg. 1. In the source the Return sits in the success branch and the failure branch only updates the adversarial text, so the loop continues. The filter helper names are ours; the paper names the steps but not the helper.

Insert (BAE-I) uses the same loop, but the mask is placed next to the token on the left or right, and the paper does not state how that side is chosen §2. BAE-R+I runs the replace step and then the insert step at the same position §2. BAE-R/I chooses between replace and insert per token, and the paper does not state the rule for that choice §2.

The source has no numbered equations, so none are cited.

Hyperparameters

Not stated: batch sizes, learning-rate schedules and optimizer settings for the LSTM and CNN models, maximum sequence length, the number of training epochs for those models, random seeds, and any number of repeated runs.

Dataset

tab. 5 lists the seven datasets (test sets are small: 100 for Amazon, Yelp and IMDB):

Results

Author-reported. Setup: three target classifiers (word-LSTM, word-CNN, BERT base uncased) trained on each dataset's training split and attacked on its test split §3. The baseline is TextFooler §3. Each cell gives test accuracy in percent (average USE similarity between original and adversarial text in parentheses), as reported in tab. 1 for Amazon, Yelp, IMDB and MR, and in tab. 2 for MPQA, Subj and TREC.

Bold marks the lowest accuracy per model and dataset (the strongest attack), matching the source's boldface.

Headline claims, author-reported:

• Drops of 40 to 80% in test accuracy for BAE attacks across datasets and models, with higher average semantic similarity than TextFooler §3.
• Replacing and inserting can push BERT accuracy down by over 80% on some datasets with a few operations §1.
• BAE-R+I is the strongest attack, with one exception §3.
• BAE-I has the highest average semantic similarity of the four modes §3.
• Effectiveness vs. perturbation budget (TREC only in the body): the BAE attacks are consistently stronger than TextFooler; classifiers are fairly robust below 20% perturbation and the effect saturates at 40 to 50% §3 fig. 2. Full per-dataset curves are in the appendix app. B fig. 3 to fig. 8 (figure numbers inferred from source order).

Ablation: Replace vs. Insert

tab. 4. Percent of test instances that need a given operation for a successful R/I attack. A is attacked by R/I but not R; B by R/I but not I; C needs both R and I §3.

Human evaluation (BERT classifier)

tab. 3 and app. C. Three datasets, 100 test samples each, mixed with successful adversarial examples from BAE-R, BAE-R+I and TextFooler; three annotators §3. Naturalness is scored 1 to 5 on a Likert scale, where 1 is "sure adversarial" and 5 is "sure original" app. C. The source table also has an "Original" column, which is shown below.

Paper vs code

No official code.

Searched: the paper and its arXiv metadata link no repository. The paper points to an implementation inside TextAttack §3 (third-party framework). Via the GitHub repository search API I tried "BAE adversarial BERT", "bert-based adversarial examples text classification", "siddhant garg adversarial", "bae bert mlm attack" and "goutham ramakrishnan". The hits were not authors' repositories: a reproducibility project by a different owner, and an Arabic-language adaptation. Neither was used.

Pitfalls

• Test-set early stopping. The source states the models are trained first and that early stopping uses "the test dataset" app. A. Any reported accuracy on those splits may be optimistic. We have no validation split to check against.
• Small test sets. Amazon, Yelp and IMDB have 100 test points each tab. 5, so each point moves accuracy by about one percentage point. No variance, seeds or repeated runs are reported. Treat close comparisons as noise.
• BERT-R versus TextFooler. The text says TextFooler is "sometimes stronger" than BAE-R on BERT, and attributes it to the shared parameter space between the masked LM and the classifier §3. In the tables, TextFooler gives lower accuracy than BAE-R on six of seven datasets for BERT: all four sentiment sets plus MPQA and Subj tab. 1 tab. 2. BAE-R is lower only on TREC. The "almost always stronger" claim holds for word-LSTM and word-CNN, not for BERT-R.
• Baseline description is inconsistent. The TextFooler paragraph says the baseline uses top K = 50 predictions from BERT-MLM and a 0.8 USE threshold §3. TextFooler normally takes synonyms from a word-embedding space. The source does not say which synonym source the reported baseline used.
• Algorithm order and state. Importance is computed once on the original sentence, but the loop then edits the adversarial text in place. The source does not state how positions are kept consistent after an insert. Our inference is that an insert shifts later indexes alg. 1.
• Algorithm return logic. The printed algorithm returns only in the success branch and otherwise continues with the chosen candidate alg. 1. A literal implementation keeps a greedy chain of edits, not a search over alternatives. Our pseudocode follows that reading.
• Choice between flipping candidates. Among flipping candidates, the source chooses the one most similar to the original full input §2. Which similarity measure is used in the choice (USE, as in the filter) is implied but not stated in the algorithm.
• Insert side and R/I rule. How the left or right side is chosen for an insert, and how BAE-R/I picks between replace and insert for a given token, are not stated §2.
• Subword handling. How BERT's WordPiece subwords map back to words, and how a multi-token filler is inserted, are not stated.
• Filter scope. The POS filter applies to R only §2 app. A. The antonym filter applies to "sentiment analysis tasks" app. A, but the source does not list which of the seven datasets count, and the body describes Amazon, Yelp, IMDB and MR as the four sentiment tasks §3.
• Cost. The attack does one masked-LM pass per position and one classifier query per surviving candidate alg. 1. The source gives no timing or query counts; these follow from the procedure as written.
• Semantic similarity is a weak check. The authors say USE can rate BAE-R+I higher than BAE-R even though human raters rated BAE-R+I lower on naturalness and sentiment §3. Use the human table as the stronger signal, and note the human study covers only three datasets and the BERT classifier.
• Figure and table numbering. Figure and table numbers are as the source's LaTeX order gives them; the appendix refers to the human-evaluation table as "Table 3", which we follow.

Lineage

• Builds on: TextFooler (Jin et al., 2019) §1 §3; BAE's token-importance scoring follows TextFooler and Ren et al. §2.
• Uses: BERT base uncased as the masked LM and as one target classifier §2 §3 app. A; the Universal Sentence Encoder for similarity §2.
• Compared against: TextFooler §3; word-LSTM and word-CNN classifiers app. A.
• Related work named by the authors: contextualised masked-LM perturbations from Li et al. (2020), posted after the original submission §2. No card in this wiki.