Annotation Artifacts in Natural Language Inference Data

In plain words

Large datasets for "does sentence A imply sentence B?" were built by showing crowd workers one sentence and asking them to write three new sentences: one that must be true, one that might be true, and one that must be false. The authors found that the written sentence often gives the answer away. A simple text classifier that never reads the first sentence does much better than guessing, so the scores on these benchmarks overstate how well models understand the relation.

The paper also looks at the habits that crowd workers fall into, such as replacing specific words with general ones, adding a reason clause for the "might be true" sentences, and using negation for the "false" ones. When strong models are tested on the examples that the simple classifier got wrong, their scores are clearly lower. The authors conclude that the task is still open and call for harder benchmarks.

TL;DR

• SNLI and MultiNLI hypotheses carry label cues: a bag-of-words-and-bigrams fastText classifier, given only the hypothesis, reaches 67.0% on SNLI and 53.9% / 52.3% on MultiNLI matched / mismatched tab. 2.
• Cue words are tied to labels: negation with contradiction, purpose clauses and modifiers with neutral, generic words and sentence shortness with entailment §3.1 §3.2 tab. 4.
• On the hard subset (examples the hypothesis-only model misclassifies), three NLI models score far below their full-test-set results §4 tab. 5.
• No official code. The method is a classifier plus a split rule, which you can reimplement from the pseudocode below.

Problem

Natural language inference (NLI) asks whether a premise sentence entails a hypothesis sentence §1. Large NLI datasets were built by having crowd workers write one entailed, one neutral and one contradicting hypothesis per premise §1. The authors argue that the way these hypotheses were written leaves label-revealing patterns in the text itself, so that a model can score well without comparing the two sentences §1 §2. Without a check of this kind, high benchmark scores can be read as evidence of inference when they reflect those patterns §4 §5.

The analysis covers SNLI and MultiNLI (matched and mismatched test sets) §2 tab. 2. Section 5 cites similar biases in other tasks and datasets from prior work §5.

Method

The paper has three parts: a premise-oblivious baseline, a lexical and length analysis, and a re-evaluation of strong models on an easy/hard split.

1. Premise-oblivious classifier §2. Train a classifier on hypotheses only, with the premise dropped. The main model is fastText, which represents text as a bag of words and bigrams §2. Two other premise-oblivious classifiers gave similar results: a logistic regression on word and character n-grams, and a decomposable-attention model with the premise removed §2, fn. 2. The classifier is trained on the training split and evaluated on the test split tab. 2.

# Premise-oblivious classifier  [§2] [tab. 2]
# Input:  train_pairs: list of (premise, hypothesis, label)
# Output: model that maps a hypothesis string to a label

def train_hypothesis_only(train_pairs):
    # Keep the hypothesis text; the premise is never used  [§2]
    texts = [hypothesis for (_, hypothesis, _) in train_pairs]
    labels = [label for (_, _, label) in train_pairs]
    # fastText-style bag of words and bigrams  [§2]
    # MultiNLI only: add character 4-grams and drop rare words  [§2, fn. 1]
    return fit_text_classifier(texts, labels)

2. Lexical choice §3.1. For each word and class, compute point-wise mutual information on the training set, using add-100 smoothing on the raw counts. Rank words per class by this score and report the top five with the share of class sentences that contain them tab. 4. The equation is unnumbered in the source, so cite it as §3.1.

\text{PMI}(w, c) = \log \frac{p(w, c)}{p(w, \cdot)\; p(\cdot, c)} §3.1

3. Sentence length §3.2 fig. 1. Compare the token-count distribution of hypotheses per class. In SNLI, neutral hypotheses are longer and entailed ones are shorter. The median neutral hypothesis has 9 tokens, and 60% of entailed hypotheses have 7 tokens or fewer §3.2. Half of hypotheses with at least 12 tokens are neutral, and a similar share of hypotheses with 5 tokens or fewer are entailed §3.2. The paper also tests whether the hypothesis is a bag-of-words subset of its premise: 8.8% of SNLI entailed hypotheses are, against 0.2% of neutral and contradicting ones §3.2. The length effect is weaker in MultiNLI, which the authors attribute to its genre mix §3.2.

4. Easy/hard re-evaluation §4. Split each test set with the premise-oblivious model. Examples it classifies correctly are "easy"; the rest are "hard" §4. Train each NLI model on the original training set of the same dataset, then evaluate on the full, hard and easy test sets §4.

# Easy/hard split of one test set  [§4]
# Input:  test_pairs: list of (premise, hypothesis, label)
#         oblivious: model from train_hypothesis_only
# Output: easy_pairs, hard_pairs

def split_test_set(test_pairs, oblivious):
    easy_pairs = []
    hard_pairs = []
    for (premise, hypothesis, label) in test_pairs:
        # Predict from the hypothesis alone  [§4]
        guess = oblivious.predict(hypothesis)
        # Correct guess goes to easy, any other to hard  [§4]
        if guess == label:
            easy_pairs.append((premise, hypothesis, label))
        else:
            hard_pairs.append((premise, hypothesis, label))
    return easy_pairs, hard_pairs

The NLI models are DAM §4, ESIM §4 and DIIN §4. The authors retrain them out of the box and do not state any training values §4.

Hyperparameters

Results

Author-reported. Setup: SNLI and MultiNLI (matched and mismatched). The baseline is the majority class; the classifier is the hypothesis-only fastText model §2 tab. 2. The metric is not named in the table; the abstract describes it as the share of examples classified correctly [abstract].

Most discriminative words per class and dataset, ranked by PMI. The percentage is the share of training sentences in that class containing the word tab. 4:

Re-evaluation of strong NLI models on the full, hard and easy test sets. Author-reported. The table's metric is not named; the columns are the split names from §4 §4 tab. 5:

The authors read these numbers as showing that most of the models' success comes from easy examples §4 tab. 5.

Paper vs code

No official code. I searched the paper and its LaTeX source for repository links. The only repositories it names are implementations of other models: AllenNLP for DAM and the NYU MultiNLI repository for ESIM §4. They are not this paper's code, so they are not used. A GitHub repository search for "annotation artifacts", "gururangan snli" and "hypothesis only nli" found no repository by the authors. Hypothesis-only repositories by other authors belong to different papers and are not used.

Pitfalls

• The easy/hard split depends on gold labels. An example is "easy" only when the hypothesis-only model matches the gold label §4. The hard set is therefore a function of the test labels and of one trained classifier. Do not reuse the split as an unbiased training filter without accounting for this.
• Which premise-oblivious model defines the split is not stated. §2 names fastText and two alternatives §2, fn. 2, while §4 says only "the premise-oblivious model" §4. Assume fastText only if you need a faithful reproduction, and say so.
• Filtering to hard examples does not remove artifacts. The authors show that dropping easy examples can leave a new skew, for instance the word "no" moving into the neutral and entailment classes §4. They suggest balancing artifacts across labels as a better direction §4.
• Training settings for the NLI models are unstated. Models were retrained out of the box §4, with no epochs, batch sizes or learning rates given. Your numbers will not match Table 5 unless you use the original code settings.
• Hyperparameters for the classifier are mostly unstated. Apart from the MultiNLI-only features and the frequency filter §2, fn. 1, nothing is given for fastText, so the baseline is hard to match exactly.
• PMI is sensitive to smoothing. Smoothing is aggressive (add 100) and the authors chose it to emphasise strongly discriminative words §3.1. Other constants will change the rankings and the percentages in Table 4.
• Length statistics are described, not tabulated. Figure 1 gives the per-class distributions, but the exact values beyond the medians and cumulative shares in §3.2 are not given fig. 1 §3.2.
• Section and table numbers here follow the source's order. Numbers are taken from the order of \section and captioned tables in the LaTeX source, not from a compiled PDF. If a citation looks off by one, check the compiled paper.
• The paper argues against overreading the numbers, not for a method. It makes no claim that a hypothesis-only model is a good NLI system. The 67.0% figure is a diagnostic, not a benchmark result §2 §5.

Lineage

• Builds on: SNLI (Bowman et al., 2015), the crowdsourced dataset analysed in §1–§4 §1. MultiNLI (Williams et al., 2017), the genre-diverse follow-up §1. fastText (Joulin et al., 2017), the classifier used in §2 §2.
• Compared against: decomposable attention (Parikh et al., 2016) §2, fn. 2 §4; ESIM (Chen et al., 2017) §4; DIIN (Gong et al., 2017) §4.
• Related work in §5: artifact findings in SICK, CNN/DailyMail, ROC stories, SQuAD, visual question answering and InferSent §5. The paper also names a parallel study of hypothesis-only biases across NLI datasets §5.
• Superseded by: not stated.