Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?

In plain words

A language model can be used as a measure of how surprising each word is in a sentence. Psycholinguists use that surprise score to predict how long people pause on each word while reading. The usual assumption is that a better language model gives a better predictor.

This paper finds the opposite for a family of modern models. The larger models, which predict the next word more accurately, give surprise scores that explain people's reading times less well. The authors check this with statistical models on two reading datasets, using several families of models.

The paper then looks at where the larger models go wrong. They tend to give very low surprise to nouns, named entities and adjectives. People take longer on those words than the models suggest. For such words the models' predictions are more accurate than human reading times suggest. The authors argue that this makes large models a weaker baseline for studies of human language processing.

TL;DR

• Across five GPT-Neo and eight OPT variants (and GPT-2), the larger models with lower perplexity give surprisal that fits reading times less well, on Natural Stories and Dundee §3.4 fig. 1.
• The poorer fit is mostly from underprediction of reading times on nouns, named entities and adjectives, where larger models give very low surprisal §4.4 fig. 3 fig. 4.
• The authors' practical advice is not to pick the largest pre-trained LM by default, and to include surprisal from smaller variants in baselines §5.
• All results are author-reported and rest on regression models fitted with lme4 §3.3.

Problem

Surprisal from a language model is a common predictor of reading difficulty §1. An earlier study showed the reverse trend for GPT-2 sizes, but only for four variants and with no significance testing §1. Trained-from-scratch models in earlier work gave a negative relation between perplexity and fit §2. The open question is whether the reversed trend holds across model families, and which words cause it §1 §5.

Method

The study has two parts. Both use the same data and regression set-up.

1. Predictive power of surprisal §3. Compute word surprisal for each model variant on each story or article. Fit a baseline mixed-effects model with low-level predictors, and one full model per variant that adds that variant's surprisal. The gain in log-likelihood over the baseline (ΔLL) is the measure of fit §3.3. Plot ΔLL against each family's perplexity and fit a least-squares line per family fig. 1 §3.4.
2. Residual analysis §4. Refit the same models without the by-word random intercepts §4.1. Compute residuals (target minus prediction) on the exploratory half of the data §4.1. Group the data points by linguistic properties from annotated syntactic trees §4.2. For each group, measure the mean squared error (MSE) of each family's models against that family's perplexity, and keep the groups with the steepest negative slope §4.3. Remove those points and repeat §4.3. Then split each group into under- and overpredicted points and compare the sum of squared errors (SSE) and the average surprisal §4.3 fig. 4.

Word surprisal is the sum of the token surprisals of a word's subword pieces §3.2. Token surprisal is the negative log probability of the token given the earlier tokens §3.2.

# Per-token surprisal with overlapping context windows  [§3.2]
# Input:  tokens (BPE ids of one story); lm; ctx (context size in tokens)
# Output: one surprisal per token, -log P(token | earlier tokens)

def token_surprisals(tokens, lm, ctx):
    surprisals = []
    # Index of the first token that has not been scored yet  [§3.2]
    scored_until = 0
    while scored_until < len(tokens):
        # The second half of the previous window is the first half of
        # the new one, so it serves only as context  [§3.2]
        window_start = max(0, scored_until - ctx // 2)
        window_end = min(window_start + ctx, len(tokens))
        # Score only the tokens that are new in this window  [§3.2]
        for position in range(scored_until, window_end):
            context = tokens[window_start:position]
            prob = lm.prob(tokens[position], context)
            surprisals.append(-log(prob))
        scored_until = window_end
    return surprisals

Key definitions, stated in the text (equations are not numbered):

S(w_t) = -\log P(w_t \mid w_{1..t-1}) §3.2

\Delta LL = LL_{\text{full}} - LL_{\text{baseline}} §3.3

Driving-subset search, run separately for each corpus and family (the paper does not give this step a number; it is described in prose in §4.3):

# Iterative slope-based search for driving subsets  [§4.3]
# Input:  residual data points for one corpus and one LM family;
#         subsets of points defined by annotation properties  [§4.2]
# Output: up to five subsets, in the order they were found

def find_driving_subsets(corpus, subsets, mse_of, log_ppl, n_top=5):
    found = []
    remaining = corpus
    for _ in range(n_top):
        # Only subsets with more than 1% of the corpus points  [§4.3]
        eligible = [s for s in subsets
                    if count_in(s, remaining) > 0.01 * len(corpus)]
        # Least-squares slope of each subset's MSE against log perplexity,
        # over that family's variants; keep the steepest negative one  [§4.3]
        best = steepest_negative_slope(eligible, remaining, mse_of, log_ppl)
        found.append(best)
        # Exclude the chosen subset's points before the next round  [§4.3]
        remaining = [p for p in remaining if p not in best]
    return found

Hyperparameters

Results

Author-reported. Setup: Natural Stories (self-paced reading) and Dundee (eye-gaze go-past durations), with GPT-2, GPT-Neo and OPT surprisal. The figures carry the plotted values, which this card does not transcribe.

Paper vs code

The code is the authors' repository, linked from the footnote in §1 §1 fn.. Its owner is the first author, and the README gives his contact address. The repository has no LICENSE file, so it is cited only. The README says the code accompanies arXiv 2402.02255, a later paper, not this one README.md.

• Scope. The repository has only the surprisal extraction scripts and the unigram surprisal script. It has no code for the lme4 regression, the filtering and partitioning of §3.1, the residual analysis, the annotation or the subset search README.md get_llm_surprisal.py. Only the surprisal calculation can be checked against the code.
• Log base. The paper gives surprisal as −log P without naming the base §3.2. The code computes base-2 surprisal (bits) [code: get_llm_surprisal.py#L128], and §4.4 quotes gaps in bits §4.4. Because predictors are centred and scaled §3.3, a change of base should not change the fitted models, but the bit values in §4.4 are base-2 values.
• Context window. The code takes the context size from the model config get_llm_surprisal.py. Windows overlap by half and the second half of a window seeds the next one get_llm_surprisal.py, which matches §3.2. The paper's sizes (1,024 and 2,048) are not checked against the config values here.
• Beginning-of-sequence token. For GPT-2, GPT-Neo and Pythia, the code prepends the BOS id, so it takes one slot of each window get_llm_surprisal.py. For OPT it does not add one and relies on the tokenizer default get_llm_surprisal.py. The paper does not say how the sequence start is handled.
• Word surprisal. The code sums token surprisals until the decoded text equals the next whitespace-separated word get_llm_surprisal.py. Punctuation attached to a word, such as "England,", is part of that word. This matches §3.2 but is not stated there.
• Model list. The code also lists Pythia checkpoints [code: get_llm_surprisal.py#L15-L19]. This paper uses GPT-2, GPT-Neo and OPT only §3.2.
• Unigram surprisal. The unigram script uses Pile counts and a Pythia tokenizer get_unigram_surprisal.py [code: get_unigram_surprisal.py#L31]. The paper does not describe unigram surprisal, so its use in this study is not stated.
• Search for other code. Searching GitHub was not needed: the footnote gave the official repository.

Pitfalls

• Log base. Keep one base for all predictors and quote bit values only with base-2 surprisal. The paper does not name the base in its formula §3.2.
• Window pseudocode versus the real computation. The pseudocode above calls the model once per token for clarity. The code runs one forward pass per window and reads the outputs for all positions [code: get_llm_surprisal.py#L120-L128]. The two should give the same scores (not run here); the practical difference is one forward pass per window, not per token.
• Numerical range. The code applies softmax and then takes log2 of the probability get_llm_surprisal.py get_llm_surprisal.py. An implementation that works from log-softmax directly avoids underflow for very improbable tokens. This was not tested here.
• Sequence start. The paper says nothing about BOS handling. The code differs between families (see Paper vs code). Use the same rule for all models in a comparison.
• Word boundaries. Words are whitespace-separated items, so punctuation stays attached. This changes which tokens are summed into a word.
• Mixed-effects fitting. The paper does not say whether the lme4 fits converged, what optimiser was used, or how fits that failed were handled. Random slopes for every fixed effect with 181 subjects is a large model.
• Likelihood versus MSE. In the Dundee data, the by-word intercepts made likelihood and MSE disagree §4.1. The residual analysis drops them, but the main ΔLL results keep them §3.3 §4.1. Compare the two sets of results with that in mind.
• Subset selection. The residual analysis runs on the exploratory half only, and the held-out half is kept for significance tests §4.1 fn.. The driving subsets are chosen by steepest slope with no held-out test reported. The per-family slope is fitted over only four, five or eight points.
• Threshold denominator. The paper says a subset must hold more than 1% of the data points in each corpus §4.3. It does not say whether the total is the full corpus or the points left after exclusion. The pseudocode uses the full corpus.
• Mechanism. The memorisation explanation is a hypothesis drawn from other work (Carlini et al., 2022, cited in §5). The paper's own evidence shows the pattern, not its cause.
• Model size confound. Within a family the variants mainly differ in capacity §3.2. Across families the training data differ, and the paper attributes the OPT versus GPT-2 difference partly to that §3.4. The paper also notes that GPT-J and GPT-NeoX differ architecturally from GPT-Neo but are grouped with it §3.2 fn..
• Corpus differences. Natural Stories uses by-subject-by-sentence intercepts and Dundee uses sentence intercepts §3.3. Results on the two corpora are not run under identical random-effects structures.
• Scope. The paper studies pre-trained models. The authors say trained-from- scratch models may form a different regime §5. Do not extend the finding to other model types without that caveat.

Lineage

• Builds on: Oh et al. (2022), which found the reversed trend for GPT-2 only §1. No card for that paper.
• Compared against: Goodkind and Bicknell (2018), Wilcox et al. (2020), Merkx and Frank (2021), which report a negative relation for trained-from-scratch models §2.
• Related: Oh et al. (2021), a character model in a parser, and Kuribayashi et al. (2021, 2022), on Japanese and on recency bias §2 §5.
• Superseded by: no known follow-up in this wiki. The code README cites arXiv 2402.02255, a later paper by the same authors, which is not read here README.md.