Concept · 2 papers · updated 2026-11-15
Benchmark contamination
How to tell whether a model has already seen a benchmark or a text it is tested on, so a score measures skill rather than recall. Four ways to measure it: data overlap for open models, an extractability ratio for closed models, a name-cloze memory probe, and web-prevalence counts as a prior. Each suits different access. Results are author-reported and not validated.
evaluationmemorizationdata-contamination
Markdown5 min read
AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
Overview
A score is only evidence of skill if the test items were not in the training data. If they were, the model may be recalling them. The shared problem is that closed models do not publish their training data, so the check has to be indirect 2310.18018 §1 2305.00118 §1.
The position paper defines three kinds of contamination: the annotation guidelines were seen, the raw text before annotation was seen, or the labels of the benchmark were seen. The effect of each depends on which split was exposed 2310.18018 §3. It places the risk at three stages: pre-training, supervised and instruction tuning, and user data gathered after deployment 2310.18018 §4.1 §4.2 §4.3.
The second paper runs a memory probe on two OpenAI chat models. It shows that books differ widely in how well the models know them, and that this knowledge follows how often the text appears on the web. Its authors connect this to evaluations of fiction, where memorised text can distort results 2305.00118 §1 §6.2 §8.
Variants
Benchmark data overlap (open models)
Measure what share of the benchmark appears in the model's public pre-training corpus, using a string match. This needs the corpus to be searchable, and it gives a direct count only for that corpus 2310.18018 §5.1. Prior audit work on open corpora uses the same kind of search, and the paper names it as the usual approach 2310.18018 §5.1. The paper does not say whether the match is exact or uses n-grams, so the rule must be fixed and reported 2310.18018 §5.1. Card: NLP Evaluation in trouble.
Extractability ratio (closed models)
Give the model the k examples that come just before each benchmark example, and check whether it reproduces that example. The contamination degree is the share of examples reproduced 2310.18018 §5.2. The method is adapted from the extractability definition of Carlini et al. (2023), which the paper cites but does not reproduce 2310.18018 §5.2. Non-verbatim outputs are judged by a human reviewer, so the step is manual and two auditors may disagree 2310.18018 §5.2. Failing to reproduce an example does not settle the question 2310.18018 §5.2. The paper has no measured rates; its only demonstration is three qualitative cases, in which the opening lines of a standard NER training split come back from three models app. A fig. 1 fig. 2 fig. 3. Card: the same position paper.
Name-cloze memory probe (closed chat models)
Take a 40 to 60 token passage that contains exactly one person name, mask the name, and ask a chat model to fill the blank. The passages contain no other named entities, so a reader who only has the passage cannot guess the name 2305.00118 §3.1 §3.2. The score for a book is the share of its sampled passages answered correctly. Among fiction, GPT-4 reaches 0.244 on pre-1923 public-domain novels, against 0.072 for ChatGPT, and scores fall to 0.02 to 0.03 for most contemporary categories 2305.00118 tab. 2. Chance-level baselines are a name-frequency guess at 0.6 percent and one annotator at 0 percent 2305.00118 §3.2. Note that the matching rule and the temperature are not stated in the paper. This variant measures recall of a text, not a benchmark score, so it is a probe for a suspected-seen corpus. Card: Speak, Memory.
Web prevalence as a prior (any model)
Count exact hits for short spans of each text in web-scale indexes, then correlate the counts with name-cloze accuracy. For books published before 1928 the Spearman correlation is 0.70 to 0.84 across the four indexes; after 1928 it falls to 0.21 to 0.41 2305.00118 tab. 3 §6.2. This needs search access and no access to the model. The authors describe the link as a correlation, and they argue that popularity drives both prevalence and memorisation 2305.00118 §6.2. Use the counts to rank texts by likely exposure, not to decide that a model saw a specific text. Card: the same archaeology paper.
Comparing groups by memorisation (downstream effect)
Split the texts into the most and least memorised deciles, then compare the model on a related task across the two groups. On year-of-publication estimates, GPT-4's mean absolute error is 0.6 years in the top decile and 14.5 years in the bottom decile 2305.00118 tab. 5. A narrative-time task does not show a significant difference between those groups 2305.00118 §7.2. The grouping uses the same accuracy the comparison then tests, so the comparison is not fully independent 2305.00118 §7.1.
When to use which
| Situation | Use | Why |
|---|---|---|
| Open weights and a public pre-training corpus | Benchmark data overlap 2310.18018 §5.1 | Direct count against the data; fix the match rule first |
| Closed API model and a benchmark to judge | Extractability ratio 2310.18018 §5.2 | Needs only generation access, but a human audits non-verbatim output and no threshold is given 2310.18018 §5 |
| Closed model and a narrative or literary text | Name-cloze probe 2305.00118 §3.1 | Works on any text with named characters; the card's method needs chat access and about $400 at 2023 prices 2305.00118 §5.1 |
| No model access, only search access | Web prevalence as a prior 2305.00118 §6.2 | Ranks texts by exposure; correlational only |
| Split of the benchmark is unknown | Check which split was exposed 2310.18018 §3 | Evaluation-split exposure invalidates the test; train or dev exposure breaks zero-shot and few-shot claims |
| Deciding whether memorisation matters for another task | Downstream comparison by memorisation group 2305.00118 tab. 5 | Shows the size of an effect on a related task, not a cause |
Implementation sketch
The variants share one core: build (context, target) pairs, query the model, and count matches. Each variant changes how the pairs are built and what counts as a match.
# Recall rate: share of items the model reproduces [arxiv:2310.18018 §5.2]
# Input: model (callable), items (list of (context, target)), matches (fn)
# Output: fraction of scored items recalled, in [0, 1]
def recall_rate(model, items, matches):
# Extractability: context = k preceding examples [arxiv:2310.18018 §5.2]
# Name-cloze: context = passage with the name masked [arxiv:2305.00118 §3.2]
hits = 0
for context, target in items:
# Ask the model and take its raw reply [arxiv:2305.00118 §3.2]
reply = model(context)
# The match rule is a choice: exact string, normalised, or n-gram
# [arxiv:2310.18018 §5.1] [arxiv:2305.00118 §3.2]
if matches(reply, target):
hits = hits + 1
# Denominator: items with a context; the first k have none
# [arxiv:2310.18018 §5.2]
return hits / len(items)Contradictions
Low recall is read differently by the two papers, and the disagreement is about interpretation, not the numbers. The position paper says that a model which does not reproduce a benchmark may still have trained on it, and calls this a question for further research 2310.18018 §5.2. The memory study treats the bottom decile of name-cloze accuracy as texts the model has not memorised, and uses that group as the unseen comparison 2305.00118 §6.1. Neither paper resolves this, so a low score cannot by itself be read as clean.
Scope differs too. The position paper concerns benchmark data; the memory study measures books and draws the benchmark link as an argument 2310.18018 §1 2305.00118 §8.
Papers
- Speak, Memory: name-cloze probe on 571 works; memorisation tracks web prevalence and affects year estimates.
- NLP Evaluation in trouble: position paper defining three contamination types and two measures (overlap for open models, extractability for closed models); no experiments.
Source and license
Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.