Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

In plain words

Large language models can do a new task when you paste a handful of worked examples into the prompt, for instance a few movie reviews each followed by "positive" or "negative", then a new review with the label left blank. This is called in-context learning. The natural assumption is that the model studies the examples and learns which inputs go with which labels.

This paper tests that assumption by deliberately lying to the model: the labels in the examples are replaced with random ones. Surprisingly, accuracy barely moves, across a dozen models up to GPT-3. So the model is mostly not learning the input-to-label rule from the examples; it already knows it from pretraining.

What the examples do provide, the authors find, is context: what kind of text to expect, which answer words are allowed, and the "input, then answer" pattern to copy. Swap the inputs for unrelated sentences, or the labels for random English words, or drop the pairing pattern, and performance falls much more. A model trained specifically to do in-context learning (MetaICL) relies on these shallow cues even more strongly.

For practitioners the takeaway is twofold. Unlabeled inputs paired with random labels already get close to the usual few-shot result. And in-context learning is unlikely to teach a model a task whose input-label relationship it did not pick up during pretraining.

TL;DR

An empirical study, not a new method. Using k=16 demonstrations, the authors compare no demonstrations, demonstrations with gold labels, and demonstrations whose labels are drawn uniformly at random from the label set, on 26 datasets with 6 decoder-only LMs (GPT-2 Large, MetaICL, GPT-J, fairseq 6.7B/13B, GPT-3), each with direct and channel scoring. Random labels lose only 0-5% absolute. Further variants isolate four aspects of the prompt (input-label mapping, input distribution, label space, pair format) and show the last three matter far more than the first. Relevant if you build few-shot prompts, design ICL evaluations, or need a stronger zero-shot baseline that uses only unlabeled inputs.

Problem

In-context learning (conditioning an LM on k input-label pairs and then a test input) beats zero-shot inference on many tasks, but it was not understood how the model uses the demonstrations or which of their properties produce the gain §1. Prior analyses were theoretical (ICL as Bayesian inference over latent concepts) or correlational (term frequency in pretraining data); the authors describe theirs as the first empirical study of why ICL beats zero-shot §2. Scope: classification and multi-choice tasks with real natural-language inputs only; generation is not studied [Limitation].

Method

All comparisons hold the model, the k training inputs and the prompt template fixed and change only one aspect of the demonstrations. Methods are written for the direct model; the channel model is obtained by swapping x and y §4.1, footnote.

Inference methods §3. Each LM is used with direct scoring, argmax over y in C of P(y | demos, x), and channel scoring, which models the input given the label (following Min et al., arXiv:2108.04106). C is the small set of candidate labels (for multi-choice, the answer options).

Main conditions §4.1:

1. No demonstrations: \arg\max_{y \in \mathcal{C}} P(y \mid x).
2. Demonstrations with gold labels: \arg\max_{y \in \mathcal{C}} P(y \mid x_1, y_1, \dots, x_k, y_k, x).
3. Demonstrations with random labels: each x_i is paired with \tilde{y}_i sampled uniformly from \mathcal{C}; predict \arg\max_{y \in \mathcal{C}} P(y \mid x_1, \tilde{y}_1, \dots, x_k, \tilde{y}_k, x).

Four aspects of a demonstration §5 fig. 7: (1) input-label mapping, whether each x_i has its correct label; (2) distribution of the input text x_1 \dots x_k; (3) label space covered by y_1 \dots y_k; (4) format, i.e. presenting inputs and labels as pairs.

Variants (which aspects each keeps: F format, L label space, I input distribution, M mapping) §5.1–5.3 fig. 8–10 tab. 4:

Algorithm 1, forming demonstrations with a% correct labels alg. 1 app. B:

# Demonstrations with a% correct labels  [alg. 1]
# Input:  k gold pairs [(x_1, y_1) ... (x_k, y_k)], label set C, a in [0, 100]
# Output: list of k (input, label) pairs

def form_demonstrations(pairs, label_set, a):
    demos = []
    # number of pairs that keep their gold label  [alg. 1]
    num_correct = k * a / 100
    # choose which positions stay correct, uniformly without replacement  [alg. 1]
    correct_positions = sample(range(1, k + 1), num_correct)
    for i in range(1, k + 1):
        x_i, y_i = pairs[i]
        if i in correct_positions:
            demos.append((x_i, y_i))
        else:
            # any label except the gold one  [alg. 1]
            demos.append((x_i, sample(label_set - {y_i})))
    return demos

Prompt construction and scoring, as implemented for GPT-3 in the official code gpt3.py (the GPT-2/MetaICL/GPT-J path lives in the MetaICL codebase, not this repo):

# Build per-option prompts and pick the lowest-loss option
# [code: gpt3.py#L23-L52] [code: gpt3.py#L76-L105]

def predict(demos, test_input, options, method):
    context = ""
    for x_i, y_i in demos:
        # direct: input then label; channel: label then input  [app. B]
        # newline inside a pair, three newlines between pairs  [app. B]
        if method == "direct":
            context += x_i + "\n" + y_i + "\n\n\n"
        else:
            context += y_i + "\n" + x_i + "\n\n\n"
    losses = []
    for option in options:
        if method == "direct":
            prompt = context + test_input + "\n"
            target = option
        else:
            prompt = context + option + "\n"
            target = test_input
        # drop tokens from the left so prompt + target <= 1024 tokens
        # [code: gpt3.py#L54-L61]
        prompt = left_truncate(prompt, target, max_tokens=1024)
        # mean negative log-prob over the target's tokens
        # [code: gpt3.py#L88-L94]
        losses.append(mean_token_nll(prompt, target))
    return options[argmin(losses)]

Random labels in the code: each demonstration's label is replaced by a uniform draw from that example's own option list (the label set for classification, the example's answer choices for multi-choice) create_data.py.

Hyperparameters

Models tab. 1:

Evaluation data: 16 classification and 10 multi-choice datasets, all with under 10K training examples, evaluated on the HuggingFace development split §3 app. A tab. 2. Ablations use 5 classification (MRPC, RTE, Tweet_eval-hate, SICK, poem-sentiment) and 4 multi-choice (OpenbookQA, CommonsenseQA, COPA, ARC) datasets §4.2. Fairseq 13B and GPT-3 use MRPC, RTE, Tweet_eval-hate, OpenbookQA, CommonsenseQA, COPA §3.

Dataset statistics tab. 2:

Results

Author-reported. The per-model numbers appear only as bar charts fig. 3 fig. 11; the paper's text gives the aggregate figures below. Default setup: k=16, minimal templates, 5 seeds §3.

Caveats the authors flag: direct GPT-2, direct GPT-J and direct fairseq 6.7B are not much better than chance on many classification datasets, and channel fairseq 13B does better with no demonstrations than with gold ones; these are discussed less §4.1, footnote.

Paper vs code

• Only data creation and GPT-3 inference are in the repo. Inference for GPT-2, MetaICL and GPT-J (test.py), the metaicl package and k-shot data building (_build_gym.py) come from merging the MetaICL repository (README). Fairseq models are not supported ("contact authors", README). So the scoring details below are verified only for the GPT-3 path.
• Scoring is length-normalized. The paper writes argmax of P(y | …) §4.1; the GPT-3 code picks the option with the lowest mean per-token negative log-likelihood of the target gpt3.py gpt3.py. For channel, the target is the test input, conditioned on the option gpt3.py.
• OOD length matching. Paper: sentences of "similar length to the test input" app. B. Code: for each test example a fresh set of k sentences is drawn without replacement with probability proportional to exp(-(Δ word count)^2 / 50), and they keep the original demonstrations' (random) labels in order create_data.py. So OOD demonstrations differ per test input.
• Random English words. For classification, one bijection gold label → random word is drawn per seed, then demo labels are drawn uniformly among the new words, while the test options stay the original labels create_data.py create_data.py. For multi-choice, each demo gets fresh, distinct random words for its options create_data.py. The paper's single \mathcal{C}_\mathrm{rand} with \lvert\mathcal{C}_\mathrm{rand}\rvert = \lvert\mathcal{C}\rvert §5.2 matches only the classification case.
• a% correct. Code computes the correct count with integer division, k * a // 100 create_data.py; with k=16 all tested values are exact.
• No labels sets each demo label to the empty string and labels only sets each input to the empty string create_data.py create_data.py; how the separators then render is decided by the MetaICL formatting code, which is not in this repo.
• Extra variants in the code not reported in the paper: permutated_labels, random_english_words_gold_labels create_data.py create_data.py.

Pitfalls

• This is not a recipe for dropping labels in general. The gap is measured on classification and multi-choice with label words the LM already understands; the authors note synthetic tasks may rely on correct labels more, per-dataset gaps reach nearly 14% (financial_phrasebank, GPT-J), and later work cited in the paper found negated labels hurt substantially [Limitation] app. C.2.
• "Random" means uniform over C, not a shuffle. Uniform sampling can unbalance labels relative to the training distribution; sampling from the true label distribution narrows the gap further app. C.2. Implement whichever matches the experiment you are reproducing.
• Incorrect-label ablation samples from C minus the gold label, so at a=0 every label is wrong, which differs from random labels (where some are correct by chance) alg. 1.
• Direct and channel behave differently. Channel models are nearly insensitive to the label space (English-word labels cost 0–2%), direct models lose 5–16% §5.2; MetaICL direct ignores the input distribution §5.1. Do not generalize an ablation result from one scoring method to the other.
• Format changes hide inside "content" ablations. A constant label or a repeated input acts like a separator and changes the format, which is why those variants score worse app. C.3. When designing your own ablations, keep the pair structure intact.
• Weak baselines distort the picture. Several direct models are near chance on classification and channel fairseq 13B prefers no demonstrations §4.1, footnote; macro-averages include these.
• Seeds and averaging. Report per-dataset mean over seeds, then the macro-average over datasets §3; fairseq 13B and GPT-3 numbers use only 6 datasets and 3 seeds and are not comparable to the 26-dataset averages (use fig. 11 for cross-model comparison).
• Minimal templates are deliberate. The paper uses the raw dataset label words (e.g. not_entailment) and multi-field inputs joined by [SEP] tab. 3; manual templates were not consistently better app. B.
• Context length. With k=16 and 256-token examples, prompts hit the 1024-token cap and the GPT-3 code drops tokens from the left, i.e. the earliest demonstrations gpt3.py test_gpt3.py.
• Code license. The repository has no LICENSE file; treat it as all rights reserved and do not copy code from it.

Lineage

• Builds on: in-context learning as introduced with GPT-3 (arXiv:2005.14165); direct vs. channel scoring from Min et al. (arXiv:2108.04106); the MetaICL model, datasets and k-shot splits (arXiv:2110.15943).
• Compared against: no prior method; the baselines are no-demonstration and gold-label ICL with the same models §4.1.
• Related analyses named by the paper: ICL as implicit Bayesian inference (Xie et al.), correlation with pretraining term frequencies (Razeghi et al.), and demonstrations as task location (Reynolds and McDonell) §2 §6.