Larger language models do in-context learning differently

In plain words

When you show a language model a few examples in its prompt, it can seem to learn the task on the spot. Two things could explain this: it might recognise the task from what it already knows about language, or it might work out the rule that links inputs to answers from the examples themselves. This paper separates the two by changing the example labels. It flips them so they contradict common sense, or replaces them with meaningless words such as "foo" and "bar".

The main finding is that small models keep following what they already know, while large models follow the examples. A large model will obey a prompt where positive reviews are labelled "negative", and it can still learn a task when the labels carry no meaning at all. Both abilities appear only as models grow. Tuning a model on instructions makes it better at learning from the examples, but it also makes it hold more firmly to its prior knowledge.

TL;DR

• Measures how in-context learning uses prior knowledge (semantic priors) against the input-label pairs shown in the prompt §1.
• With flipped labels, only large models override their priors; this is reported as emerging with scale §3.
• With meaningless target words, large models keep most of their accuracy, and small models drop sharply §4.
• Instruction tuning improves learning from the examples, but strengthens priors more §5. Results are author-reported on 100 evaluation examples per dataset §2.3.

Problem

Few-shot prompting is often explained by semantic priors: the model already knows the task, and the examples only show the format. Earlier work found that random ground-truth labels in the prompt barely hurt performance, which supports that view §7.1. Other work shows that simple transformers trained from scratch can learn linear-regression-like rules in context §7.2. The paper asks what happens with pretrained language models when the examples contradict priors, or when labels carry no semantic content §1, §3, §4.

Without a control like flipped or unrelated labels, a high score cannot distinguish "knew the task" from "learned the mapping". The paper's experiments are built to separate these cases. Its chance-level reference of 50% holds only for binary tasks §3.

Method

This is an empirical study, not a training method. The paper defines three prompt settings and measures accuracy across model families and sizes.

Regular ICL. Prompts contain k exemplars per class with natural-language targets, followed by one evaluation input §2.3 app. D.

Flipped-label ICL §3. For a binary task, a chosen fraction of the exemplar labels is replaced by the opposite label. The evaluation labels are not changed. Accuracy below 50% means the model follows the flipped exemplars rather than its priors. At 50% flipped, the labels are no better than random §3.

Semantically-unrelated label ICL (SUL-ICL) §4. Natural-language targets are replaced by unrelated words, by default "Foo" and "Bar" (for example, negative becomes "foo" and positive becomes "bar" for SST-2). Learning is possible only through the input-label mapping in the exemplars §4.

Instruction tuning comparison §5. PaLM is compared with its instruction-tuned version Flan-PaLM, in the same ICL and flipped-label experiments. InstructGPT is not compared with GPT-3, because the base models are not known to match §5.

Linear classification §6. Points in N dimensions are labelled by a hidden linear threshold. Exemplars are drawn on both sides of the threshold, and the model must classify a new point. The generation procedure is given below alg. 1, app. C.4.

Exemplars are sampled separately for each evaluation example, from the training split, and evaluation examples come from the validation split §2.3 app. A.

# Flipped-label ICL  [§3]
# Input:  exemplars as (text, label) pairs for a binary task; flip_fraction
# Output: exemplars with a fraction of labels replaced by the other label

def flip_labels(exemplars, flip_fraction):
    # Choose the exemplars whose labels are flipped, at random  [§3]
    flip_count = round(flip_fraction * len(exemplars))
    flipped = random_sample(range(len(exemplars)), flip_count)
    result = []
    for index, (text, label) in enumerate(exemplars):
        if index in flipped:
            # Swap to the other class; evaluation labels stay unchanged  [§3]
            label = other_class(label)
        result.append((text, label))
    return result

# SUL-ICL prompt  [§4] [app. D]
# Input:  exemplars (text, label) with natural labels; evaluation text
# Output: prompt string

def build_sul_prompt(exemplars, eval_text, natural_to_foo_bar):
    # Map each natural-language label to a meaningless target  [§4]
    lines = []
    for text, label in exemplars:
        target = natural_to_foo_bar[label]
        lines.append("Input: " + text)
        lines.append("Output: " + target)
    # The evaluation input ends the prompt; the model writes the target
    # [app. D]
    lines.append("Input: " + eval_text)
    lines.append("Output:")
    return "\n\n".join(lines)

# Linear classification data generation  [alg. 1, app. C.4]
# Input:  n (dimension), k (exemplars per class)
# Output: training exemplars, evaluation point and its label

def generate_eval(n, k):
    # Ground-truth coefficients and a pivot point  [alg. 1]
    coefficients = random_vector(n)
    pivot = random_vector(n)
    # Threshold separating positive and negative examples  [alg. 1]
    threshold = dot(coefficients, pivot)
    train_x = []
    train_y = []
    for _ in range(k):
        # Positive point: score above the threshold  [alg. 1]
        positive = random_vector_above(coefficients, threshold)
        # Negative point: score at or below the threshold  [alg. 1]
        negative = random_vector_below(coefficients, threshold)
        train_x += [positive, negative]
        train_y += [1, -1]
    eval_x = random_vector(n)
    if dot(eval_x, coefficients) > threshold:
        eval_y = 1
    else:
        eval_y = -1
    return train_x, train_y, eval_x, eval_y

Coordinates are random integers from 1 to 1000 §6 app. C.4. The prompt uses SUL-ICL targets: negative examples are "Foo" and positive examples are "Bar" app. D.

Hyperparameters

Results

Author-reported. Evaluation uses 100 examples per dataset, across seven binary or multi-class tasks §2.1 §2.3. Headline numbers are from the text and appendix.

Overall, the paper's central claim is that flipped-label override and SUL-ICL both appear only above a certain scale in the tested families §3 §4 §6. The exact threshold depends on the model family and task app. C.1. For example, all GPT-3 models are classed as small, and davinci behaves like a small model in the SUL setting §3 app. C.2.

Paper vs code

No official code. The paper does not link a code repository in the abstract or the TeX source, and no official repository was found on GitHub. Only the paper's own LaTeX source was used to write this card.

Pitfalls

• Unknown model sizes. The authors use OpenAI API models and Google PaLM. The sizes of InstructGPT and Codex are not public, and the authors only assume they increase in size §2.2. Scale claims for those families rest on that assumption.
• Binary chance level. 50% flipped labels equal random for binary tasks. TREC has six classes and is excluded from the flipped-label experiments §3.
• Context-length exclusions. RTE is excluded for PaLM because prompts at k=16 per class exceed the context window app. C.1. Some sweep lines stop early app. C.2. Results on those datasets are therefore not comparable across models.
• Small evaluation sets. 100 evaluation examples per dataset, with no reported confidence intervals or repeated runs. Several claims rest on gaps of a few points, such as the 9.6 points in SUL-ICL §5. Some datasets (FP, ETHOS, WSC) have fewer than 250 examples and use all available ones app. B.3.
• Output outside the label set. Small models sometimes answer with text that is not a label, so accuracy under flipped labels is not always 100% minus the reported accuracy app. B.1.
• Flan-PaLM one-shot. Flan-PaLM scores near 0% with one exemplar per class on many datasets, and the authors note that this does not happen with two or more. The cause is not known app. C.3.
• Dataset choices. FP uses the sentences_allagree subset, ETHOS uses the binary subset, and TREC uses the six coarse labels app. A. QNLI and WSC were dropped from the main experiments app. B.3.
• Semantic label words. "0" and "1" are not fully neutral, because they correlate with sentiment. The authors judge the effect small app. B.3.
• Remapping. Remapping input words hurts all but the largest models, and even the largest fail at 100% remapping, except TREC, which the authors could not explain app. B.2.
• Semantic templates. Semantically relevant templates reduce accuracy for the smallest Codex model. Only SST-2, TREC and RTE were tested, on Codex only app. B.5.
• Linear classification. Neither PaLM nor Codex beat an SVM baseline app. C.4. The scaling factor behind Codex's behaviour is not identified.
• Decoding and prompting details. Temperature, decoding method, maximum output length and random seeds are not stated, so exact reproduction with the same API models may not be possible.

Lineage

• Builds on: in-context learning as cited to GPT-3 §1 and the semantic-prior view from earlier label-randomisation work §7.1.
• Compared against: the random-label findings of min2022rethinking §1 §7.1 and the implicit linear-learning results of Garg et al. and follow-up transformer work §7.2.
• Related: instruction tuning studies §5, emergent-ability work on scaling §7.3.
• Superseded by: not stated in this card.