Using cognitive psychology to understand GPT-3

In plain words

The authors wanted to know whether a large language model thinks like a person. They borrowed experiments that psychologists have used for decades to study human judgement, and ran them on GPT-3, a language model that predicts the next word. The experiments cover how people make choices between gambles, how they look for information, how they balance trying new options against sticking with known good ones, how they plan in multi-step problems, and how they reason about cause and effect.

GPT-3 often gives answers that look human, and sometimes better than people. It makes the same kinds of mistakes people make on some classic puzzles. But small rewordings of the same puzzle can push it far off course. It also does not try new options on purpose the way people do, and it fails a cause-and-effect task that people handle.

The point is less the single score than the method: a model's behaviour can be measured with the same tools used for people, and the comparison shows where its apparent reasoning is shallow.

TL;DR

• Ran classic cognitive-psychology experiments on GPT-3 through its text API, at temperature 0 § Results.
• Vignette tasks: 6 of 12 answered correctly, and all 12 gave human-like answers; small rewordings changed the answers § Results / Problems with vignette-based investigations.
• Task-based tests: good at gambles and at a bandit task, shows model-based planning, but no directed exploration and fails the causal task § Results.
• Author-reported, single model, no variance estimates across repeated runs; the code does not reproduce the vignette results. See Pitfalls.

Problem

Benchmarks report what a model gets right, not how it gets there. The authors argue that a model solving a task does not show it reasons like a person, and that vignette-style questions may have appeared in the training data § Introduction § Results / Problems with vignette-based investigations. Their question is whether tools from cognitive psychology give a more informative picture of GPT-3's decision-making, information search, deliberation and causal reasoning § Introduction.

Method

The paper runs two kinds of experiment on GPT-3 § Introduction:

1. Vignette-based: short textual scenarios from the literature (for example the Linda problem, cab problem, hospital problem, Wason card task, cognitive reflection test, Blicket and mature-causal-reasoning items) are entered as prompts, and the answer is classified as correct, or as a mistake that people commonly make § Results / Vignette-based investigations fig. 1A. Where there were two options, a third option that is neither correct nor plausibly chosen by people was added § Results / Vignette-based investigations. The vignette prompts and answers are in the SI, not in the main text [SI: Vignettes].
2. Task-based: experiments are generated programmatically, and the model answers trial by trial. The trial history is updated with the model's earlier answers and the feedback it received § Results / Task-based investigations.

Sampling for all runs: the OpenAI API, the most capable GPT-3 model ("Davinci"), temperature 0 for deterministic answers, and default values for all other parameters § Results, paragraph after fig. 1 (see Pitfalls on the engine id and temperature).

Decisions from descriptions § Results / Decision-making: Decisions from descriptions

Each problem shows two gambles, each a list of outcomes with probabilities (options F and J) fig. 2A. The model's choice is compared with the expected value of each option. The regret of a choice is the expected value of the best option minus the expected value of the chosen option § Results / Decision-making: Decisions from descriptions.

# Regret of one choice  [§ Results / Decisions from descriptions]
# option_values: expected value of each option, e.g. (F, J)
# chosen: index of the option the model picked
def regret(option_values, chosen):
    # Best expected value minus the expected value of the choice made
    best = max(option_values)
    return best - option_values[chosen]

• Problems: over 13,000 gamble pairs from an existing benchmark § Results / Decision-making: Decisions from descriptions fig. 2B.
• Four GPT-3 sizes compared (the "Davinci" model and three smaller ones) § Results / Decision-making: Decisions from descriptions.
• Cognitive biases: the authors tested six contrasts from Kahneman & Tversky using the choice probabilities of the model, as log-odds ratios between paired problems [SI: Decisions from descriptions] fig. 2C.

Information search: horizon task § Results / Information search: Directed and Random Exploration

A two-armed bandit with forced-choice trials and then free-choice trials fig. 3A. The horizon is either 1 or 6 free choices § Results / Information search: Directed and Random Exploration. The first four trials are forced and give either equal information (two observations per arm) or unequal information (one observation from one arm, three from the other) § Results / Information search: Directed and Random Exploration.

Behavioural classification in the first free-choice trial § Results / Information search: Directed and Random Exploration:

• Equal-information condition: choosing the arm with the lower estimated mean is random exploration.
• Unequal-information condition: choosing the less-observed arm is directed exploration.

Analysis: one logistic regression per information condition, with predictors reward difference, horizon, their interaction and a bias term § Results / Information search: Directed and Random Exploration.

Environment as coded (true means and noise are not given in the paper):

# Horizon task episode  [fig. 3A] [§ Results / Information search]
# Two arms; one arm has mean 40 or 60, the other is shifted by a gap
def sample_episode():
    # Free-choice rounds after the forced ones: 1 or 6  [§ Results]
    horizon = random_choice([1, 6])
    # Better-arm mean is 40 or 60; the other arm is shifted by a gap
    # drawn from a fixed list of values  [code: HorizonTask/envs/bandits.py#L35-L38]
    better_mean = random_choice([40, 60])
    gap = random_choice([-30, -20, -12, -8, -4, 4, 8, 12, 20, 30])
    # Each reward is the mean plus Gaussian noise, rounded  [code: HorizonTask/envs/bandits.py#L40]
    # noise sd is 8  [code: HorizonTask/envs/__init__.py#L6]
    return horizon, better_mean, gap

Forced-choice sequence bandits.py:

# Four forced trials, shuffled  [code: HorizonTask/envs/bandits.py#L52-L60]
def forced_arms(unequal):
    if unequal:
        # One arm seen once, the other three times
        arms = random_choice([[0, 0, 0, 1], [0, 1, 1, 1]])
    else:
        # Two observations per arm
        arms = [0, 0, 1, 1]
    return shuffle(arms)

Deliberation: two-step task § Results / Deliberation: Model-based and model-free reinforcement learning fig. 4

Two first-stage actions (spaceship to planet X or Y) lead to a second stage with two aliens each. The spaceship reaches the selected planet with probability 0.7 and the other planet with 0.3 § Results / Deliberation: Model-based and model-free reinforcement learning. Each alien pays treasure with some probability; these start uniform between 0.25 and 0.75 and drift slowly § Results / Deliberation: Model-based and model-free reinforcement learning.

# One simulated run: 20 trials  [§ Results / Deliberation] [fig. 4B]
def simulate_run(num_trials=20):
    # Treasure probability for each of the four aliens  [§ Results]
    # [code: TwoStepTask/query.py#L29]
    alien_probs = uniform(0.25, 0.75, size=4)
    for trial in range(num_trials):
        # The model chooses planet X or Y, given the history so far
        first = ask_model(stage_one_prompt(history))
        # Common transition 0.7, rare 0.3  [code: TwoStepTask/query.py#L16-L20]
        planet = transition(first)
        # The model chooses one of the two aliens on that planet
        second = ask_model(stage_two_prompt(planet))
        # Treasure with the alien's current probability
        treasure = bernoulli(alien_probs[second])
        # Slow random drift, kept inside [0.25, 0.75]
        # [code: TwoStepTask/query.py#L80-L81]
        alien_probs = clip(
            alien_probs + normal(0, 0.025, size=4),
            0.25,
            0.75,
        )

Analysis: stay probability of the first-stage action, split by transition (common or rare) and outcome (treasure or junk) § Results / Deliberation. The paper reports 200 simulations § Results / Deliberation.

Causal reasoning: wine casks § Results / Causal reasoning: Interventions after passive observations fig. 5

The model sees 20 casks with the presence or absence of substances A, B and C (observational data), plus a causal structure: A causes B and A causes C in the common-cause condition, and B causes A which causes C in the causal-chain condition § Results / Causal reasoning fig. 5A. It is then asked how many of 20 new casks will contain C, under an observation (B is known to be present) or an intervention (B is added by hand) § Results / Causal reasoning fig. 5A.

The normative answer for the intervention differs from the observation only in the common-cause condition. Intervention cuts the arrows into the intervened variable (Pearl's do-operator) § Results / Causal reasoning.

Hyperparameters

Results

Author-reported. Model is GPT-3 "Davinci" through the OpenAI API at temperature 0 unless stated § Results, paragraph after fig. 1. Human comparisons are from the cited literature. No confidence intervals beyond the reported statistics.

The paper's headline claims, as summarised in the Discussion: GPT-3 solves many vignettes, outperforms humans in the bandit task, shows model-based signatures in the two-step task, and fails the causal task § Discussion.

Paper vs code

Official code: https://github.com/marcelbinz/GPT3goesPsychology at commit c1dcdaf309e9b2b40bc5610d1772f2a27282f88c (default branch, shallow clone). The owner is marcelbinz, the first author. The repository has no LICENSE file, so its license is unknown and no code is quoted verbatim below; each item is a pseudocode paraphrase with a citation. Its README links a preprint with the same title (not checked against the arXiv version).

Searched: paper text and the arXiv source (no code link); the GitHub search for the title and for the author's repositories found this one. The repository contains scripts for the decisions-from-descriptions, horizon, two-step and causal experiments and their analyses. It contains no code for the vignette experiments (Linda, cab, hospital, Wason, CRT, Blicket, mature causal) and no code for the SI contrast tables beyond the 17 problems in the query script.

Discrepancies:

• Temperature. Paper: temperature 0 for all runs § Results, paragraph after fig. 1. Code: two-step queries use temperature 1 query.py. The SI says log-odds analysis used temperature 1 [SI: Decisions from descriptions], but the query scripts use 0.0 query.py query_choice13k.py. Which setting produced each reported number is unclear.
• Engine. Paper: "Davinci" with no version. Code: text-davinci-002 query.py for every task, and text-ada-001, text-babbage-001, text-curie-001 for the smaller models analyze_performance.py.
• Horizon regressor. Paper: regression on "estimated reward difference" § Results / Information search. Code: the true generative mean difference from the environment analyze_choice.py analyze_choice.py analyze_choice.py, not the subject's own estimate from observations.
• Hard-coded plot values. The causal figure bars for GPT-3 and the ideal and human values are typed in, not computed from the query outputs analyze.py. The model-based means for the two-step figure are also typed in analyze_modelbased.py. The human DFD replication probabilities are typed in analyze_contrast.py. These numbers cannot be reproduced from the repository's data alone.
• Causal dataset. The paper does not list the 20 casks. The code uses a fixed table of 20 rows, shuffled before each prompt query_common.py. The chain condition uses the same table query_chain.py.
• Causal parsing. The code asks for a number with max_tokens=100 and suffix casks. query_common.py. The paper does not say how the answer was parsed into a count.
• Two-step drift. The paper says only "drift slowly". The code uses a Gaussian step with sd 0.025, clipped to [0.25, 0.75] query.py.
• Two-step statistics. The t-tests pool trials across the 200 runs as independent samples analyze.py analyze.py. The degrees of freedom in the paper (1982, 1814) match this pooling.
• Invalid answers. For the 13k task, if the answer is not in the top two log-probabilities, the code sets the missing option's log-probability to -9999 query_choice13k.py. The paper does not mention this rule.

No discrepancies found for the decisions-from-descriptions problem wording, the horizon prompt text, or the two-step transition probabilities and trial counts (200 simulations, 20 trials).

Pitfalls

• Model identity. The paper says "Davinci" with no version, and the code names text-davinci-002 query.py. Engine versions change over time; the reported numbers cannot be reproduced exactly with a different or retired engine. Not stated in the paper beyond the family name.
• Temperature is not one setting. Treat temperature 0 as the paper's default, but check each experiment against its script (two-step at 1, SI log-odds analysis at 0 in code). See Paper vs code.
• Single run per item. Each experiment is one deterministic answer per prompt at temperature 0. The paper reports no run-to-run variance for the vignettes or the single-prompt tasks, and no seeds. Only the two-step task has repeated simulations.
• Prompt sensitivity is the paper's own main caveat. Small rewordings changed vignette answers (cab problem, Wason card task, CRT, Blicket) § Results / Problems with vignette-based investigations. Results from the task-based tests may change with the generating program § Discussion.
• Contamination. The vignettes are famous psychology problems and may be in training data § Results / Problems with vignette-based investigations. The paper cannot rule this out.
• Horizon parameters. Reward means, the gap set and the noise sd (8) are not in the paper; they come from the environment code bandits.py [code: HorizonTask/envs/init.py#L6]. An implementer reproducing the task must take these from the code.
• Regression variable. Use the environment's true mean difference, as the code does, not the model's estimate, or the coefficients will not match analyze_choice.py.
• Pooled tests. Trial-level pooling makes p-values optimistic if trials within a simulation are correlated. The paper does not address this.
• Causal failure may depend on phrasing. The paper reports only the cask setup; the causal result is one table with one wording query_common.py.
• Figures are not reproducible from the text alone. For the causal and two-step figures the plotting script uses typed-in values (see Paper vs code).
• Licensing. The repository has no license, so only paraphrased pseudocode is given here; do not copy code from it.

Lineage

• Builds on: GPT-3 (Brown et al., 2020) is the model under study; the introduction also cites GPT-2 § GPT-3. No arXiv id is given in this card, and neither paper is in the wiki.
• Compared against: human data from the cited psychology literature, e.g. Peterson et al. (decisions from descriptions), Wilson et al. (horizon task), Daw et al. (two-step task), Waldmann & Hagmayer (causal cask task) § Results.
• Superseded by: not stated in the paper.