Emergent Analogical Reasoning in Large Language Models

In plain words

The authors asked whether a large language model can solve analogy puzzles it was never trained on, with no examples and no extra training. They used GPT-3, a text-only model, and gave it four kinds of puzzle: grid puzzles that look like a well-known intelligence test but use digits, letter patterns such as "abcd changes to abce, so ijkl changes to ...", word pairs such as "cabbage is to vegetable as beetle is to ...", and short stories where one story matches another by cause and effect.

The same puzzles were given to university students. On most puzzle types the model did as well as or better than the average student. On the story puzzles it did worse than the students, and it was reliable only when the two stories used the same characters.

TL;DR

Text-davinci-003 (GPT-3) was given four analogy tasks with no examples in the prompt §2 §4.2. On a text version of Raven-style matrix problems it beat the average undergraduate, with odds ratio 1.88 for generated answers and 6.27 for multiple choice §2.1 fig. 3. On letter-string analogies and on three of four word-analogy sets it matched or beat the human average §2.2 §2.3 fig. 6 fig. 7. On story analogies it was weaker than students, with odds ratio 0.37 §2.4 fig. 8. All numbers are author-reported.

Problem

Analogy is a core test of reasoning, and deep networks have usually needed large task-specific training sets to do well on it §1. The paper asks whether a generic language model can solve analogy problems zero-shot, that is, with only a task description or none at all, and how its behaviour compares with human behaviour across task types §1 §2. The evidence is behavioural: the same item sets were run on the model and on student samples §4.3 §4.4 §4.5 §4.6.

Method

This is an empirical evaluation of a fixed model, not a new architecture or training method. It has three parts: building the four task sets (§4.3 to §4.6), querying GPT-3 through the OpenAI API with fixed settings (§4.2), and comparing model accuracy with human data collected by the authors or taken from earlier studies §2 §4.3.2 §4.4.3 §4.5 §4.6.2.

1. Model and settings §2 §4.2. The main model is text-davinci-003, which the authors describe as the first GPT-3 variant trained with reinforcement learning from human feedback §2. Temperature is 0 in all simulations, and the other parameters are OpenAI defaults §4.2.
2. Scoring multiple choice §4.2 §4.3.3. Each answer choice is added to the prompt, and the model's average token log-probability over the choice tokens is its score. The highest-scoring choice is the answer. Brackets are not counted.
3. Generative answers §4.3.3 §4.4.2 §4.6.3. The model writes the answer itself. Digit answers are cut at the closing bracket. Letter-string answers are cut at the closing bracket, or at a period in sentence format §4.4.2. Story answers are read from a forced-choice question.
4. Digit Matrices §2.1 §4.3. The problems are 3 x 3 grids of digits, emulating Raven's Standard Progressive Matrices. Each grid is written as bracketed cells, row by row, and the last cell is left open §4.3.3. Rules are constant, distribution-of-3 and progression for transformations, and OR, AND and XOR for logic problems §4.3.1 fig. 2. Each problem has eight answer choices: the correct one and seven procedurally made distractors §4.3.1.
5. Letter-string analogies §2.2 §4.4. Six transformation types (sequence extension, successor, predecessor, removal of a redundant letter, fixing a sequence, sorting) are combined with up to three generalisations (letters to numbers, grouping, longer target, reversed order, interleaved distractor, larger interval) §4.4.1 fig. 5. A real-world variant uses words such as cold, cool, warm, hot §4.4.1.
6. Four-term verbal analogies §2.3 §4.5. Four datasets are used: UCLA Verbal Analogy Test (80 items), Sternberg and Nigro (200), Turney's SAT set (374, five choices each) and Jones et al. (120). Each item is scored like the multiple-choice case above. Items are fed to the model in sequence, and each answer is added to the context for the next item §4.5.
7. Story analogies §2.4 §4.6. Eighteen source stories from Gentner et al. each have four targets: near and far analogies, each correct or incorrect §4.6.1. Each source is compared with two targets in both orders, so 72 comparisons in all, with the context cleared after each §4.6.3. A third option, "both equally analogous", is allowed §4.6.3.
8. Problem solving §2.5 §4.7. Gick and Holyoak's radiation problem and a gumball transfer task are given as single sessions, and the answers are read qualitatively §2.5 §4.7.

# Digit Matrices: prompt, generated answer and choice score  [§4.3.3] [§4.2]
# Input:  problem: 3 x 3 grid of digit lists; choices: 8 digit lists
# Output: generated answer text; score for each choice

def build_prompt(problem):
    # Write the first eight cells in brackets, one row per line  [§4.3.3]
    prompt = format_rows(problem, skip_last_cell=True)
    # Open the bracket of the missing ninth cell  [§4.3.3]
    return prompt + "["

def generated_answer(prompt):
    # Temperature 0, at most 10 tokens; cut at the closing bracket  [§4.2]
    text = complete(prompt, temperature=0, max_tokens=10)
    return text.split("]")[0]

def choice_score(prompt, choice):
    # Append the choice and a closing bracket, then score the choice tokens
    # Score is the mean log-probability, not the sum  [§4.2]
    prompt_with_choice = prompt + choice + "]"
    token_logprobs = score_tokens(prompt_with_choice)
    return mean(token_logprobs_of_choice(token_logprobs))

def predict(problem, choices):
    # The highest-scoring choice is the model's answer  [§4.2]
    prompt = build_prompt(problem)
    scores = [choice_score(prompt, c) for c in choices]
    return argmax(scores)

Key equations: none are numbered in the paper. Odds ratios and p-values come from logistic regression at trial level; the statistical tests are described in §4.3.4, §4.4.4 and §4.6.5.

Hyperparameters

Results

Author-reported. Setup: GPT-3 text-davinci-003, zero-shot, compared with student samples of 20 to 241 people, on four analogy task families §2 §4.3 §4.4 §4.5 §4.6. The paper reports no tables; results are in figures, and the numbers below are those given in the text. The baseline is the human average, unless the row says chance.

Paper vs code

Official code: the first author's repository taylorwwebb/emergent_analogies_LLM, linked in the paper's Data and Code Availability statement, at commit 145e72b5be6b845dfdc69450075f88930e9a661f. Its LICENSE file is a short notice that the code is licensed under Apache-2.0; code_license is recorded from that notice. I checked the following against the code under raw/code/2212_09196/:

• Model text-davinci-003, temperature 0, and max_tokens 10 (Digit Matrices and verbal), 40 (letter strings) and 256 (story) match §4.2 eval_gpt_matprob.py eval_GPT3_letterstring_prob.py eval_GPT3_story_analogies.py.
• The letter-string prompt "Let's try to complete the pattern:" matches §4.4.2 eval_GPT3_letterstring_prob.py. The Digit Matrices code writes the last cell open, as §4.3.3 describes eval_gpt_matprob.py.
• The story comparison prompt matches §4.6.3, and each pair is run in both orders eval_GPT3_story_analogies.py.
• Eight answer choices per Digit Matrix problem match §4.3.1 eval_gpt_matprob.py.

Discrepancies:

• Digit Matrices stop rule. The paper says generated answers are cut at the closing bracket §4.3.3. The code also sets stop to a newline eval_gpt_matprob.py. The paper does not mention this.
• Letter strings, sentence format. The paper says responses are cut at a period §4.4.2. The sentence-format call sets no stop sequence eval_GPT3_letterstring_prob.py. The cut is presumably made in the analysis code, which I did not check.
• Story access route. The paper says the story stimuli were entered "directly into the OpenAI playground" §4.6.3. The code calls the completions API with the same settings eval_GPT3_story_analogies.py. The paper also says GPT-4 was run through the ChatGPT web interface §4.6.4; the repository holds gpt4_data.csv, which I did not check.
• The statistical analysis scripts (R and Python) and the human data pipelines were not checked against the paper's numbers.

No other discrepancies found in the parts checked.

Pitfalls

• Numbering follows the arXiv v3 LaTeX source. The LaTeX footnote says the paper was published in Nature Human Behaviour (2023). The journal version may number sections, figures and methods differently. Check the journal version before quoting numbers.
• The model is not fully specified. The paper names text-davinci-003 and the OpenAI API, but gives no API snapshot date and does not say whether the model can still be queried §4.2. Current models cannot be read off these results §4.2 [suppl. "GPT-3 model variants"].
• Results live in figures. The paper has no tables. Per-condition values appear only in figures, and the text gives only selected numbers fig. 3 fig. 6 fig. 7 fig. 8. The author repository holds the data files, which would be needed for exact values.
• Deterministic, single-run scoring. Temperature 0 means one answer per item. The 20 runs of the progression experiment are the only repeated runs §4.3.3. Variance across runs is not reported for the main results.
• Mean versus sum log-probability. Choices are scored by mean token log-probability §4.2. Choices of different length are therefore compared on an average, not a total. The paper does not discuss this choice.
• Human samples differ by task. The student samples are UCLA undergraduates for UCLA VAT, Yale undergraduates for Sternberg and Nigro, and Wayne State undergraduates for Jones et al. §4.5. The verbal results text instead describes the human comparison group for UCLA VAT and Sternberg and Nigro as US high-school graduates recruited on Amazon Mechanical Turk §2.3. The two descriptions disagree.
• Excluded participants. Three of 43 participants were excluded from the first Digit Matrices experiment as producing apparently random answers §4.3.2. The paper does not state a pre-set exclusion rule for the other experiments. Sample sizes were not pre-determined §4.3.2 §4.4.3 §4.6.2.
• Story chance level. The three-option story task is tested against 0.5 "to be conservative" §4.6.5. The chance level for a three-option task is not the same, so this test is lenient for the model.
• Story results are mixed. GPT-3 was not significantly above chance on far analogies §2.4. Students picked the analogous story on every trial in 15 of 54 cases §2.4. The GPT-4 result is a preliminary check on one task §2.4 [suppl. "GPT-3 model variants"].
• Contamination is argued, not excluded. The authors found no evidence of memorisation, but the verbal and story materials appear in a public supplement to an earlier paper. The paper says it cannot rule out exposure [suppl. "Presence of test materials in GPT-3's training data"].
• Context and length limits. In the progression experiment the prompt grew with each answer. Early problems were dropped once it exceeded GPT-3's 4,096-token window §4.3.3.
• Problem solving is qualitative. The analogical problem-solving results are single sessions with no counts, and the authors call them preliminary §2.5 §4.7.
• Text-only input. The Digit Matrices are text, not images, so they test rule induction without the visual parsing used in Raven's matrices §2.1. The paper suggests that physical reasoning limits the problem-solving results §2.5.

Lineage

• Builds on: GPT-3 (Brown et al., 2020) for the model family, and the instruction-tuned variant text-davinci-003 §1 §2. The Digit Matrices follow the rule structure of Raven's Standard Progressive Matrices and the dataset design of Matzen et al. (2010) §2.1 §4.3.1. The problem-solving paradigm is Gick and Holyoak (1980) §2.5.
• Compared against: human samples from earlier work, namely Lu et al. (2019) for UCLA VAT, Sternberg and Nigro (1980), Turney (2003) for SAT, Jones et al. (2022), and Gentner et al. (1993) for stories §4.5 §4.6.1.
• Superseded by: not stated in this paper. The authors report a preliminary GPT-4 check on story analogies only §2.4.