Towards Expert-Level Medical Question Answering with Large Language Models

In plain words

Doctors' exams and consumer health questions are a hard test for a language model: the answer has to be correct, and it also has to read as safe and useful to a patient. This paper takes a strong general-purpose language model and teaches it medicine in two ways. First, it is trained further on medical question and answer data. Second, it is asked to answer with its reasoning written out, and then a second pass reads several of those reasoning attempts together and writes a better final answer.

The authors measure the result in two ways. On multiple-choice exam questions the model scores higher than earlier systems on several benchmarks. On open consumer questions, doctors and lay raters judge its written answers on a set of quality axes, and they usually prefer them to answers written by physicians. The authors say that these results are a step towards expert-level answers, not proof that the model is ready for clinical use.

TL;DR

• Med-PaLM 2 is a medical LLM built on PaLM 2 §3.2. It is instruction-tuned on MultiMedQA training data §3.2 tab. 3.
• Its new prompting method, ensemble refinement, samples 11 chain-of-thought answers and conditions 33 refined answers on them, then takes a plurality vote §3.3 fig. 2.
• Author-reported: 86.5% on MedQA (USMLE), a new state of the art at publication §4.1 tab. 4.
• For long-form questions, physician raters preferred its answers to physician-written answers on 8 of 9 axes in pairwise ranking §4.2.
• No official code. The base model and training details are not described enough to reproduce.

Problem

Med-PaLM passed the USMLE-style passing mark, but its written answers still trailed physicians' answers on the quality axes the authors used §1. Earlier multiple-choice benchmarks measure knowledge, but not whether a long answer is safe, complete and aligned with clinical expectations §1. The authors also note that multiple-choice gains do not show real-world clinical value, and that their evaluation is one step in that direction, not a settled method §4.1 §6.

Method

Base model and instruction tuning. Med-PaLM 2 builds on PaLM 2 §3.2. Instruction tuning follows the protocol of Chung et al. (2022) §3.2. The training data are the training splits of MultiMedQA: MedQA, MedMCQA, HealthSearchQA, LiveQA and MedicationQA §3.2. The unified model uses the dataset mixture in tab. 3. The authors state that the mixture and the choice of datasets were set empirically §3.2. A second variant is tuned only on multiple-choice questions and is used for the best MedQA result §3.2 §4.1.

Few-shot and chain-of-thought prompting. Few-shot prompts are the same as those in Singhal et al. (2022) §3.3. Chain-of-thought (CoT) exemplars give a step-by-step explanation before the answer §3.3 app. A.3.1.

Self-consistency. The model samples 11 CoT explanations and answers, and the most common answer wins §3.3.

Ensemble refinement (ER). ER runs in two stages §3.3 fig. 2:

1. Given a few-shot CoT prompt and a question, the model writes 11 generations by sampling. Each generation holds an explanation and an answer.
2. The model is conditioned on the original prompt, the question and all 11 stage-1 generations. It writes a refined explanation and answer. The authors run this stage 33 times and take a plurality vote over the 33 answers.

The authors describe ER as a generalisation of self-consistency, which votes over answers directly, while ER lets the model weigh the explanations it has already written §3.3. ER is applied only to multiple-choice questions in this paper, because of its cost §3.3.

# Ensemble refinement for one multiple-choice question  [§3.3] [fig. 2]
# Input:  question, cot_prompt (few-shot chain-of-thought exemplars)
# Output: one answer option

def ensemble_refinement(question, cot_prompt):
    # Stage 1: sample 11 explanation + answer generations  [§3.3]
    # The sampling temperature is not stated.
    first_stage = []
    for _ in range(11):
        first_stage.append(sample_model(cot_prompt, question))
    # Stage 2: condition on prompt, question and all stage-1 generations,
    # then sample a refined explanation + answer, 33 times  [§3.3]
    refined = []
    context = cot_prompt + question + concatenate(first_stage)
    for _ in range(33):
        refined.append(sample_model(context))
    # Final answer: plurality vote over the stage-2 answers  [§3.3]
    return most_common(answer_of(refined))

PubMedQA prompting. For PubMedQA, the best strategy on the development set used 3-shot prompts without CoT. The exemplars were drawn at random from the training split for each evaluation question, with updated instructions app. A.3.2.

Long-form answers. Answers come from prompts per dataset, taken from Singhal et al. (2022) and not tuned for performance app. A.3.4. Long-form answers were sampled at temperature 0.0 §3.5.

Overlap analysis. A question counts as overlapping if the whole question, or at least 512 contiguous characters, appear in any document of the base model's training corpus. Answer options are excluded from the search. Questions whose answers are not in the corpus still count as overlapping §3.4.

Human evaluation. Physicians wrote one answer per question, with no time limit and access to references, for lay readers §3.5. Raters scored each answer independently on the rubric of Singhal et al. (2022) §3.5. In pairwise ranking, raters picked the preferred answer, or a tie, on nine axes §3.5. The axes are: medical consensus, reading comprehension, knowledge recall, reasoning, irrelevant content, omitted information, demographic bias, harm extent and harm likelihood §3.5.

Statistics. Confidence intervals use 10,000 bootstrap iterations. Hypothesis tests are two-tailed permutation tests with 10,000 iterations, blocked by answer when an answer has several ratings §3.5.

Hyperparameters

Results

Author-reported. Setup: MultiMedQA multiple-choice benchmarks, with the best strategy per column; the base model is PaLM 2; the baselines are Flan-PaLM and GPT-4 as cited in the table §4.1. Model size, hardware and training cost are not stated. Bold values are the paper's own marks for the best result in a row tab. 4.

Med-PaLM 2 with each prompting strategy, unified model tab. 5:

Overlap with the base model's training corpus, 512-character rule. Delta is without-overlap minus with-overlap, so a negative value means the model did better on overlapping questions tab. 6 §4.1:

Long-form results, author-reported, with the paper's own figure numbers:

• Adversarial sets, physician raters: Med-PaLM 2 was rated significantly higher than Med-PaLM on every axis, p < 0.001 §4.2 fig. 3.
• MultiMedQA 140, physician raters: against Med-PaLM, the only significant differences favoured Med-PaLM 2, on evidence of reasoning, incorrect knowledge recall and incorrect reasoning, at p < 0.05 §4.2 fig. 3.
• MultiMedQA 140, lay raters: Med-PaLM 2 was rated more helpful and more relevant than Med-PaLM, p ≤ 0.002 §4.2 fig. 4.
• Pairwise, MultiMedQA 1066 (physicians): Med-PaLM 2 was preferred to physician answers on 8 of 9 axes, p < 0.001. The exception was inaccurate or irrelevant content §4.2 fig. 1 fig. 5.
• Pairwise, against Med-PaLM: on the same 8 axes Med-PaLM 2 was preferred. On inaccurate or irrelevant content it was 18.4% vs 21.5%, not significant, p = 0.12 §4.2.
• Consensus: Med-PaLM 2 answers better reflected medical consensus in 72.9% of comparisons with physician answers §1.
• Harm: 90.6% of Med-PaLM 2 answers were rated low risk of harm, against 79.4% for Med-PaLM §1.
• Length: median answer length on MultiMedQA 140 was 794 characters for Med-PaLM 2, 565.5 for Med-PaLM and 337.5 for physicians. On adversarial questions the medians were 964 and 518 for Med-PaLM 2 and Med-PaLM §4.2.
• Inter-rater agreement: on MultiMedQA 140, 10 of 12 alignment questions had κ > 0.8, and the other two had κ > 0.6 §3.5 fig. A.1.

Paper vs code

No official code. Searched: the paper's source and arXiv metadata for code links (none); the paper's project pages (none linked); a GitHub repository search for "med-palm" (the top 18 of 64 results were not by the authors' organisation); a search for "med-palm" within org google-research (no results); and a search for "ensemble refinement medical" within org google (no results).

Pitfalls

• Not reproducible from the text. The base model is PaLM 2, and its size and training are not described here. Learning rate, steps, batch size, optimiser and the instruction-tuning data order are not stated §3.2. An implementer can reproduce the prompting layer, but not the model. Use a comparable base model and record that the numbers will differ.
• "Best" mixes models and prompts. The 86.5% MedQA result comes from the MedQA-only variant, not the unified model. The unified model with ER gets 85.4% §3.2 §4.1. The "best" column in tab. 4 also takes the best prompting method per row, so it is not one configuration.
• PubMedQA 81.8% is tuned. The best PubMedQA prompt was chosen on the development set, and the 81.8% uses 11-sample self-consistency. The test set has 500 questions. The authors also attribute much of the remaining error to label noise §4.1.
• "Over 19%" is absolute. The abstract's gain is 86.5 minus 67.2, that is 19.3 percentage points, not a relative gain. The relative gain is about 28.7% (our calculation from the abstract's figures).
• ER cost is hidden. Each question takes 11 stage-1 samples and 33 stage-2 samples, and every stage-2 prompt contains all 11 stage-1 generations §3.3. Sampling temperature is not stated for either stage, and it is not stated whether stage-1 answers enter the final vote.
• ER is untested on long-form. The authors apply ER only to multiple-choice questions because of its cost. They suggest it could aggregate long-form answers, but do not test that §3.3.
• Mixture counts do not match ratios. MedQA (10,178) and MedMCQA (182,822) have the same 37.5% share. The table does not say how share maps to sampling tab. 3.
• Overlap analysis is conservative but partial. Questions whose answers are absent from the training corpus still count as overlapping. The corpus is not described. The threshold changes the picture: MedQA overlap is 0.9% at 512 characters and 11.15% at 120 characters, and 48.0% of MMLU medical genetics questions overlap at 512 characters §3.4 §4.1 tab. A.1. Several overlap subsets are small, so the intervals are wide. For example, MedQA with overlap is 12 questions, with a 95% interval of 76.0 to 100.0 tab. 6.
• Minor text/table mismatch. The text gives GPT-4-base MedMCQA as 73.66, and tab. 4 gives 73.7 §4.1.
• Physician baseline is narrow. Physicians wrote one answer per question, with no clinical scenario and no time limit, and the authors say this may not match real practice §6. Longer answers may also drive the preferences §4.2 §6.
• Rubric not validated. The authors say the rating rubric is not a formally validated instrument. Pairwise and individual evaluation differ on three axes (reading comprehension, knowledge recall, reasoning), so the axes are not directly comparable §3.5 §6.
• Excluded ratings. Because of display issues, 8 of 1066 answers were not shown in the Med-PaLM 2 vs physician comparison, and 11 of 1066 were not shown in the Med-PaLM 2 vs Med-PaLM comparison. They are excluded from those figures §3.5.
• Small adversarial sets. The adversarial sets have 58 and 182 questions. The authors say they are not a comprehensive assessment of safety, bias or equity §6.
• Delta sign. The header order in tab. 6 reads like "with, then delta" but the delta is without minus with. Read the sign rule above before quoting it.

Lineage

• Builds on: Med-PaLM, which introduced MultiMedQA, the physician-and-lay rubric and the few-shot prompts reused here §2 §3.3 §3.5. Chain-of-thought prompting: Chain-of-thought §3.3. Self-consistency: Self-consistency §3.3. Base model PaLM 2 is cited as google2023palm2, with no card §3.2.
• Compared against: Flan-PaLM, GPT-4 and GPT-4-base, cited in tab. 4. No cards for these papers in this wiki.
• Superseded by: not known from this wiki.