Large Language Models Encode Clinical Knowledge

In plain words

Medical question answering is hard. A good answer needs the right facts, a correct line of reasoning and no dangerous mistakes. Multiple-choice exam scores are easy to measure, but they miss whether a long, free-text answer would be safe and useful to a patient or a doctor.

The paper does two things. First, it collects a benchmark, MultiMedQA, that mixes medical exam questions, medical research questions and questions real people search for online. Second, it asks clinicians and lay people to grade long answers along several axes, such as agreement with medical consensus, possible harm, missing content and bias. The language model is a large general-purpose one. It is first tested with carefully written prompts, then given a small amount of extra training on clinician-written example answers. That extra training is cheap because only a short block of learned input vectors is trained while the model itself stays frozen.

The base model scores well on the exam-style questions. Human graders still find clear gaps in its long answers, and the extra training closes much of that gap, though not all of it.

TL;DR

• MultiMedQA is a medical QA benchmark of seven datasets, six existing and one new (HealthSearchQA) §3.1.
• Flan-PaLM 540B with few-shot, chain-of-thought and self-consistency prompting reaches 67.6% on MedQA (USMLE, 4 options) §4.1.
• Instruction prompt tuning adds a 100-token learned prefix (1.84M parameters) trained on 40 clinician-written exemplars, giving Med-PaLM §3.3.3 app. A.1.
• Clinicians rate Med-PaLM answers much better than Flan-PaLM on most human-evaluation axes, though still below clinicians on several §4.5 app. A.7.
• Everything here is author-reported. The paper links no code and no weights [Paper vs code].

Problem

Accuracy on multiple-choice medical exams says little about long-form answers. Reading comprehension, recall of facts, reasoning, safety and bias all matter in clinical use, and BLEU-style metrics do not measure them §1 §3.2. The authors also argue that full fine-tuning is too expensive, and that clinician data for it is scarce §3.3.2. The method therefore has to work with few examples and few trainable parameters.

Method

Benchmark: MultiMedQA §3.1 tab. 1

MultiMedQA has seven English datasets. The six existing ones are MedQA (USMLE), MedMCQA, PubMedQA, LiveQA (TREC-2017), MedicationQA and MMLU clinical topics. The seventh, HealthSearchQA, is new: 3,375 consumer questions gathered from seed medical conditions and symptoms, using questions a search engine shows to every user who enters those terms §3.1.1 §3.1 tab. 1.

The authors use the multiple-choice sets for accuracy. They do not use the reference long answers that come with PubMedQA, LiveQA and MedicationQA, because those answers come from inconsistent sources §3.1.1. The long-form questions are rated by humans instead.

Split sizes as reported in tab. 1: MedQA 11,450 dev / 1,273 test; MedMCQA 187K dev / 6.1K test; PubMedQA 500 / 500; MMLU clinical subsets 123 / 1,089; LiveQA 634 / 104; MedicationQA NA / 674; HealthSearchQA 3,375 (no split).

Human evaluation framework §3.2

Clinician axes §3.2.1 tab. 2. Twelve items, each answered by one clinician per answer:

1. Agreement with scientific and clinical consensus (aligned, opposed, or no consensus).
2. Extent of possible harm.
3. Likelihood of possible harm.
4. Evidence of correct reading comprehension.
5. Evidence of correct recall of knowledge.
6. Evidence of correct reasoning steps.
7. Evidence of incorrect comprehension.
8. Evidence of incorrect recall.
9. Evidence of incorrect reasoning.
10. Inappropriate or incorrect content.
11. Missing content.
12. Possibility of bias for a demographic group.

Harm is judged only for physical and mental health. Severity follows an AHRQ-style scale, and likelihood is judged on the assumption that a reader acts on the answer §3.2.1.

Lay axes §3.2.2 tab. 3. Two items: whether the answer captures the intent of the question, and how helpful it is. Five raters without a medical background, all in India, rated these §3.2.2.

Models §3.3.1

• PaLM, a decoder-only transformer, at 8B, 62B and 540B parameters. Training corpus: 780 billion tokens, one epoch §3.3.1.
• Flan-PaLM, the instruction-tuned counterpart at the same three sizes §3.3.1.
• Med-PaLM: Flan-PaLM 540B with instruction prompt tuning §3.3.3 §3.3.4.

Prompting strategies §3.3.2

• Few-shot. Demonstrations are written with a panel of clinicians, one set per dataset app. A.8. Five input-output examples are used for consumer datasets, and three or fewer for PubMedQA because the abstract must also fit in the prompt §3.3.2.
• Chain-of-thought. Each demonstration includes a step-by-step explanation. Prompts were written with clinicians §3.3.2 app. A.9.
• Self-consistency. Sample several reasoning paths, then take the majority or plurality answer §3.3.2 §4.4.

Instruction prompt tuning §3.3.3

Standard prompt tuning learns a soft prompt that replaces the hand-written prompt. Here the learned soft prompt is a shared prefix placed in front of a normal, human-written task prompt. That prompt holds the instructions and/or few-shot exemplars, which may include chain-of-thought examples, followed by the question or context. Only the soft prompt is trained §3.3.3.

The authors describe this as a hybrid of hard and soft prompting, and say it is the first published example of a learned soft prompt placed before a full hard prompt with instructions and few-shot exemplars §3.3.3.

# Instruction prompt tuning  [§3.3.3] [app. A.1]
# Only soft_prompt is trained; model weights stay frozen.
# soft_prompt shape: (100, 18432)  # 1.84M parameters

def train_candidate(model, exemplars, learning_rate, weight_decay):
    # Uniform init over [-0.5, 0.5]  [app. A.1]
    soft_prompt = uniform(-0.5, 0.5, shape=(100, 18432))
    # AdamW with the grid-searched values  [app. A.1]
    optimizer = adamw(learning_rate, weight_decay)
    # 200 steps at batch size 32  [app. A.1]
    for step in range(200):
        batch = sample_exemplars(exemplars, size=32)
        # Soft prompt goes first, then the instruction and few-shot text,
        # then the question  [§3.3.3]
        inputs = concat(soft_prompt, task_prompt(batch))
        # Loss is not stated in the paper  [§3.3.3]
        loss = training_loss(model, inputs, batch)
        soft_prompt = optimizer.update(soft_prompt, loss)
    return soft_prompt

# Checkpoint selection, by clinician judgement  [app. A.1]
# Not an automatic validation metric: the authors argue that likelihood
# metrics do not track human judgements of free-text answers  [app. A.1]

def select_checkpoint(candidates, held_out_questions, clinician):
    # Clinician ranks responses on held-out HealthSearchQA, MedicationQA
    # and LiveQA questions  [app. A.1]
    return clinician.rank_best(candidates, held_out_questions)

Training data: 40 free-text question-answer exemplars, written by clinicians. They were sampled from HealthSearchQA, MedicationQA and LiveQA. Clinicians dropped any question for which they could not write an ideal answer §3.3.4.

Self-consistency for multiple choice §4.4

# Self-consistency for multiple-choice questions  [§3.3.2] [§4.4]
# 11 chain-of-thought paths per question  [§4.4]

def answer_with_self_consistency(model, question, options):
    # Sampling temperature is not stated; the paper says only that
    # sampling used temperature  [app. A.2]
    final_answers = []
    for path in range(11):
        explanation, answer = sample_cot_answer(model, question, options)
        final_answers.append(answer)
    # Majority or plurality vote over the final answers  [§3.3.2]
    return most_common(final_answers)

Hyperparameters

Results

Author-reported. Setup: Flan-PaLM and Med-PaLM are 540B models. Multiple-choice results use few-shot, chain-of-thought or self-consistency prompting as noted. Human results are from one clinician or lay rater per answer, with 100-replica bootstrap intervals §4.5.

Multiple-choice accuracy (%)

The MMLU clinical-knowledge figure is discussed under Pitfalls.

Model size ablation, few-shot (%) tab. 5

Prompting strategy ablation, Flan-PaLM 540B (%) tab. 6 tab. 7

MMLU clinical topics, Flan-PaLM 540B (%) app. A.3

Human evaluation, 140 questions (%) §4.5 app. A.7

The set is 100 HealthSearchQA, 20 LiveQA and 20 MedicationQA questions, none used for training. Nine clinicians rated answers from the expert panel, Flan-PaLM and Med-PaLM, without knowing the source §4.5.

Lay raters (five people without a medical background, all in India):

Med-PaLM improved on Flan-PaLM on most axes §4.5 fig. 6 fig. 11. The exception is inappropriate or incorrect content, where Med-PaLM is worse (18.7% vs 16.1%) app. A.7 fig. 8. Its answers stayed below expert answers on the axes shown above.

Paper vs code

No official code. The paper links no repository, and its arXiv metadata has no comment with a link. Searches: med-palm and MultiMedQA HealthSearchQA on the GitHub API, and med-palm restricted to the google-research organisation. None returned a repository from the authors. Candidate repositories carry names such as "Med-PaLM" but belong to third parties, and none of them are cited here.

Pitfalls

• "Surpassing by over 17%" is absolute points. 67.6 vs 50.3 is 17.3 points, a relative gain of about 34%. The abstract's wording is ambiguous §4.1 tab. 4.
• Scaling "over 14%" is also points. Flan-PaLM 540B minus 62B on MedQA is 60.3 − 46.1 = 14.2 points §4.4 tab. 5.
• The MMLU clinical-knowledge number disagrees. The text gives 84.0% for Flan-PaLM 540B on clinical knowledge §4.3. The appendix table gives 80.4% for self-consistency on the same topic app. A.3. Professional medicine (83.5%) matches in both places.
• Figure number in the text. The PubMedQA paragraph says "Figure 2 below". Counting in source order, the comparison chart is fig. 3, and fig. 2 is the instruction-tuning diagram. Read the figure number from the compiled PDF.
• Clinician panel size is inconsistent. Med-PaLM exemplars: five clinicians §3.3.4. Instruction prompt tuning on MCQ data (preliminary): five app. A.6. Few-shot prompt construction: four §6.4. Clinicians in the evaluation: nine §4.5.
• Human-evaluation numbers differ between text, captions and tables. The table values cited above are from app. A.7, which the figure captions mostly match. The text uses other values for the same axes, shown in parentheses. The Med-PaLM "aligned" row reads 92.9 in the text but 92.6 in the table. The "No consensus" row in the table holds the 92.9 / 92.6 / 61.9 values that the text reports as "aligned". The "Aligned" row has 5.0 / 7.4 / 19.1. Check the source table before using these numbers.
• The likelihood-of-harm table repeats the extent-of-harm values. Its caption cites 19.4% (Flan-PaLM), 3.1% (Med-PaLM) and 1.6% (clinicians), which the table body does not contain app. A.7.
• Few-shot counts vary by dataset. Five for consumer data, three or fewer for PubMedQA §3.3.2. Demonstration text for each dataset is in app. A.8.
• Self-consistency details are missing. The sampling temperature, top-p and the final-answer extraction method are not stated app. A.2 §4.4. Only the path counts (11 for MCQ, 41 for deferral) are given. The 0.078 variance has no unit.
• Soft prompt cost at inference. The 100 learned vectors are prepended to every prompt, so inference uses 100 extra tokens of context §3.3.3.
• Checkpoint selection is manual. One clinician ranked candidate checkpoints on held-out questions app. A.1. The candidate set is not described (for example, which steps or grid points were compared), so the selection cannot be reproduced exactly.
• Training sampling is not stated. The paper gives 200 steps at batch 32, which is 6,400 exemplar presentations for 40 exemplars, or about 160 passes on average. The sampling scheme (uniform, with or without replacement) and the loss function are not stated app. A.1 §3.3.3.
• Preliminary MCQ result is one run. The Med-PaLM MedQA figure (67.2%) is described as roughly matching Flan-PaLM SC app. A.6. No variance is given.
• Human evaluation is thin. One rater per answer, nine clinicians, five lay raters, 140 questions §4.5 §6.3. The authors say the rating axes are subjective and only pilot-validated.
• Benchmark availability. The paper says HealthSearchQA is published for the community §3.1.1, but the source gives no URL. Check the release before depending on it.
• Consensus can be out of date. Answers can reflect the scientific consensus of the training data's date, not today's §4.5.
• Harm is judged for physical and mental health only §3.2.1. Financial and moral harms are out of scope.
• The PubMedQA ceiling argument rests on one number. The authors suggest PubMedQA has a ceiling because a single rater scored 78.0% §4.2. That is one human rater, and the paper does not show a range.
• Compute. The 6,144-chip figure is for PaLM 540B pretraining. The paper does not give the compute cost of instruction prompt tuning §3.3.1.

Lineage

• Builds on: PaLM §3.3.1, Flan-PaLM §3.3.1, prompt tuning §3.3.3, chain-of-thought prompting §3.3.2, self-consistency §3.3.2, few-shot prompting §3.3.2. These are named in the paper but have no card in this wiki.
• Compared against: PubMedGPT, DRAGON, BioLinkBERT, Galactica, PubMedBERT, GPT-Neo tab. 4, and BioGPT §4.2.
• Later work in this wiki: Med-PaLM M, a multimodal extension by several overlapping authors. The paper itself does not say that Med-PaLM M supersedes this one.