Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine

In plain words

A doctor's diagnosis can be explained step by step. A large language model usually gives only the answer, so nobody can easily check how it got there. This paper asks whether a model can be prompted to reason the way doctors are taught to: list possible causes, weigh the clues, or update the odds of each cause as new facts arrive.

The authors write four prompts, each named after a style of clinical reasoning, plus one generic step-by-step prompt used as the baseline. Every prompt shows the model two worked examples, each a patient case with a written explanation, and then asks it to explain its reasoning for a new case before naming a diagnosis. The cases come from a medical exam question set, rewritten so the model must name a diagnosis without choosing from options.

The result is that the newer model, GPT-4, reasons this way without losing accuracy compared with the baseline prompt. The older model, GPT-3.5, does not match the baseline on most of the clinical styles. The explanations give a doctor something to check, but the paper does not test whether those explanations are always logical or free of invented facts.

TL;DR

• Five few-shot prompts (one baseline, four clinician-style) were tested on free-response diagnosis questions from a modified MedQA USMLE set p. 1 tab. 2.
• GPT-4 keeps its accuracy with the clinical prompts: 72-78% against 76% for the baseline, and no comparison reaches significance p. 3.
• GPT-3.5 is significantly worse on the differential-diagnosis, analytic and Bayesian prompts, by 4 to 8 points, and matches the baseline with intuitive reasoning p. 3.
• Grading was done by physicians, and the rationales were not graded for accuracy or hallucination; the authors list that as future work p. 3.

Problem

Clinical decisions made by language models are hard to inspect. The authors argue that a model whose answer comes with a rationale in clinical terms would let physicians check whether it can be trusted p. 1. Early assessments of LLM clinical reasoning were limited because they studied answers to multiple- choice questions p. 1. This study therefore uses free- response questions and only the diagnosis questions of USMLE Steps 2 and 3 (Step 1 is excluded because it is mostly fact recall) p. 1.

Method

Data. The MedQA USMLE set p. 1 was converted to free response by removing the multiple-choice options p. 1. Questions about diagnosis were kept p. 1. The development (training) set has 95 questions and the test set 518 p. 1. The full test set is in Supplemental Information I p. 5.

Prompt structure. Each prompt has an instruction, then two example questions with rationales, then the test question tab. 2 p. 1. The two examples are the same for every prompt tab. 1. Each rationale states the reasoning strategy of its prompt tab. 2.

Prompts. The baseline is a traditional chain-of-thought prompt that asks for a step-by-step deduction tab. 2 p. 1. The four clinical prompts are:

1. Differential diagnosis: list a differential diagnosis, then deduce the answer from it tab. 2.
2. Intuitive reasoning: link symptoms, signs and laboratory findings to diseases, then deduce the answer tab. 2.
3. Analytic reasoning: deduce the physiology or biochemistry of the patient, then identify the answer tab. 2.
4. Bayesian inference: set a prior probability for a diagnosis, update it with new history into a posterior probability, then pick the final diagnosis tab. 2.

Prompt development. Prompts were tried in several rounds on the development set, scoring GPT-3.5 accuracy each time p. 4. The authors found that step-by-step prompts without a fixed list of steps did better, and that a prompt focused on one strategy beat one that mixed strategies p. 4. The prompts used in the study are the top-scoring one in each family in Supplemental Information II p. 5. Development scores are in the table below.

Models. GPT-3.5 is OpenAI's Davinci-003, called through the API p. 4. GPT-4 is called through the OpenAI API p. 4. Its model version is not stated.

Grading. Two blinded physicians grade each response. A third physician settles disagreements. A response that is as correct and specific as the reference answer counts as correct p. 4.

Statistics. Each prompt is compared with the traditional chain-of-thought baseline using McNemar's test for paired proportions, two-tailed, at alpha 0.05 p. 4. Agreement between graders uses Cohen's kappa p. 4.

# Few-shot clinical reasoning prompt  [tab. 2] [p. 4]
# Input:  strategy (instruction text), examples (2 question/rationale
#         pairs), case (free-response question, no answer options)
# Output: model response (rationale and diagnosis)

def diagnose(strategy, examples, case):
    # Start with the instruction for one reasoning strategy  [tab. 2]
    prompt = strategy
    # Add each worked example: its question, then its rationale  [tab. 1]
    for example in examples:
        prompt = prompt + example.question + example.rationale
    # Append the test case; how it is joined is not stated  [p. 4]
    prompt = prompt + case
    # Send the prompt to the model and return its text  [p. 4]
    response = llm(prompt)
    return response

Hyperparameters

Development-set accuracy of each selected prompt for GPT-3.5, out of 95 questions p. 5 (Supplemental Information II):

Results

Author-reported. Setup: MedQA-derived free-response diagnosis questions, GPT-3.5 (Davinci-003) and GPT-4, compared against the traditional chain-of-thought prompt on the same questions p. 3 tab. 2.

Example rationales for one GPT-3.5 and one GPT-4 case are shown in fig. 1 and fig. 2. Full per-question results are in Supplemental Information I p. 5.

Paper vs code

No official code repository was found. The paper points to notebooks for the GPT-3.5 and GPT-4 API calls, the complete graded test results and the dataset in a Google Drive folder p. 5 (Supplemental Information I and IV). That folder was not read for this card. No repository was found by GitHub search for the title or for "diagnostic reasoning prompts medqa gpt-4".

Pitfalls

• Unequal baselines. GPT-3.5 used the DSP library and self-consistency on its chain-of-thought prompts; GPT-4 used neither, because they were not available for GPT-4 at submission p. 4. The GPT-4 comparison is therefore not like-for-like with GPT-3.5.
• Self-consistency scope is ambiguous. The paper says it was applied to "all GPT-3.5 Chain-of-Thought prompts" p. 4. It does not say whether the four clinical prompts were also sampled in this way, and the sample count is not stated.
• Sampling settings are missing. Temperature, maximum output length and stop conditions are not stated p. 4. Accuracy may vary with them.
• Prompt text is abridged. tab. 2 omits the full text of the example questions and rationales for readability tab. 2 caption. Exact wording is only in the supplement.
• How the test case is joined to the prompt is not stated p. 4. The table above is the only description of the format.
• Answer extraction is not stated. The paper does not say how a diagnosis is pulled from the response text, only that physicians grade the final answer p. 4.
• Rationales are not graded. The paper's claim that the rationales mimic clinical reasoning rests on the examples in fig. 1 and fig. 2. It says the rationales still need checking for logic, accuracy and hallucinations p. 3.
• Questions removed. The GPT-4 API failed on 21 questions, so GPT-4 was scored on 497 rather than 518 p. 3. The paper does not say whether the same 21 were removed from GPT-3.5's run.
• Small effects, wide intervals. Most GPT-4 differences have confidence intervals that include zero, and the Bayesian GPT-4 result is borderline (P=0.07) p. 3.
• Prompt choice came from the training set only. The prompts were picked on the 95-question development set with GPT-3.5 p. 4, and then reused for GPT-4 p. 4. GPT-4 was not used to pick them.
• Limited search. The authors say they did not test all possible prompt styles p. 4.
• Free-response grading is subjective. Graders accept an answer that is "equally correct and specific" to the reference p. 4. The paper does not define that threshold beyond this wording.

Lineage

• Builds on: chain-of-thought prompting, the traditional baseline prompt p. 1 (arXiv:2201.11903, ref. 15) and few-shot learning p. 1 (arXiv:2005.14165, ref. 12).
• Also builds on: self-consistency sampling p. 4 (arXiv:2203.11171, ref. 20), used only on the GPT-3.5 chain-of-thought prompts.
• Compared against: the traditional chain-of-thought prompt, on the same questions p. 3.
• Superseded by: not stated in the paper.