DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4

In plain words

Hospital notes are full of details that identify a patient: names, dates, places, ages and contact details. Laws such as HIPAA require those details to be removed before records are shared. Doing it by hand is slow and error-prone. Older automated tools tend to need rules or training data written for each new dataset.

The authors ask whether a large chat model can do the job from instructions alone. They write a prompt that says what to remove, lists the kinds of detail to look for, gives examples, and asks the model to replace each identifying piece with the word "[redacted]". The note goes in with that prompt, and the cleaned text comes back. No training runs are involved.

The result depends heavily on how the prompt is written. Explicit prompts with concrete examples did much better than a bare request to anonymize the note. The authors also warn that a hospital cannot send patient text to an outside service, so this approach raises a data-privacy question of its own.

TL;DR

The method is prompt-only de-identification: a prompt built from HIPAA-style PHI categories asks an LLM to replace identifiers in a clinical note with "[redacted]", with no training §IV, Designing Prompts fig. 8. Prompt wording matters a lot. On the 2014 i2b2/UTHealth set, the paper reports ChatGPT rising from 0.686 to 0.929 when an explicit prompt replaces an implicit one §V, Results tab. 2. GPT-4 scores 0.99 with the explicit prompt, but those numbers come from 50 samples run by hand in the web interface, because GPT-4 had no public API at the time §IV, Accessing ChatGPT and GPT-4 §V, Experimental Design. The released code does not include the prompt and calls gpt-3.5-turbo chatgpt_submission_public.py.

Problem

Clinical text carries identifiers that HIPAA requires removing before sharing. The paper lists 18 identifier categories tab. 1 §IV, Designing Prompts. Manual redaction is costly and error-prone. The paper cites a reported manual recall of 63% to 94%, depending on the annotator §II, Manual Methods. Rule-based methods depend on patterns and dictionaries that need tuning for each dataset and miss context §II, Rule-based Methods. Learning-based methods can become unstable when the data shifts from the training domain §II, Learning-based Methods. The paper's goal is a method that needs no fine-tuning and no procedural change across hospitals, languages or data formats §IV, A Simple and Versatile Framework.

Method

The pipeline has four parts: extract the note text, build one prompt, send each note to the model, and score the replaced spans against gold annotations. The prompt has three segments in this order: a task statement, a command, and specific rules §IV, Designing Prompts fig. 8.

The paper also maps the HIPAA identifiers to the dataset's own PHI categories. For each identifier it scores semantic similarity against each category, using the same GPT model. It assigns the best category if the score passes a threshold and otherwise assigns an "Others" category fig. 3. The threshold value is not stated.

1. Extract. Read the XML TEXT element of each file as the note. Keep the TAGS annotations as gold PHI for scoring §IV, Data Preprocessing chatgpt_submission_public.py process_xml_public.py.
2. Build the prompt. The task statement is "Please anonymize the following clinical note." The command is "Replace all the following information with the term "[redacted]":". The rules cover names (patients, doctors, M.D. or Dr., initials and acronyms), pager and medical-staff names, locations and addresses, ages, dates, IDs and numbers, clinic and hospital names, professions, and contact information. Each rule carries an example or a category tag §IV, Designing Prompts fig. 8.
3. Send. The prompt goes in the system role and the note in the user role fig. 7. There are no examples in the prompt, so the setting is zero-shot §IV, A Simple and Versatile Framework §V, Results.
4. Read the output. The model returns the note with each detected PHI span replaced by the token. The text and Figure 8 use "[redacted]", but the caption of Figure 7 writes just 'redacted' §IV, Designing Prompts fig. 7.
5. Score. Entity-wise accuracy is the percentage of gold PHI removed after processing §V, De-Identification Quality Evaluation. The paper gives a formula for it, eq. 1, which does not match the text or the code; see Pitfalls.

Figure 9 lists four prompt failures the authors saw: a task-only prompt, a stray period that stopped the command being read, a prompt with several tasks in one, and a prompt that did not state the output format fig. 9. Figure 8 is the prompt the authors call optimal fig. 8.

Algorithm 1 alg. 1 loops over "in-context learning epochs" and calls a train step on the data and prompt, then runs DeID-GPT on the dataset. The paper does not define train for a model that is only reached through an API, and its results describe zero-shot prompting. This card implements the single-call version above.

# Zero-shot redaction of one clinical note  [§IV] [fig. 7]
# Input:  note (string); prompt (string: task, command, rules)  [fig. 8]
# Output: redacted note (string), PHI spans replaced by "[redacted]"

def deidentify_note(note, prompt):
    # Prompt goes in the system role, the note in the user role  [fig. 7]
    messages = [
        {"role": "system", "content": prompt},
        {"role": "user", "content": note},
    ]
    # One call per note, no examples and no weight updates  [§IV] [§V]
    # Temperature 0.05 is the code default; the paper does not state it
    # [code: chatgpt_submission_public.py#L21]
    reply = chat_model(messages, temperature=0.05)
    # The model's reply is the redacted note
    # [code: chatgpt_submission_public.py#L59]
    return reply

# Entity-wise accuracy as the released evaluator computes it
# [code: process_xml_public.py#L5-L6] [code: process_xml_public.py#L122-L126]
# Input:  redacted (string); gold (dict: category -> list of PHI strings)
# Output: 1 - (gold strings still present) / (all gold strings), per note

def note_accuracy(redacted, gold):
    # Split the output on whitespace; each token is compared as a whole
    # [code: process_xml_public.py#L5-L6]
    tokens = set(redacted.split())
    # Count distinct gold strings that still appear as a token
    # [code: process_xml_public.py#L86-L120]
    remaining = 0
    for strings in gold.values():
        remaining += len(set(strings) & tokens)
    # Total number of gold strings, duplicates included
    # [code: process_xml_public.py#L123]
    total = sum(len(strings) for strings in gold.values())
    return 1 - remaining / total

Key equation, as numbered in the paper:

Accuracy = \frac{TP+TN}{TP+TN+FP+FN} eq. 1

Hyperparameters

Source: the LaTeX source that wikillm fetch retrieved for this arXiv record. Values the paper does not give are marked "not stated". Code values come from the released repository at code_commit.

Results

Author-reported. Setup: entity-wise accuracy on the 2014 i2b2/UTHealth de-identification test data §III §V. ChatGPT and the other prompted models and the fine-tuned baselines were run on all test cases. GPT-4 was run by hand on 50 random test cases through the web interface, and the same scoring script was applied to all outputs §V, Experimental Design. The paper does not report run-to-run variance.

Full table as printed in the paper. Dashes mark cells the paper does not fill. "N/A" marks rows the paper does not run in that setting tab. 2:

Source for every row: tab. 2.

Claims the text makes about these numbers:

• GPT-4 with an optimal explicit prompt reaches "over 0.99" in the zero-shot setting, and the text says it outperforms ChatGPT and all other baselines §V, Results. The table prints 0.99 tab. 2.
• An optimal prompt raises ChatGPT from 0.686 to 0.929. GPT-4 is less sensitive to prompt defects, scoring 0.908 even on the implicit prompt §V, Results tab. 2.
• RoBERTa and ClinicalBERT score above 90% after fine-tuning §V, Results tab. 2.
• Error analysis: GPT-4 and ChatGPT mostly miss entities. The Llama and Falcon models mostly misread the task, sometimes repeating the prompt §V, Error Analysis.

Paper vs code

The repository linked from the paper is github.com/yhydhx/ChatGPT-API at commit d3ca165840ead46de313325df7e574016520524e. It is the only code the paper links, so it is used as the official code. Its README marks the code "In submission". Its LICENSE is MIT, and its copyright line names "Jupiter", not an author. Paths below are relative to the repository root.

• The prompt is not in the code. The paper sends the prompt in the system role and the note in the user role fig. 7. The released request has one user message containing only the note chatgpt_submission_public.py chatgpt_submission_public.py. The released script therefore does not reproduce the prompt-based method. The prompt must come from fig. 8.
• Model. The paper's ChatGPT arm uses GPT-3.5 through the API §IV, Methodology. The script defaults to gpt-3.5-turbo chatgpt_submission_public.py. The GPT-4 arm has no code path; it was run by hand §V, Experimental Design.
• Metric. The paper defines accuracy as (TP+TN)/(TP+TN+FP+FN) eq. 1 but describes it in the text as the percentage of sensitive entities removed §V, De-Identification Quality Evaluation. The text does not define TN, so the formula cannot be applied as written. The code computes 1 minus (remaining gold strings / all gold strings) instead process_xml_public.py.
• Entity matching. The code counts an entity as still present only when its whole text equals one whitespace-separated token of the output process_xml_public.py. A multi-word name can never match, and a token with trailing punctuation will not match its bare entity. Both cases make the code count an entity as removed when it may survive (inferred from the code; not tested).
• Averaging. The code averages per-note accuracies without weighting by entity count process_xml_public.py process_xml_public.py. The paper does not say how it averages.
• Preprocessing. The paper describes in-house extraction scripts §IV, Data Preprocessing. The released script contains the equivalent extraction from TEXT chatgpt_submission_public.py.
• Category mapping. The semantic-similarity mapping of HIPAA identifiers to dataset categories fig. 3 is not in the repository.
• Placeholders. The script ships with an empty API key and empty input directory chatgpt_submission_public.py chatgpt_submission_public.py. Both must be filled in before it runs.
• Output. The script writes each reply to <name>_anonymized.txt without post-processing chatgpt_submission_public.py.

Pitfalls

• The prompt exists only as a figure. The full optimal prompt is in Figure 8, not in the text fig. 8. The LaTeX source also holds a commented-out older prompt; it is not part of the published paper, so this card does not use it.
• The HIPAA list may not be in the prompt. The introduction says the HIPAA identifiers were integrated into the prompt §I. Figure 8 shows category rules and no list of the 18 identifiers fig. 8. Figure 3 maps identifiers to categories with an unstated threshold fig. 3. An implementer has to choose whether to include the 18-item list.
• Algorithm 1 is not implementable as written. It loops over "epochs" of train on the API model alg. 1. A frozen API model has no training step, and the results are zero-shot §V, Results. Use the single-call form in Method.
• Accuracy is not defined consistently. The formula eq. 1 needs true negatives, which the text never defines. The text describes recall-like removal, and the released code uses a different count again (see Paper vs code). Numbers from the three definitions are not comparable.
• GPT-4 results are not reproducible from code. They come from 50 random samples run by hand in the web interface, with no model version, date or sampling settings given §V, Experimental Design. The text says "over 0.99" while the table prints 0.99 §V, Results tab. 2.
• "Outperforms all baselines" holds only for the explicit prompt. The implicit-prompt GPT-4 score of 0.908 is below RoBERTa (0.947) and ClinicalBERT (0.974) tab. 2.
• Llama rows conflict with the text. The discussion says LLaMA failed to produce coherent output for all test cases and is "not directly comparable" §VI. Yet the table reports Llama1-7b and Llama2-7b scores, and the two rows are identical tab. 2. Llama2 is not discussed anywhere in the text. The setups for mT0, Falcon-7b and Flan-T5-base are not described in the text either.
• The "General prompt" column is undefined. No text explains it, and one cell reads "N/Aw" in the source tab. 2.
• Figure 7 caption and text disagree on the token. The caption says the term is 'redacted', the text says "[redacted]" fig. 7 §IV, Designing Prompts.
• Prompt sensitivity is high. The authors observed that an extra period stopped the command being read, and that combining tasks confused the model fig. 9. Test any prompt edit against the scorer.
• Dataset statement conflicts. The introduction says the paper uses no original medical data but "synthesized public medical datasets in which private information has been filtered" §I. Section III describes the i2b2/UTHealth data, whose PHI was replaced by surrogates, and access is granted on request §III. Do not redistribute the dataset.
• Privacy of the method itself. Sending a note to an external API transmits PHI to a third party. The authors state that API-only models cannot be used in a hospital because patient data cannot be sent to an unauthorised external party, and that local models are needed §VI, Locally-Deployed DeID-GPT for Hospital Use. This is the main limit on use.
• No variance reported. The paper gives single numbers and no seeds, repeats or confidence intervals §V. The released script uses temperature 0.05 chatgpt_submission_public.py, which is low but not zero.
• Validation is limited. The paper describes the method as zero-shot and uses one dataset §III §V. Generalisation to other data is discussed as future work §VI, Applying LLM to Anonymize Other Data.

Lineage

• Builds on: prompt-based in-context learning §II, Zero-shot and Few-Shot In-Context Learning. The paper does not give arXiv ids for its prior work.
• Compared against: fine-tuned BERT, RoBERTa and ClinicalBERT §V, Experimental Design tab. 2; prompted mT0, Falcon-7b, Flan-T5-base, Llama1-7b and Llama2-7b tab. 2.
• Follow-up named by the authors: local open-source models, fine-tuning on domain data and quantisation for hospital deployment §VI.
• Superseded by: not stated in the wiki content.