---
kind: concept
slug: text-de-identification
title: Text de-identification
summary: >-
  Removing details that identify a person (names, dates, places, IDs, contact data) from text before it is shared. One paper so far: prompt-only redaction of clinical notes with ChatGPT and GPT-4. Author-reported entity accuracy on the i2b2/UTHealth set is 0.929 for ChatGPT and 0.99 for GPT-4, the latter on 50 hand-run samples.
tags: [privacy, clinical-text, prompt-engineering, hipaa]
papers: ["2303.11032"]
related: [named-entity-recognition, in-context-learning, prompt-engineering, differential-privacy]
status: draft
updated_at: 2026-10-19
---

## Overview

De-identification removes the details that identify a person from a text and
leaves the rest usable. For clinical notes the target is the 18 identifier
categories that HIPAA lists [arxiv:2303.11032 tab. 1]. Every method aims for
the same end state: the identifying spans no longer appear in the output
[arxiv:2303.11032 §V].

Manual redaction is costly and error-prone. Rule-based tools need patterns
tuned for each dataset, and learned taggers can lose accuracy when the data
drifts from their training domain [arxiv:2303.11032 §II].

## Variants

### Prompt-only redaction with a chat LLM

The model gets a prompt that states the task, the replacement command and a
set of rules with examples for each identifier category. Each note is sent in
a separate call, with no training. The replaced spans are marked
"[redacted]" [arxiv:2303.11032 §IV] [arxiv:2303.11032 fig. 8]. Card:
[DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](/papers/2303.11032).

Reported entity accuracy: ChatGPT 0.686 with an implicit prompt and 0.929 with
an explicit one; GPT-4 0.908 and 0.99 [arxiv:2303.11032 tab. 2]. The GPT-4
numbers come from 50 random samples run by hand in the web interface
[arxiv:2303.11032 §V].

### Fine-tuned encoder taggers

The same paper reports fine-tuned BERT at 0.798, RoBERTa at 0.947 and
ClinicalBERT at 0.974 on the same test data [arxiv:2303.11032 tab. 2]. The
fine-tuning recipe is not stated [arxiv:2303.11032 §V]. These are baselines
in that paper; no card exists for the models themselves.

### Open prompted models

The paper also prompts mT0, Falcon-7b, Flan-T5-base and two Llama models. They
score between 0.597 and 0.827 [arxiv:2303.11032 tab. 2]. The text does not
describe their setups, and the Llama rows conflict with the discussion (see
Contradictions).

## When to use which

The only evidence on this page is one paper on one dataset, so the choice
below rests on that. Treat each row as a starting point to validate on your
own data.

| Situation | Use | Why |
|---|---|---|
| Notes may not leave the hospital | A local model, not a hosted API | Sending notes to an external API transmits PHI; the authors say local models are needed [arxiv:2303.11032 §VI] |
| Labelled PHI spans exist and the domain is stable | Fine-tuned encoder tagger | ClinicalBERT scores 0.974 in-domain [arxiv:2303.11032 tab. 2], but the recipe is not stated, so results need a careful rebuild |
| No labels and no time to train | Prompt-only redaction with a strong chat model, checked on a gold set | Zero-shot, no training [arxiv:2303.11032 §IV]; ChatGPT rose by 0.24 when the prompt was made explicit [arxiv:2303.11032 tab. 2] |
| Results must be reproducible | Either, with a pinned model version and the full test set | The GPT-4 figures rest on a hand-run sample without model version or sampling settings [arxiv:2303.11032 §V] |

## Implementation sketch

Prompt-only redaction of one note. The prompt has three parts: task,
command and rules. The exact prompt text is only in a figure
[arxiv:2303.11032 fig. 8].

```pseudocode
# Prompt-only redaction of one note  [arxiv:2303.11032 §IV]
# Input:  note (string); prompt (task, command, rules)
# Output: note with identified spans replaced by "[redacted]"

def redact(note, prompt):
    # Prompt in the system role, note in the user role  [arxiv:2303.11032 fig. 7]
    messages = [
        {"role": "system", "content": prompt},
        {"role": "user", "content": note},
    ]
    # One call per note; no examples and no weight updates
    return chat_model(messages)
```

## Contradictions

- **Llama results.** The discussion says Llama failed to produce coherent output
  and is "not directly comparable" [arxiv:2303.11032 §VI]. The table still
  reports scores for two Llama models, and the rows are identical
  [arxiv:2303.11032 tab. 2].
- **GPT-4 wording.** The text says "over 0.99" [arxiv:2303.11032 §V]; the table
  prints 0.99 [arxiv:2303.11032 tab. 2].
- **Outperforming baselines.** The text says the explicit-prompt GPT-4 beats all
  baselines [arxiv:2303.11032 §V]. The implicit-prompt GPT-4 score of 0.908 is
  below RoBERTa (0.947) and ClinicalBERT (0.974) [arxiv:2303.11032 tab. 2].
- **Accuracy definition.** The paper gives accuracy as (TP+TN)/(TP+TN+FP+FN)
  [arxiv:2303.11032 eq. 1], but the text describes the share of sensitive
  entities removed [arxiv:2303.11032 §V], and TN is never defined. The two
  readings are not comparable.
- **Dataset.** The introduction mentions synthesized public data
  [arxiv:2303.11032 §I]; the evaluation uses the i2b2/UTHealth set, which is
  available on request [arxiv:2303.11032 §III].

## Pitfalls

- **The prompt is only in a figure.** Transcribe it from [arxiv:2303.11032 fig. 8]
  and check the wording. The paper's linked repository does not contain it;
  see the card's Paper vs code section.
- **Prompt wording is fragile.** A stray period stopped the command from being
  read, and mixing several tasks in one prompt confused the model
  [arxiv:2303.11032 fig. 9]. Test every prompt edit against the scorer.
- **The HIPAA list may be missing from the prompt.** The introduction says the
  identifiers were built in [arxiv:2303.11032 §I], but the prompt shows
  category rules [arxiv:2303.11032 fig. 8]. Decide whether to include the
  18-item list.
- **Algorithm 1 cannot run on a frozen API model.** Its training loop has no
  step to run against an API model, and the reported results are zero-shot
  [arxiv:2303.11032 alg. 1] [arxiv:2303.11032 §V]. Use the single-call form.
- **Single numbers, no variance.** The paper reports no seeds, repeats or
  intervals [arxiv:2303.11032 §V].
- **One dataset.** Generalisation to other data is named as future work
  [arxiv:2303.11032 §VI].
- **Privacy of the method.** Any hosted model receives the PHI it is meant to
  remove [arxiv:2303.11032 §VI].

## Lineage

- Builds on prompt-based in-context learning [arxiv:2303.11032 §II].
- Compared with fine-tuned BERT, RoBERTa and ClinicalBERT
  [arxiv:2303.11032 §V].
- Follow-up named by the authors: local open-source models, fine-tuning on
  domain data and quantisation for hospital deployment [arxiv:2303.11032 §VI].

## Papers

- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](/papers/2303.11032):
  prompt-only redaction of clinical notes with ChatGPT and GPT-4, compared
  with fine-tuned encoder taggers.