Unlearn What You Want to Forget: Efficient Unlearning for LLMs

In plain words

Sometimes a model has learned from data that someone later asks to have removed, for privacy reasons. Retraining the whole model without that data is the clean fix, but for a large model it is slow and expensive. This paper asks whether the model can instead be adjusted so that it behaves as if it never saw that data, while still doing its job well.

The approach keeps the big model fixed and adds a small, separate piece that is trained only to forget. The training pushes the model's answers on the data to keep toward the original model's answers, and pushes its answers on the data to forget away from them. A second term keeps the model good at its real task, and a third term makes the forgotten text harder to recover from the model's word-filling ability. Each new deletion request gets its own small piece, and a final step combines these pieces into one, so that several requests can be handled in turn.

The authors test this on two text tasks, sentiment classification of movie reviews and summarising chat conversations, using two sizes of the T5 model. The tests simulate requests to delete all data tied to a given person or name.

TL;DR

• Add small unlearning layers to a frozen T5 model and train them with a selective teacher-student KL objective, a task loss on retained data and a negated masked-LM loss on forgotten data §3.1 eq. 5.
• Several unlearning layers are merged into one by a closed-form least-squares formula over stored Gram matrices, so no data is kept and no retraining is needed §3.2 eq. 6 eq. 7.
• Author-reported: on IMDB with T5-base, forget-set accuracy falls to 65.7 at 0.5% forgotten, against 92.5 for re-training, and the update takes 1200 s against 6685 s tab. 2.
• MEND is faster in every block of the main tables tab. 2 tab. 3, and MEND is not evaluated on SAMSum §4.3.
• The official repository, at the pinned commit, has no unlearning code. Implement from the paper and the equations here README.md.

Problem

Privacy rules such as the GDPR and CCPA require that personal data can be deleted on request §1. For an LLM that has memorised such data, the exact fix is to retrain without it. The paper says this is extremely expensive for large models §2.2. The SISA method trains on isolated shards and retrains only the affected shard, but it needs a lot of storage §1 and gets lower accuracy in the authors' runs §4.5. Differentially private training slows convergence and can hurt accuracy §1. Methods of both kinds must redo the whole model, so they do not handle a sequence of deletion requests well §1 §2.2.

The paper's goal is a method that is efficient, handles a sequence of deletions, and keeps task accuracy §1 §3.

Method

Setting. The base model F is trained on data D = {(x, y)}. A deletion request gives a forget set D^f, and the retained data is D^r = D − D^f. The target is a model F' with mutual information between F's and F''s outputs that is zero on D^f and one on D^r eq. 1 §3. The paper does not state how this is tested in practice. Its experiments use output accuracy, MLM loss, and a membership attack instead §4.2 §4.6.

Unlearning layer. The new model is F'(·) = F(f(·)). The adapter f(·; W) has far fewer parameters than F, and only f is updated, while the LLM stays frozen §3.1. The layers are plugged into transformer layers after the feed-forward networks fig. 1. The paper tunes 0.5% of the parameters §4.4.

Selective teacher-student KL eq. 2. The KL term rewards the student for matching the teacher on retained data and rewards it for diverging from the teacher on forget data. A hyperparameter alpha balances the two §3.1. The paper states the formula as written in eq. 2, with a minus sign on the forget term.

Task loss eq. 3. On retained data, a task loss keeps performance. For classification this is cross-entropy §3.1.

Negated LM loss eq. 4. On forget data, the original pre-training objective is negated, so the forgotten text becomes harder to predict. For T5 the masked-LM loss is used, with the instruction "Predict the masked word" prepended to the input §3.1.

Total objective eq. 5. The three terms are combined with weights lambda and gamma §3.1 eq. 5. The paper optimises this min-max form by alternating an epoch on retained data with an epoch on forget data §3.1. The paper does not say which terms are computed on which epoch. The pseudocode below assigns each term to the data it is defined on; this is the card's reading, not the paper's wording.

Fusion eq. 6 eq. 7. For each forget set i, a layer f_i with weights W_i is learned. The fused layer W_m minimises the squared difference between its output and each f_i's output on the forget inputs X_i^f eq. 6. The closed form is W_m = (Σ_i X_i^T X_i)^{-1} Σ_i X_i^T X_i W_i eq. 7. Only the Gram matrices X_i^T X_i of the hidden states before the unlearning layers are stored, not the data §3.2. Fusion needs no extra training §3.2.

Steps for one deletion request (pseudocode follows eq. 2 to eq. 5 and the alternation in §3.1):

# Learn one unlearning layer for one deletion request  [§3.1] [eq. 5]
# Input:  teacher (frozen F); adapter (weights W); retained and forgotten
#         data; alpha, lam, gam (hyperparameters)
# Output: trained adapter weights W

def train_unlearning_layer(
    teacher, adapter, retained, forgotten, alpha, lam, gam
):
    # Alternate: one epoch on retained data, then one on forgotten  [§3.1]
    for each round:
        for batch in retained:
            # Student follows teacher on retained data  [eq. 2]
            kl_retain = KL(teacher(batch) || student(batch))
            # Task loss keeps accuracy on retained data  [eq. 3]
            task_loss = loss(student(batch), labels(batch))
            update(adapter, alpha * kl_retain + lam * task_loss)
        for batch in forgotten:
            # Student moves away from teacher on forgotten data  [eq. 2]
            kl_forget = KL(teacher(batch) || student(batch))
            # Negated masked-LM loss, with the masking prefix  [eq. 4] [§3.1]
            lm_loss = -masked_lm_loss(student(prefixed(batch)))
            update(adapter, -kl_forget + gam * lm_loss)
    return adapter

# Fuse the unlearning layers of several deletion requests  [§3.2] [eq. 7]
# Input:  weights[i] (d, d_out) per request; grams[i] = X_i^T X_i (d, d)
# Output: fused weights W_m (d, d_out)

def fuse_unlearning_layers(weights, grams):
    # Sum of the stored Gram matrices over all forget sets  [eq. 7]
    total_gram = sum(grams)
    # Sum of each Gram matrix times its layer's weights  [eq. 7]
    weighted_sum = sum(grams[i] @ weights[i] for i in requests)
    # Closed-form solution, written with an inverse in the paper  [eq. 7]
    return inverse(total_gram) @ weighted_sum

Key equations, verbatim where the exact form matters:

L_{KL} = \alpha \sum_{x^r} KL(F(x^r)||F(f(x^r))) - \sum_{x_f} KL(F(x^f)||F(f(x^f))) eq. 2

L_{EUL} = L_{KL} + \lambda L_{TASK} + \gamma L_{LM} eq. 5

W_m = (\sum_i {X^f_i}^TX^f_i)^{-1}\sum_i({X^f_i}^TX^f_iW_i) eq. 7

Hyperparameters

Dataset sizes, as split counts tab. 1:

• IMDB, classification: 20000 training, 2000 dev, 25000 test examples.
• SAMSum, summarization: 14732 training, 818 dev, 819 test examples.

Results

Author-reported. Setup: T5-base and T5-3b, fine-tuned on each dataset; baselines Re-train, Fine-tune, SISA, Reverse-Gradient and MEND §4.3. The paper marks the best value per block in bold; those marks are not reproduced here. Columns are the paper's: Test and Retained are accuracy (IMDB) or ROUGE-1/2/L (SAMSum), Forgot is the same metric on the forget set (lower is better), MLM loss is higher when the forgotten information is harder to extract, and time is the update time in seconds §4.2. EUL is the paper's method (marked † in the paper).

IMDB, full tab. 2

SAMSum, full tab. 3

Cells are ROUGE-1/2/L. MEND has no row because the authors could not design the editing target for summarization §4.3.

IMDB, mislabelled training labels, T5-base tab. 4

10% of training labels were changed at random, and the model was asked to unlearn them. Columns: Test, Retained, Forgot, and update time in seconds. Averaged over 5 runs §4.5 tab. 4.

Ablation, IMDB, T5-base, 0.5% forgotten tab. 5

Removing one objective at a time. Run count not stated.

Sequence of removals, IMDB, T5-base, single order tab. 6

Cells are Test/Retained accuracy after each step in the order set 2, set 1, set 3. Each set is 1% of the training data §4.5. The averaged curves for 1 to 5 sets are in fig. 2, not tabulated.

EUL keeps one unlearning layer and trains it on each set in turn. EUL-fuse learns one layer per set and merges them with eq. 7 §4.5.

Membership attack, 1% forgotten, T5-base tab. 7

Accuracy of a classifier that tries to tell forgotten from retained inputs. Closer to 0.5 means harder to tell apart §4.6.

Paper vs code

The paper's abstract footnote links https://github.com/SALT-NLP/Efficient_Unlearning. The owner is SALT-NLP, the "SALT Lab" the acknowledgment thanks, and one of the authors is Diyi Yang. We cloned it at 6abab5ba7159586322da4e0184bf1e1a268dd288 and read it.

• No unlearning code. The repository contains no implementation of the selective KL objective, the task or LM terms, the alternation, or the fusion. Searching all files for "unlearn" or "forgot" finds only the README. README.md says "The codes will be updated soon."
• README commands do not exist. The README gives python src/run.py --unlearn ... and python src/run.py --merge ... README.md README.md. The path is src/models/run.py, and that script has no such arguments: it parses only the Hugging Face argument classes run.py.
• The script is a generic classifier fine-tuner. It loads GLUE-style or local CSV/JSON classification data and adds a classification head run.py. It has no generation path, while the paper's SAMSum results use summarization with T5 §4.1.
• Sequence length default. The default maximum length is 128 run.py. This matches the paper's IMDB setting §4.4, but not the 800 used for SAMSum §4.4. The paper's per-task settings are not in the code.
• Adapter default. The adapter configuration defaults to "pfeiffer" training.py. The paper does not name an adapter architecture §4.4, so this cannot be checked against the paper, and the repository does not show whether it is the one used for the results.
• Licence. The repository is Apache-2.0 LICENSE, which allows reuse and is not an NC or copyleft licence.

Net result: the code cannot be used to reproduce any table. Implement from the equations and tables above, and treat the numbers as author-reported.

Pitfalls

• No reference code. Nothing at the pinned commit implements the method, so every design choice not stated in the paper (adapter architecture, optimizer, epoch-to-term mapping, stopping rule) is open. The code's README promises an update, so check the commit again before relying on it.
• Faster claim. The results text says EUL updates faster than the previous unlearning baselines §4.5. MEND is faster in every block of both tables: for example 34 s vs 1200 s on IMDB T5-base at 0.5% tab. 2. The claim holds only against the other baselines.
• Text and table disagree. The results text reports 65.6% forgot-set accuracy and "1/6 of the updating time" §4.5. The table value is 65.7 tab. 2.
• Unbounded forget terms. The forget-side KL in eq. 2 and the negated LM loss in eq. 4 have no bound in the paper. The paper gives no clipping, early stopping, or stopping rule. The number of alternation rounds is not stated §3.1.
• Term-to-epoch mapping. The paper does not say which terms are computed on the retain epoch and which on the forget epoch. The pseudocode assigns each term to the data it is defined on; the paper does not confirm this.
• Hyperparameter search. alpha, lambda and gamma are chosen from one grid §4.4, but the split used for the search, and whether it was a full product grid, is not stated.
• Epoch counts. The paper gives 3 or 5 epochs without saying which setting uses which §4.4. The optimizer is not named §4.4.
• Fusion stability. The closed form in eq. 7 needs the summed Gram matrix to be invertible. The paper does not mention regularisation, a pseudo-inverse, or the matrix size. The formula also assumes each unlearning layer is linear in its input eq. 6 §3.2, while the adapter's internal structure is not stated in the paper, so the paper does not show that the closed form applies to the trained layer as used.
• No bias term or activation stated for the fusion eq. 6, so an implementer should decide this and state it.
• Gram matrices are the stored state. Fusion keeps one (d, d) Gram matrix per forget set §3.2. The memory cost for a given hidden size is not stated.
• Name-based forget sets. For IMDB, entities are extracted with AllenNLP's pretrained NER §4.1. The model version is not stated, and the forget sets depend on its output. Names in dev and test are replaced by special tokens §4.1.
• MEND is not run on SAMSum. The authors say MEND cannot be applied to summarization §4.3, so the SAMSum table has no MEND row tab. 3.
• Few runs, no spread. Results average 5 runs (IMDB) or 3 runs (SAMSum) tab. 2 tab. 3. Standard deviations are not reported, and the ablation and sequence tables give no run count tab. 5 tab. 6.
• Membership attack is weak evidence. The attack is barely above chance even before unlearning (0.542 on IMDB) §4.6 tab. 7. The paper concludes that the deleted data could not be easily inferred before or after unlearning §4.6. Because the classifier is near chance before unlearning, this result cannot show that forgetting happened.
• Forgetting is measured by task accuracy. The paper's evaluation uses accuracy on the forget set and MLM loss. Whether the model can still recall or be identified from the forgotten content is left to future work §6.
• Simulated requests only. All deletion requests are simulated §6. The paper does not test on real deletion workloads.
• Mislabelled setting differs. The 10% mislabelled test tab. 4 asks the model to forget examples with wrong labels rather than privacy data §4.5. The paper only reports T5-base for it tab. 4.

Lineage

• Builds on: the selective teacher-student formulation of Kurmanji et al. (2023), which the paper cites for its objective §3.1 eq. 2. Parameter-efficient adapters are cited from Houlsby et al. (2019) and Chien et al. (2023) §3.1.
• Compared against: Re-train and Fine-tune, SISA §4.3 (Kumar et al., 2022), Reverse-Gradient (Liu et al., 2022) §4.3, and MEND (Mitchell et al., 2021), a model editor §4.3.
• Related work: differential privacy methods §2.2, and model editing, which the paper contrasts with unlearning §2.2.
• Superseded by: not known from this card's sources.