Machine unlearning

Overview

Machine unlearning asks a trained model to forget some training data: a user withdraws consent, a record is found to be harmful or copyrighted, or a training point is wrong. The obvious answer is to retrain on the remaining data, which is always valid but often too expensive 1907.05012 §1 1907.05012 §3.

The cards use the term for three different things, and the difference drives every choice below.

• Exact deletion. The model after deletion should be distributed as if the point had never been used 1907.05012 §3. The question is how to get there faster than retraining. Certified removal relaxes this to an (ε, δ) bound on the output distribution 1911.03030 §2 eq. (1). Influence-function updates approximate the retrained weights in closed form, and carry a bound only for strongly convex losses 2108.11577 §5.2 2108.11577 §6.
• Behavioural unlearning. The model should stop producing the forgotten outputs. The LLM paper uses only negative examples, measures success on outputs, and does not claim the result matches a retrained model 2310.10683 §2 2310.10683 §6. The T5 adapter paper states a retrain-like target 2310.20150 eq. 1 but measures accuracy, not that target 2310.20150 §4.2.
• Adversarial evaluation. Not an unlearning method. It tests whether a model that was unlearned still produces the forgotten content when the prompt is chosen against it 2310.11868 §3.

Variants

Retrain from scratch

The baseline for every method. It gives exact deletion by definition, at the cost of a full training run per request 1907.05012 §1 1907.05012 §5. The deletion-efficient methods below are measured against it.

Quantized k-means (Q-k-means)

Keeps the k-means centroids fixed for most deletions. Each Lloyd step snaps the centroids to a lattice with a random phase. A deletion recomputes the perturbed centroid and retrains only if it snaps to a different lattice point 1907.05012 §4.1 1907.05012 alg. 5. The paper's expected deletion cost is independent of n for a constant number of deletions and a constant ε 1907.05012 Thm. 4.1. Card: Making AI Forget You: Data Deletion in Machine Learning.

Author-reported amortized speedups over k-means++ retraining run from 2.2x (MNIST) to 584x (Botnet) 1907.05012 tab. 4. The loss ratio is at most 1.158 on all six datasets (highest on Celltype) 1907.05012 tab. 1. On MNIST the retrain count per request rises at a near-constant rate, so quantization does not stabilise the centroids for long deletion streams 1907.05012 app. D.3.1.

Divide-and-conquer k-means (DC-k-means)

Splits the data into w random leaves, clusters each leaf, then clusters the pooled leaf centroids at the root. A deletion re-clusters one leaf and the root 1907.05012 §4.2 1907.05012 alg. 6. The paper uses depth one; deeper trees are not evaluated 1907.05012 §4.2. Author-reported speedups run from 13x to 71x 1907.05012 tab. 4. The worst loss ratio is 1.439 on Celltype 1907.05012 tab. 1. Card: the same paper.

Certified removal with a Newton step (convex linear and logistic models)

Trains a regularised linear or logistic model with a random linear term added to the loss 1911.03030 §3.1 alg. 1. A deletion takes one Newton step from the trained weights, using the Hessian of the reduced data 1911.03030 §3.1 eq. 3. That step leaves a gradient residual, which the paper bounds in norm, with a data-dependent form that is much tighter than the worst case 1911.03030 §3.1 thm. 1 1911.03030 §3.2 cor. 1. The random term hides the residual. With Gaussian noise scaled to the bound, the removed model is (ε, δ)-certified against a retrained one [arxiv:1911.03030 §3.1 thm. 3]. Each removal adds its bound to a running total. When the total passes σε/c, the model is retrained from scratch 1911.03030 §3.2 alg. 2. For least squares the residual is zero and the step is exact, with no noise 1911.03030 §3.2. Card: Certified Data Removal from Machine Learning Models.

Author-reported, on linear models over fixed features, a removal takes 0.04 to 0.48 s against 15.6 to 124 s for training 1911.03030 tab. 1. On LSUN with ten one-vs-all tasks, test accuracy is 83.3% while supporting over 10,000 removals at ε = 1, against 88.6% with no removal support 1911.03030 §4.2. On SVHN, a linear layer on a DP extractor is updated in 0.27 s, against more than 1.5 h to retrain the CNN, and reaches 71.2% test accuracy against 22.7% for a fully DP-trained CNN at ε ≈ 0.1 1911.03030 §4.3 1911.03030 tab. 1. The Hessian can be formed at training time at O(d²n) cost, and each removal then needs an O(d³) inversion, so large feature dimensions get slow 1911.03030 §3.1. The guarantee does not cover non-convex losses 1911.03030 §6.

Influence-function update for features and labels

Changes the values of selected features or labels on the affected points, rather than deleting those points 2108.11577 §4.2. Each affected point is paired with a corrected copy, and the update moves the trained weights by the gradient difference between the corrected and the original points 2108.11577 §4.1 eq. 5 2108.11577 §5.2 eq. 8. Two rules are given. The first-order rule is a step scaled by a rate that must be calibrated 2108.11577 §5.1 eq. 6. The second-order rule multiplies the difference by the inverse Hessian and needs no rate 2108.11577 §5.2. Revoking a feature zeroes it, applies the update, then removes the input units; the paper shows that zeroing and training equals removing only when the first layer is linear 2108.11577 §4.2 lem. 1. For strongly convex L2-regularised losses, noise added to the objective gives an (ε, δ) certificate against retraining, using a residual bound that the paper states for each rule 2108.11577 §6 thm. 1 2108.11577 §6 thm. 3. For large models the inverse Hessian is approximated by a truncated Neumann-series recursion, which weakens the guarantee 2108.11577 §5.2 2108.11577 app. A.3. Card: Machine Unlearning of Features and Labels.

Author-reported runtimes on logistic regression, 100 combinations of 100 features on the Drebin data, are 6.82 s for retraining, 0.02 s for the first-order update and 0.63 s for the second-order update 2108.11577 tab. 2. The text gives 90x for the first-order update; the runtimes in that table give about 341x 2108.11577 §7.1 2108.11577 tab. 2. In the appendix, second-order updates are slower than retraining on Enron (0.3x) and faster on Adult (98x) 2108.11577 tab. 5. On a character LSTM with a canary number repeated in the training data, both updates bring the measured exposure to 0 in the table, against 43 to 109 for the original model. Fine-tuning leaves it at 31 to 57 2108.11577 tab. 3. The first-order update is reported as about three orders of magnitude faster than retraining there, and the second-order update 28x 2108.11577 §7.2. On a CIFAR-10 CNN with 2500 poisoned labels, both updates and fine-tuning come close to the clean model's accuracy; retraining takes over 15 minutes 2108.11577 §7.3. With 10000 poisoned labels the paper reports that no method restores it 2108.11577 §7.3.

Gradient ascent on forget data (LLMs)

Lowers the likelihood of undesirable answers by gradient ascent on the answer tokens only 2310.10683 §4 2310.10683 eq. 3. Plain ascent collapses the model, so the method adds a random-mismatch term, which trains each forget prompt toward an unrelated answer, and a KL term that keeps normal-data predictions close to the original model 2310.10683 §4 2310.10683 eq. 2. Card: Large Language Model Unlearning.

Author-reported on harmful answers, the unlearned harmful rate falls from 47-54% to 1-6% on OPT-1.3B, OPT-2.7B and Llama 2 (7B) 2310.10683 tab. 3. Copyright leaks fall to 0-1% 2310.10683 tab. 4. Hallucination falls much less, to 8.5-15% of unlearned answers across the GA and GA+Mismatch rows 2310.10683 tab. 5. The author-reported cost is about 2% of an RLHF pipeline's run time for OPT-1.3B 2310.10683 §9.1. Plain ascent without the extra terms reaches near-zero harm, but the paper reports nonsense on normal prompts 2310.10683 §3.2 2310.10683 tab. 2. Finetuning, the paper's baseline, leaves the harmful rate at 34.5% for OPT-1.3B 2310.10683 tab. 3.

Selective teacher-student KL with unlearning adapters (EUL, T5)

Freezes a T5 model and trains a small adapter, about 0.5% of the parameters, for each deletion request 2310.20150 §3.1 2310.20150 §4.4. On retained data the adapter's output is pulled toward the teacher's, with a task loss added; on forget data it is pushed away from the teacher, with a negated masked-LM loss added 2310.20150 eq. 2 2310.20150 eq. 5. Several per-request adapters can be merged into one by a closed-form least-squares formula over stored Gram matrices of the hidden states. This needs no extra training and no stored data 2310.20150 §3.2 2310.20150 eq. 7. Card: Unlearn What You Want to Forget: Efficient Unlearning for LLMs.

Author-reported, on IMDB with T5-base at 0.5% forgotten: forget-set accuracy 65.7 against 92.5 for re-training, with update time 1200 s against 6685 s. Test accuracy is 93.0 against 92.8 2310.20150 tab. 2. On the same row Reverse-Gradient reaches 68.6 and MEND reaches 73.5 with 34 s, so MEND is faster in every block of the table. The text's claim that EUL updates faster than the earlier baselines holds only against those baselines 2310.20150 §4.5 2310.20150 tab. 2. The MLM loss on forgotten data rises to 1.78 against 1.46 for the original model, which the paper reads as the forgotten text being harder to extract 2310.20150 §4.2 2310.20150 tab. 2. On SAMSum with T5-base at 0.5%, forget-set ROUGE-1 is 38.4 against 42.4 for re-training, at 17060 s against 28000 s 2310.20150 tab. 3.

No reference code exists at the pinned commit. The paper does not name the optimizer, does not say which loss terms run on which epoch, does not state the number of alternation rounds, and does not say whether the summed Gram matrix in the fusion is invertible 2310.20150 §3.1 2310.20150 §3.2 2310.20150 §4.4.

Adversarial prompt search against unlearned diffusion models (evaluation)

Not an unlearning method. It tests whether a text-to-image model that was unlearned to refuse a concept still produces it. The adversary has the weights (white-box) and prepends 3 to 5 optimised token slots to the prompt 2310.11868 §3. The slots are scored with the victim's own denoising error on a target image that contains the forgotten content, so no second model is needed 2310.11868 §4 2310.11868 §5.1. Card: To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now.

Author-reported attack success on I2P nudity prompts against ESD: 20.42% with no attack, 69.71% with P4D and 76.05% with this attack 2310.11868 tab. 2. Against SLD the rate rises from 33.10% to 82.39%. Against FMN it is 88.03% with no attack and 97.89% with either attack 2310.11868 tab. 2. Attack time is 26.29 min per prompt against 34.70 min for P4D, on one A6000 GPU 2310.11868 tab. 2 2310.11868 §5.2. The reported rate adds the pre-attack and post-attack rates, so prompts that already succeed count 2310.11868 §5.1. The nudity judge is NudeNet and the harm judge is Q16, and the paper gives no sensitivity analysis for either 2310.11868 §5.1.

When to use which

Constraints that change the choice:

• Hyperparameters. The paper's ε and w follow rules of thumb that are not implemented in the released code 1907.05012 app. D.1.2. Certified removal's λ and noise scale σ are swept rather than fixed per experiment, and its δ is 1e-4 in the reported runs 1911.03030 §4.1 fig. 1. The LLM learning rates differ by model and method, from 2e-6 to 2e-4 2310.10683 app. B. The T5 adapter uses alpha 0.8, lambda 1.0 and gamma 0.2, chosen from one grid whose search split is not stated 2310.20150 §4.4.
• Calibration. The first-order rate is only meaningful relative to the loss scale and the corrected set; the feature paper calibrates it with an exposure metric and gives its CNN rate without a procedure 2108.11577 §5.1 2108.11577 app. A.9. Each further update round adds to the residual bound 2108.11577 app. A.6, and each second-order round needs a new Hessian 2108.11577 app. A.7.3.
• Evaluation. The LLM harmful rate depends on the PKU moderation classifier, which was trained on the same forget data 2310.10683 §6. Measure utility with a reward model and similarity to the original outputs, not with the forget-set loss, which can keep rising while harmful answers persist 2310.10683 §3.2 2310.10683 tab. 1. The T5 membership attack is near chance even before unlearning (0.542 on IMDB), so it cannot show that forgetting happened 2310.20150 tab. 7 2310.20150 §4.6.
• Threat model. Output-level tests on fixed prompts can report success while an adversarial prompt still restores the concept 2310.11868 tab. 2. The LLM paper tests unseen similar prompts, not adversarial ones 2310.10683 §6. Run an adversarial test alongside any plain test when the model will be public. Fidelity alone can mislead: fine-tuning keeps accuracy close to retraining while leaving the canary exposed 2108.11577 tab. 3 2108.11577 §7.2. Certified removal's guarantee holds only if the random term stays hidden from whoever checks for the removal, and the paper does not say how that term is stored or discarded 1911.03030 §3.1.
• Reimplementation. The diffusion attack's paper names AdamW, while the public code uses Adam with coupled weight decay and a different loss scale. The card lists the details 2310.11868 §5.1.

Implementation sketch

The k-means deletion check is cheap and falls back to retraining. Each variant only changes what the check touches.

# Deletion request for a model that can delete locally  [arxiv:1907.05012 alg. 5]
# Returns the updated model, or retrains when the stored state is stale

def handle_deletion(model, index, data):
    # A deleted initial seed always forces a retrain  [arxiv:1907.05012 alg. 5]
    if model.is_initial_seed(index):
        return retrain(data, exclude=index)
    # Check each stored step; any change in the snapped centroid forces retrain
    for step in model.accepted_steps():
        perturbed = model.perturb_centroid(step, index)
        if model.snap(step, perturbed) != model.stored_snap(step, index):
            return retrain(data, exclude=index)
        model.store_perturbed(step, perturbed)
    return model

The LLM variant changes only the loss. The forget set enters with a negative sign, and two regularisers keep the rest of the model intact 2310.10683 eq. 2.

# One unlearning step with gradient ascent and two regularisers  [arxiv:2310.10683 eq. 2]
# theta: model being unlearned, theta_o: frozen original model

def unlearning_step(theta, theta_o, forget_batch, normal_batch, random_answers):
    # Ascent on the answer tokens only, prompt tokens excluded  [arxiv:2310.10683 eq. 3]
    loss_forget = -answer_nll(theta, forget_batch)
    # Push each forget prompt toward an unrelated answer  [arxiv:2310.10683 eq. 5]
    loss_random = answer_nll(theta, forget_batch.prompts, random_answers)
    # Keep the output distribution on normal data close to theta_o  [arxiv:2310.10683 eq. 6]
    loss_normal = forward_kl(theta_o, theta, normal_batch)
    return eps1 * loss_forget + eps2 * loss_random + eps3 * loss_normal

Where variants differ: the k-means check runs per deletion request and never updates weights; the feature and label update replaces the removal set with corrected points, and its step is the gradient difference of eq. 8 2108.11577 §5.2 eq. 8; the LLM step updates weights and must be validated on outputs after training 2310.10683 §6. The T5 adapter step has the same sign pattern as the LLM step, with a teacher-student KL in place of the answer loss 2310.20150 eq. 2. The adversarial search does not update weights at all: it optimises prompt tokens against a fixed model and stops at the first unsafe image 2310.11868 §4. Certified removal updates weights once per removal, and a running budget check can force a full retrain instead 1911.03030 alg. 2.

Contradictions

• What counts as success. The clustering paper defines deletion against retraining in distribution 1907.05012 §3. The LLM paper measures success on outputs and calls blank or nonsensical replies acceptable 2310.10683 §2 2310.10683 §6. The T5 paper states a retrain-like target 2310.20150 eq. 1 but measures accuracy and a near-chance membership attack 2310.20150 §4.6. The certified-removal paper defines removal as (ε, δ) indistinguishability from retraining 1911.03030 §2 eq. (1), and its experiments report accuracy, timing and the residual bound 1911.03030 §4. The results cannot be compared as one guarantee.
• Do output-level results survive adversarial prompts? The LLM paper reports low harmful rates on unlearned and unseen similar prompts 2310.10683 §6. The diffusion paper reports that adversarial prefixes restore nudity to 76.05% on an ESD-unlearned model 2310.11868 tab. 2. The two papers study different model families and different tests, so they are not directly comparable, but both point the same way.
• Feature paper's own numbers. Its text gives 90x for the first-order update on Drebin, and its table's runtimes give about 341x. The text gives 4x for the second-order update, and the runtimes give about 11x 2108.11577 §7.1 2108.11577 tab. 2. Its printed recursion in Algorithm 1 neither divides by the scale parameter nor carries the minus sign of eq. 8 2108.11577 alg. 1 2108.11577 §5.2 eq. 8; the paper card's Pitfalls section lists the conventions to check (Machine Unlearning of Features and Labels).
• Speedup claims. The k-means abstract's "over 100x" holds for the mean of the Q-k-means speedups (about 226x) but not for DC-k-means (about 30x), computed from 1907.05012 tab. 4. The paper calls the three methods "competitive" in quality 1907.05012 §5.2.

Papers

• Making AI Forget You: Data Deletion in Machine Learning: Q-k-means and DC-k-means, which delete one training point from k-means without retraining from scratch.
• Certified Data Removal from Machine Learning Models: a Newton-step removal for convex linear and logistic models, with random loss perturbation for an (ε, δ) certificate and a budget that triggers retraining.
• Machine Unlearning of Features and Labels: influence-function updates that revoke features or replace labels, first-order with a calibrated rate and second-order with no rate, with an (ε, δ) bound for strongly convex losses.
• Large Language Model Unlearning: gradient ascent on forget answers with random-mismatch and KL terms, applied to harmful, copyrighted and hallucinated answers.
• Unlearn What You Want to Forget: Efficient Unlearning for LLMs: per-request adapters on a frozen T5 with a selective teacher-student KL, merged in closed form for a sequence of deletions.
• To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now: an adversarial prompt-prefix search that restores concepts erased from text-to-image diffusion models.