---
kind: concept
slug: machine-unlearning
title: Machine unlearning
summary: >-
  Removing a training point's influence from a trained model without full retraining. Exact deletion must match a retrained model; behavioural unlearning only stops forgotten outputs, and adversarial prompts can bring some back. Author-reported: k-means deletion 2.2x to 584x faster than retraining; certified linear removal 250x to 870x; LLM gradient ascent cuts harmful outputs to 1-6%.
tags: [privacy, unlearning, data-deletion, llm-alignment, adapters, diffusion-safety]
papers: ["1907.05012", "1911.03030", "2108.11577", "2310.10683", "2310.20150", "2310.11868"]
related: [k-means-clustering, differential-privacy, influence-functions, data-poisoning, rlhf, hallucination-mitigation, membership-inference, model-editing, parameter-efficient-fine-tuning, diffusion-models, red-teaming]
status: draft
updated_at: 2026-11-01
---

## Overview

Machine unlearning asks a trained model to forget some training data: a user
withdraws consent, a record is found to be harmful or copyrighted, or a
training point is wrong. The obvious answer is to retrain on the remaining
data, which is always valid but often too expensive [arxiv:1907.05012 §1]
[arxiv:1907.05012 §3].

The cards use the term for three different things, and the difference drives
every choice below.

- **Exact deletion.** The model after deletion should be distributed as if the
  point had never been used [arxiv:1907.05012 §3]. The question is how to get
  there faster than retraining. Certified removal relaxes this to an (ε, δ)
  bound on the output distribution [arxiv:1911.03030 §2 eq. (1)]. Influence-function
  updates approximate the retrained weights in closed form, and carry a bound only
  for strongly convex losses [arxiv:2108.11577 §5.2] [arxiv:2108.11577 §6].
- **Behavioural unlearning.** The model should stop producing the forgotten
  outputs. The LLM paper uses only negative examples, measures success on outputs,
  and does not claim the result matches a retrained model [arxiv:2310.10683 §2]
  [arxiv:2310.10683 §6]. The T5 adapter paper states a retrain-like target
  [arxiv:2310.20150 eq. 1] but measures accuracy, not that target [arxiv:2310.20150 §4.2].
- **Adversarial evaluation.** Not an unlearning method. It tests whether a model
  that was unlearned still produces the forgotten content when the prompt is
  chosen against it [arxiv:2310.11868 §3].

## Variants

### Retrain from scratch

The baseline for every method. It gives exact deletion by definition, at the
cost of a full training run per request [arxiv:1907.05012 §1] [arxiv:1907.05012 §5].
The deletion-efficient methods below are measured against it.

### Quantized k-means (Q-k-means)

Keeps the k-means centroids fixed for most deletions. Each Lloyd step snaps
the centroids to a lattice with a random phase. A deletion recomputes the
perturbed centroid and retrains only if it snaps to a different lattice point
[arxiv:1907.05012 §4.1] [arxiv:1907.05012 alg. 5]. The paper's expected
deletion cost is independent of n for a constant number of deletions and a
constant ε [arxiv:1907.05012 Thm. 4.1]. Card: [Making AI Forget You: Data Deletion in Machine Learning](/papers/1907.05012).

Author-reported amortized speedups over k-means++ retraining run from 2.2x
(MNIST) to 584x (Botnet) [arxiv:1907.05012 tab. 4]. The loss ratio is at
most 1.158 on all six datasets (highest on Celltype) [arxiv:1907.05012 tab. 1].
On MNIST the retrain count per request rises at a near-constant rate, so
quantization does not stabilise the centroids for long deletion streams
[arxiv:1907.05012 app. D.3.1].

### Divide-and-conquer k-means (DC-k-means)

Splits the data into w random leaves, clusters each leaf, then clusters the
pooled leaf centroids at the root. A deletion re-clusters one leaf and the
root [arxiv:1907.05012 §4.2] [arxiv:1907.05012 alg. 6]. The paper uses depth
one; deeper trees are not evaluated [arxiv:1907.05012 §4.2]. Author-reported
speedups run from 13x to 71x [arxiv:1907.05012 tab. 4]. The worst loss ratio
is 1.439 on Celltype [arxiv:1907.05012 tab. 1]. Card: the same paper.

### Certified removal with a Newton step (convex linear and logistic models)

Trains a regularised linear or logistic model with a random linear term added
to the loss [arxiv:1911.03030 §3.1 alg. 1]. A deletion takes one Newton step
from the trained weights, using the Hessian of the reduced data
[arxiv:1911.03030 §3.1 eq. 3]. That step leaves a gradient residual, which the
paper bounds in norm, with a data-dependent form that is much tighter than the
worst case [arxiv:1911.03030 §3.1 thm. 1] [arxiv:1911.03030 §3.2 cor. 1]. The
random term hides the residual. With Gaussian noise scaled to the bound, the
removed model is (ε, δ)-certified against a retrained one [arxiv:1911.03030 §3.1
thm. 3]. Each removal adds its bound to a running total. When the total passes
σε/c, the model is retrained from scratch [arxiv:1911.03030 §3.2 alg. 2]. For
least squares the residual is zero and the step is exact, with no noise
[arxiv:1911.03030 §3.2]. Card: [Certified Data Removal from Machine Learning Models](/papers/1911.03030).

Author-reported, on linear models over fixed features, a removal takes 0.04 to
0.48 s against 15.6 to 124 s for training [arxiv:1911.03030 tab. 1]. On LSUN
with ten one-vs-all tasks, test accuracy is 83.3% while supporting over 10,000
removals at ε = 1, against 88.6% with no removal support [arxiv:1911.03030 §4.2].
On SVHN, a linear layer on a DP extractor is updated in 0.27 s, against more
than 1.5 h to retrain the CNN, and reaches 71.2% test accuracy against 22.7% for
a fully DP-trained CNN at ε ≈ 0.1 [arxiv:1911.03030 §4.3] [arxiv:1911.03030 tab. 1].
The Hessian can be formed at training time at O(d²n) cost, and each removal
then needs an O(d³) inversion, so large feature dimensions get slow
[arxiv:1911.03030 §3.1]. The guarantee does not
cover non-convex losses [arxiv:1911.03030 §6].

### Influence-function update for features and labels

Changes the values of selected features or labels on the affected points,
rather than deleting those points [arxiv:2108.11577 §4.2]. Each affected point
is paired with a corrected copy, and the update moves the trained weights by
the gradient difference between the corrected and the original points
[arxiv:2108.11577 §4.1 eq. 5] [arxiv:2108.11577 §5.2 eq. 8]. Two rules are
given. The first-order rule is a step scaled by a rate that must be calibrated
[arxiv:2108.11577 §5.1 eq. 6]. The second-order rule multiplies the difference
by the inverse Hessian and needs no rate [arxiv:2108.11577 §5.2]. Revoking a
feature zeroes it, applies the update, then removes the input units; the paper
shows that zeroing and training equals removing only when the first layer is
linear [arxiv:2108.11577 §4.2 lem. 1]. For strongly convex L2-regularised
losses, noise added to the objective gives an (ε, δ) certificate against
retraining, using a residual bound that the paper states for each rule
[arxiv:2108.11577 §6 thm. 1] [arxiv:2108.11577 §6 thm. 3]. For large models
the inverse Hessian is approximated by a truncated Neumann-series recursion,
which weakens the guarantee [arxiv:2108.11577 §5.2] [arxiv:2108.11577 app. A.3].
Card: [Machine Unlearning of Features and Labels](/papers/2108.11577).

Author-reported runtimes on logistic regression, 100 combinations of 100
features on the Drebin data, are 6.82 s for retraining, 0.02 s for the
first-order update and 0.63 s for the second-order update [arxiv:2108.11577 tab. 2].
The text gives 90x for the first-order update; the runtimes in that table give
about 341x [arxiv:2108.11577 §7.1] [arxiv:2108.11577 tab. 2]. In the appendix,
second-order updates are slower than retraining on Enron (0.3x) and faster on
Adult (98x) [arxiv:2108.11577 tab. 5]. On a character LSTM with a canary
number repeated in the training data, both updates bring the measured exposure
to 0 in the table, against 43 to 109 for the original model. Fine-tuning leaves
it at 31 to 57 [arxiv:2108.11577 tab. 3]. The first-order update is reported as
about three orders of magnitude faster than retraining there, and the
second-order update 28x [arxiv:2108.11577 §7.2]. On a CIFAR-10 CNN with 2500
poisoned labels, both updates and fine-tuning come close to the clean model's
accuracy; retraining takes over 15 minutes [arxiv:2108.11577 §7.3]. With 10000
poisoned labels the paper reports that no method restores it [arxiv:2108.11577 §7.3].

### Gradient ascent on forget data (LLMs)

Lowers the likelihood of undesirable answers by gradient ascent on the answer
tokens only [arxiv:2310.10683 §4] [arxiv:2310.10683 eq. 3]. Plain ascent
collapses the model, so the method adds a random-mismatch term, which trains
each forget prompt toward an unrelated answer, and a KL term that keeps
normal-data predictions close to the original model [arxiv:2310.10683 §4]
[arxiv:2310.10683 eq. 2]. Card: [Large Language Model Unlearning](/papers/2310.10683).

Author-reported on harmful answers, the unlearned harmful rate falls from
47-54% to 1-6% on OPT-1.3B, OPT-2.7B and Llama 2 (7B) [arxiv:2310.10683 tab. 3].
Copyright leaks fall to 0-1% [arxiv:2310.10683 tab. 4]. Hallucination falls
much less, to 8.5-15% of unlearned answers across the GA and GA+Mismatch rows
[arxiv:2310.10683 tab. 5]. The author-reported cost is about 2% of an RLHF
pipeline's run time for OPT-1.3B [arxiv:2310.10683 §9.1]. Plain ascent without
the extra terms reaches near-zero harm, but the paper reports nonsense on
normal prompts [arxiv:2310.10683 §3.2] [arxiv:2310.10683 tab. 2]. Finetuning,
the paper's baseline, leaves the harmful rate at 34.5% for OPT-1.3B
[arxiv:2310.10683 tab. 3].

### Selective teacher-student KL with unlearning adapters (EUL, T5)

Freezes a T5 model and trains a small adapter, about 0.5% of the parameters,
for each deletion request [arxiv:2310.20150 §3.1] [arxiv:2310.20150 §4.4]. On
retained data the adapter's output is pulled toward the teacher's, with a task
loss added; on forget data it is pushed away from the teacher, with a negated
masked-LM loss added [arxiv:2310.20150 eq. 2] [arxiv:2310.20150 eq. 5]. Several
per-request adapters can be merged into one by a closed-form least-squares
formula over stored Gram matrices of the hidden states. This needs no extra
training and no stored data [arxiv:2310.20150 §3.2] [arxiv:2310.20150 eq. 7].
Card: [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](/papers/2310.20150).

Author-reported, on IMDB with T5-base at 0.5% forgotten: forget-set accuracy
65.7 against 92.5 for re-training, with update time 1200 s against 6685 s.
Test accuracy is 93.0 against 92.8 [arxiv:2310.20150 tab. 2]. On the same row
Reverse-Gradient reaches 68.6 and MEND reaches 73.5 with 34 s, so MEND is faster
in every block of the table. The text's claim that EUL updates faster than the
earlier baselines holds only against those baselines [arxiv:2310.20150 §4.5]
[arxiv:2310.20150 tab. 2]. The MLM loss on forgotten data rises to 1.78 against
1.46 for the original model, which the paper reads as the forgotten text being
harder to extract [arxiv:2310.20150 §4.2] [arxiv:2310.20150 tab. 2]. On SAMSum
with T5-base at 0.5%, forget-set ROUGE-1 is 38.4 against 42.4 for re-training,
at 17060 s against 28000 s [arxiv:2310.20150 tab. 3].

No reference code exists at the pinned commit. The paper does not name the
optimizer, does not say which loss terms run on which epoch, does not state
the number of alternation rounds, and does not say whether the summed Gram
matrix in the fusion is invertible [arxiv:2310.20150 §3.1] [arxiv:2310.20150 §3.2]
[arxiv:2310.20150 §4.4].

### Adversarial prompt search against unlearned diffusion models (evaluation)

Not an unlearning method. It tests whether a text-to-image model that was
unlearned to refuse a concept still produces it. The adversary has the weights
(white-box) and prepends 3 to 5 optimised token slots to the prompt [arxiv:2310.11868 §3].
The slots are scored with the victim's own denoising error on a target image
that contains the forgotten content, so no second model is needed [arxiv:2310.11868 §4]
[arxiv:2310.11868 §5.1]. Card: [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](/papers/2310.11868).

Author-reported attack success on I2P nudity prompts against ESD: 20.42% with
no attack, 69.71% with P4D and 76.05% with this attack [arxiv:2310.11868 tab. 2].
Against SLD the rate rises from 33.10% to 82.39%. Against FMN it is 88.03% with
no attack and 97.89% with either attack [arxiv:2310.11868 tab. 2]. Attack time
is 26.29 min per prompt against 34.70 min for P4D, on one A6000 GPU
[arxiv:2310.11868 tab. 2] [arxiv:2310.11868 §5.2]. The reported rate adds the
pre-attack and post-attack rates, so prompts that already succeed count
[arxiv:2310.11868 §5.1]. The nudity judge is NudeNet and the harm judge is Q16,
and the paper gives no sensitivity analysis for either [arxiv:2310.11868 §5.1].

## When to use which

| Situation | Use | Why |
|---|---|---|
| Exact removal is required and the model is k-means | Q-k-means or DC-k-means | The paper gives cost bounds for both [arxiv:1907.05012 §4] |
| Few deletions, not targeted at initial seeds | Q-k-means | Most deletions leave the centroids unchanged; the largest speedups are on Botnet and Gaussian [arxiv:1907.05012 tab. 4] |
| Deletions target the initial seeds or chosen leaves | Retrain, or DC-k-means with care | Adversarial seed deletion forces Q-k-means to retrain; leaf-targeted deletion can break DC-k-means [arxiv:1907.05012 app. A.1] |
| Clustering loss matters more than speed | Q-k-means over DC-k-means | On Celltype the loss ratio is 1.158 for Q-k-means and 1.439 for DC-k-means [arxiv:1907.05012 tab. 1] |
| Convex model (linear or logistic), a formal bound is needed, and removals are few | Certified removal with a Newton step | Removal takes under a second in the paper's linear runs [arxiv:1911.03030 tab. 1]; the bound is (ε, δ) against retraining [arxiv:1911.03030 §3.1] |
| Convex model with a long stream of removals | Certified removal, sized for the stream | The budget σε/c sets how many removals fit before a full retrain [arxiv:1911.03030 §3.2 alg. 2]; the LSUN run reports over 10,000 supported removals at ε = 1 [arxiv:1911.03030 §4.2] |
| Deep models, exact removal required | Retrain | No card gives an exact-deletion method for neural networks; certified removal is limited to convex losses [arxiv:1911.03030 §6]; the feature and label update is only checked by outside tests for neural networks [arxiv:2108.11577 §8]; EUL's target is stated but not tested [arxiv:2310.20150 §4.2] |
| Feature values or labels must be corrected in place, on a convex model, with a rate that can be calibrated | Influence-function update, first-order | Fastest in the paper's linear run: 0.02 s against 6.82 s for retraining [arxiv:2108.11577 tab. 2]; the rate needs calibration [arxiv:2108.11577 §5.1] |
| Same setting, but no rate may be tuned, or the loss scale is unknown | Influence-function update, second-order | No rate is needed [arxiv:2108.11577 §5.2]; 0.63 s in the same run [arxiv:2108.11577 tab. 2]; slower than retraining on Enron [arxiv:2108.11577 tab. 5] |
| A formal (ε, δ) certificate is required for a strongly convex L2-regularised model | Second-order influence-function update with calibrated noise | The bound holds for that loss class and for a residual bound estimated from sampled feature combinations [arxiv:2108.11577 §6 thm. 1] [arxiv:2108.11577 §7.1] |
| A neural network memorised a string and the string must stop being extractable | Influence-function update, checked by an extraction test | Exposure reaches 0 in the table; fine-tuning does not [arxiv:2108.11577 tab. 3] |
| Deletion requests arrive one at a time for a T5-size model and the base weights must stay frozen | EUL adapter per request | Trains about 0.5% of parameters per request [arxiv:2310.20150 §4.4] |
| Several per-request adapters must be served at once | EUL-fuse | The closed-form merge needs no extra training [arxiv:2310.20150 §3.2]; the paper does not check invertibility of the summed Gram matrix |
| LLM must stop emitting specific harmful or copyrighted text, only negative examples exist | Gradient ascent with random-mismatch and KL terms | The paper's only setting without positive examples [arxiv:2310.10683 §2] |
| LLM must still give helpful answers | Not supported by these cards | The authors report blank or nonsense outputs on the forgotten prompts and treat that as the goal [arxiv:2310.10683 §2] [arxiv:2310.10683 §6] |
| Hallucinated facts must go | Gradient ascent with random-mismatch and KL terms | Works less well: 8.5-15% of unlearned answers still hallucinate [arxiv:2310.10683 tab. 5] |
| Text-to-image model unlearned for a concept, before release | Adversarial prompt search, with a no-attack baseline | Shows the gap between plain and adversarial prompts (ESD nudity 20.42% vs 76.05%) [arxiv:2310.11868 tab. 2]. The paper assumes white-box access and does not test black-box attacks [arxiv:2310.11868 §3] |

Constraints that change the choice:

- **Hyperparameters.** The paper's ε and w follow rules of thumb that are not
  implemented in the released code [arxiv:1907.05012 app. D.1.2]. Certified
  removal's λ and noise scale σ are swept rather than fixed per experiment, and
  its δ is 1e-4 in the reported runs [arxiv:1911.03030 §4.1] [fig. 1]. The LLM
  learning rates differ by model and method, from 2e-6 to 2e-4 [arxiv:2310.10683 app. B].
  The T5 adapter uses alpha 0.8, lambda 1.0 and gamma 0.2, chosen from one
  grid whose search split is not stated [arxiv:2310.20150 §4.4].
- **Calibration.** The first-order rate is only meaningful relative to the loss
  scale and the corrected set; the feature paper calibrates it with an exposure
  metric and gives its CNN rate without a procedure [arxiv:2108.11577 §5.1]
  [arxiv:2108.11577 app. A.9]. Each further update round adds to the residual
  bound [arxiv:2108.11577 app. A.6], and each second-order round needs a new
  Hessian [arxiv:2108.11577 app. A.7.3].
- **Evaluation.** The LLM harmful rate depends on the PKU moderation classifier,
  which was trained on the same forget data [arxiv:2310.10683 §6]. Measure
  utility with a reward model and similarity to the original outputs, not with
  the forget-set loss, which can keep rising while harmful answers persist
  [arxiv:2310.10683 §3.2] [arxiv:2310.10683 tab. 1]. The T5 membership attack
  is near chance even before unlearning (0.542 on IMDB), so it cannot show that
  forgetting happened [arxiv:2310.20150 tab. 7] [arxiv:2310.20150 §4.6].
- **Threat model.** Output-level tests on fixed prompts can report success while
  an adversarial prompt still restores the concept [arxiv:2310.11868 tab. 2].
  The LLM paper tests unseen similar prompts, not adversarial ones [arxiv:2310.10683 §6].
  Run an adversarial test alongside any plain test when the model will be public.
  Fidelity alone can mislead: fine-tuning keeps accuracy close to retraining while
  leaving the canary exposed [arxiv:2108.11577 tab. 3] [arxiv:2108.11577 §7.2].
  Certified removal's guarantee holds only if the random term stays hidden from
  whoever checks for the removal, and the paper does not say how that term is
  stored or discarded [arxiv:1911.03030 §3.1].
- **Reimplementation.** The diffusion attack's paper names AdamW, while the
  public code uses Adam with coupled weight decay and a different loss scale.
  The card lists the details [arxiv:2310.11868 §5.1].

## Implementation sketch

The k-means deletion check is cheap and falls back to retraining. Each
variant only changes what the check touches.

```pseudocode
# Deletion request for a model that can delete locally  [arxiv:1907.05012 alg. 5]
# Returns the updated model, or retrains when the stored state is stale

def handle_deletion(model, index, data):
    # A deleted initial seed always forces a retrain  [arxiv:1907.05012 alg. 5]
    if model.is_initial_seed(index):
        return retrain(data, exclude=index)
    # Check each stored step; any change in the snapped centroid forces retrain
    for step in model.accepted_steps():
        perturbed = model.perturb_centroid(step, index)
        if model.snap(step, perturbed) != model.stored_snap(step, index):
            return retrain(data, exclude=index)
        model.store_perturbed(step, perturbed)
    return model
```

The LLM variant changes only the loss. The forget set enters with a negative
sign, and two regularisers keep the rest of the model intact [arxiv:2310.10683 eq. 2].

```pseudocode
# One unlearning step with gradient ascent and two regularisers  [arxiv:2310.10683 eq. 2]
# theta: model being unlearned, theta_o: frozen original model

def unlearning_step(theta, theta_o, forget_batch, normal_batch, random_answers):
    # Ascent on the answer tokens only, prompt tokens excluded  [arxiv:2310.10683 eq. 3]
    loss_forget = -answer_nll(theta, forget_batch)
    # Push each forget prompt toward an unrelated answer  [arxiv:2310.10683 eq. 5]
    loss_random = answer_nll(theta, forget_batch.prompts, random_answers)
    # Keep the output distribution on normal data close to theta_o  [arxiv:2310.10683 eq. 6]
    loss_normal = forward_kl(theta_o, theta, normal_batch)
    return eps1 * loss_forget + eps2 * loss_random + eps3 * loss_normal
```

Where variants differ: the k-means check runs per deletion request and never
updates weights; the feature and label update replaces the removal set with
corrected points, and its step is the gradient difference of eq. 8
[arxiv:2108.11577 §5.2 eq. 8]; the LLM step updates weights and must be validated on outputs
after training [arxiv:2310.10683 §6]. The T5 adapter step has the same sign
pattern as the LLM step, with a teacher-student KL in place of the answer loss
[arxiv:2310.20150 eq. 2]. The adversarial search does not update weights at
all: it optimises prompt tokens against a fixed model and stops at the first
unsafe image [arxiv:2310.11868 §4]. Certified removal updates weights once per
removal, and a running budget check can force a full retrain instead
[arxiv:1911.03030 alg. 2].

## Contradictions

- **What counts as success.** The clustering paper defines deletion against
  retraining in distribution [arxiv:1907.05012 §3]. The LLM paper measures
  success on outputs and calls blank or nonsensical replies acceptable
  [arxiv:2310.10683 §2] [arxiv:2310.10683 §6]. The T5 paper states a
  retrain-like target [arxiv:2310.20150 eq. 1] but measures accuracy and a
  near-chance membership attack [arxiv:2310.20150 §4.6]. The certified-removal
  paper defines removal as (ε, δ) indistinguishability from retraining
  [arxiv:1911.03030 §2 eq. (1)], and its experiments report accuracy, timing and
  the residual bound [arxiv:1911.03030 §4]. The results cannot be compared as
  one guarantee.
- **Do output-level results survive adversarial prompts?** The LLM paper reports
  low harmful rates on unlearned and unseen similar prompts [arxiv:2310.10683 §6].
  The diffusion paper reports that adversarial prefixes restore nudity to 76.05%
  on an ESD-unlearned model [arxiv:2310.11868 tab. 2]. The two papers study
  different model families and different tests, so they are not directly
  comparable, but both point the same way.
- **Feature paper's own numbers.** Its text gives 90x for the first-order update
  on Drebin, and its table's runtimes give about 341x. The text gives 4x for the
  second-order update, and the runtimes give about 11x [arxiv:2108.11577 §7.1]
  [arxiv:2108.11577 tab. 2]. Its printed recursion in Algorithm 1 neither divides
  by the scale parameter nor carries the minus sign of eq. 8
  [arxiv:2108.11577 alg. 1] [arxiv:2108.11577 §5.2 eq. 8]; the paper card's
  Pitfalls section lists the conventions to check
  ([Machine Unlearning of Features and Labels](/papers/2108.11577)).
- **Speedup claims.** The k-means abstract's "over 100x" holds for the mean of
  the Q-k-means speedups (about 226x) but not for DC-k-means (about 30x),
  computed from [arxiv:1907.05012 tab. 4]. The paper calls the three methods
  "competitive" in quality [arxiv:1907.05012 §5.2].

## Papers

- [Making AI Forget You: Data Deletion in Machine Learning](/papers/1907.05012): Q-k-means and DC-k-means, which delete one training point from k-means without retraining from scratch.
- [Certified Data Removal from Machine Learning Models](/papers/1911.03030): a Newton-step removal for convex linear and logistic models, with random loss perturbation for an (ε, δ) certificate and a budget that triggers retraining.
- [Machine Unlearning of Features and Labels](/papers/2108.11577): influence-function updates that revoke features or replace labels, first-order with a calibrated rate and second-order with no rate, with an (ε, δ) bound for strongly convex losses.
- [Large Language Model Unlearning](/papers/2310.10683): gradient ascent on forget answers with random-mismatch and KL terms, applied to harmful, copyrighted and hallucinated answers.
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](/papers/2310.20150): per-request adapters on a frozen T5 with a selective teacher-student KL, merged in closed form for a sequence of deletions.
- [To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now](/papers/2310.11868): an adversarial prompt-prefix search that restores concepts erased from text-to-image diffusion models.