---
kind: concept
slug: model-editing
title: Model editing
summary: >-
  Changing a trained model's answer to specific inputs without retraining it. Variants: fine-tuning with a locality penalty (baseline), learned gradient editors (MEND), memory-based editors (SERAC), and in-context editing. The trade-off is edit success against drawdown on unrelated inputs, and against weight changes or per-query cost. Weight editors often miss implied facts.
tags: [model-editing, knowledge-editing, fine-tuning, retrieval, factual-recall, interpretability]
papers: ["2110.11309", "2206.06520", "2304.14767", "2307.12976", "2310.20150", "2404.13501"]
related: [continual-learning, meta-learning, retrieval-augmented-generation, activation-patching, parameter-efficient-fine-tuning, in-context-learning, machine-unlearning]
status: draft
updated_at: 2026-10-19
---

## Overview

A trained language model sometimes gives a wrong answer that should change, for
example after a fact goes out of date. Model editing changes the answer to a
given input, or a set of inputs, and leaves the rest of the model's behaviour
alone. An edit is given either as an input and a desired output, or as a
free-text description [arxiv:2206.06520 §2].

The two properties every editor is judged on are the same across papers.
Edit success (ES) is how often the inputs the edit should cover get the
intended answer [arxiv:2110.11309 §4] [arxiv:2206.06520 eq. 4]. Drawdown (DD)
is how much the model changes on inputs it should not touch
[arxiv:2110.11309 §4] [arxiv:2206.06520 eq. 5]. The hard cases lie on both
sides: in-scope inputs that need reasoning, and out-of-scope inputs that look
like the edit [arxiv:2206.06520 §2]. Fine-tuning on one example tends to fail
both tests. It changes the answer, but rephrasings of the question and
unrelated questions can change too [arxiv:2110.11309 §1].

A third test asks whether the edit carries to facts that follow from it. If
the mother of a person changes, the children of the new mother are that
person's siblings, and those answers should change as well. The RippleEdits
benchmark scores this with six criteria: logical generalization, two
compositionality tests, subject aliasing, preservation and relation
specificity [arxiv:2307.12976 §3.1]. Parametric editors pass the aliasing
test, at least 86.8 in every setting, but on the Popular subset ROME and MEMIT
on GPT-2 and GPT-J score 5.5 to 7.0 on logical generalization
[arxiv:2307.12976 tab. 3] [arxiv:2307.12976 tab. 5].

A survey of agent memory places editing among the parametric forms of
memory. Its other form is textual memory kept in the prompt or a store. Its
editing entry only names MAC, MEND and KnowledgeEditor, and the survey runs no
experiments [arxiv:2404.13501 §5.2.2] [arxiv:2404.13501 §6]. Its point is that
parametric memory adds nothing to the prompt at read time, while textual memory
is cheaper to write [arxiv:2404.13501 §5.2.3]. That is the authors' reasoning,
with no measurements behind it. Card:
[Agent memory survey (Zhang et al.)](/papers/2404.13501).

## Variants

### Fine-tuning with a locality penalty (baseline)

Plain fine-tuning on the corrected example is the baseline. Adding a KL term
on unrelated data (FT + KL) keeps the model closer to its original behaviour
[arxiv:2110.11309 §4.1]. On the two decoder models in the MEND paper it
lowers perplexity drawdown, but it also lowers edit success on GPT-J, from
0.80 to 0.36 [arxiv:2110.11309 §4.1]. Fine-tuning reaches 0.87 ES on T5-XXL
with drawdown below 0.001 [arxiv:2110.11309 §4.1]. Card:
[MEND](/papers/2110.11309).

### Learned gradient editors (MEND)

A small editor network is trained ahead of time for each shape of edited
weight matrix. At edit time it takes a layer's fine-tuning gradient, kept as
two short factor vectors, and returns a low-rank weight change. The edit is a
single update with no iteration [arxiv:2110.11309 alg. 2]. Training uses an
edit set of tuples and a loss that combines edit success on an equivalent
input with a locality KL on unrelated inputs. The base model is not updated
[arxiv:2110.11309 §3.2]. The paper edits the MLP blocks of the last layers
[arxiv:2110.11309 §4]. Card: [MEND](/papers/2110.11309).

On the RippleEdits benchmark, MEND has the lowest CI and CII scores of the
parametric methods on GPT-2 XL, at 17.0 and 13.6 averaged over the subsets
[arxiv:2307.12976 tab. 6]. Its LG score on the same model is the highest of
the three, at 39.1 [arxiv:2307.12976 tab. 6].

### Memory-based editors (SERAC)

The base model is frozen. Each edit is stored as its descriptor in a cache,
and a scope classifier decides whether an input falls under an edit. If the
best-scoring edit scores at least 0.5, a small counterfactual model answers
from the edit and the input. Otherwise the base model answers [arxiv:2206.06520
§3.1] [arxiv:2206.06520 eq. 1]. The authors put the memory at about 3 KB per
edit [arxiv:2206.06520 §5.2]. Card: [SERAC](/papers/2206.06520).

### In-context editing (ICE)

No weights change. The edit is written as a sentence and placed before the
question, in the form "Imagine that <o*> would have been <P_r>", where P_r is a
manual proposition for the relation, such as "the mother of <e>"
[arxiv:2307.12976 §5.1]. RippleEdits uses this as a baseline, and the paper
reports it as the best on average across its three subsets [arxiv:2307.12976
§5.2]. The edit lives in the prompt, so it has to be sent with every query that
needs it. The paper does not measure that cost [arxiv:2307.12976 §5.1]. Card:
[RippleEdits](/papers/2307.12976).

### Adapter-based unlearning (a neighbouring goal)

Unlearning removes the influence of a set of training data, rather than
changing one answer. Chen and Yang add small unlearning layers to a frozen T5
model. Each layer is trained with a selective teacher-student KL term, a task
loss on retained data and a negated masked-LM loss on the forgotten data
[arxiv:2310.20150 §3.1]. Several such layers are merged in closed form from
stored Gram matrices, so no data is kept [arxiv:2310.20150 §3.2]. On IMDB with
T5-base and 0.5% of the data forgotten, forget-set accuracy is 65.7 against
92.5 for re-training, and the update takes 1200 s against 6685 s
[arxiv:2310.20150 tab. 2]. Card:
[Unlearn What You Want to Forget](/papers/2310.20150).

### Locating facts inside the model (analysis, not an editor)

Geva et al. trace how GPT-2 XL and GPT-J answer subject-relation queries. Blocking
attention from the last position to the subject tokens lowers the answer's
probability by up to 60% in middle-upper layers [arxiv:2304.14767 §5]. Early
MLP updates at the subject enrich its representation, and zeroing them lowers
the attribute rate by about 88% on average [arxiv:2304.14767 §6.3]. Upper
attention heads extract the attribute for 68.2% of GPT-2 XL queries
[arxiv:2304.14767 tab. 2]. The paper runs no editing experiment. Its results
say where recall happens, which helps choose layers for the variants above.
Card: [Dissecting recall of factual associations](/papers/2304.14767).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You can train a helper for each base model and have gradients through it | Gradient editor (MEND) | One step per edit. Reported ES 0.81 to 0.89 on the four large models, with lower perplexity drawdown than FT on GPT-Neo and GPT-J [arxiv:2110.11309 §4.1] |
| You cannot change the base weights, or edits must be added and removed as data | Memory-based (SERAC) | Base weights stay fixed and edits are stored as text [arxiv:2206.06520 §3.1]. The cost is a scope classifier and a counterfactual model, and their training data [arxiv:2206.06520 §4] |
| A few edits, a large encoder-decoder model, and no training budget | Fine-tuning, with a locality term if needed | Fine-tuning matched MEND on T5-XXL [arxiv:2110.11309 §4.1]. On GPT-J the picture differs: plain fine-tuning reached 0.80 ES, and adding the KL term cut it to 0.36 [arxiv:2110.11309 §4.1] |
| Edit must carry to implied facts, such as siblings or two-hop questions | ICE, or a weight editor only after its LG and CI scores are checked | On the Recent subset, LLaMA with ROME scores 16.7 on LG and 47.8 on CI, against 59.6 and 74.8 with ICE [arxiv:2307.12976 tab. 3]. ICE stores nothing, so the edit is only as durable as the prompt that carries it [arxiv:2307.12976 §5.1] |
| Many edits applied at once | Memory-based (SERAC), or check the MEND batch results | SERAC keeps ES minus DD near constant as edits grow to 75 on QA [arxiv:2206.06520 §5.1]. MEND reports 0.89 ES at 25 batched edits, on BART-base zsRE, not on the large models [arxiv:2110.11309 §4.3] |
| A set of training examples must be forgotten, e.g. after a deletion request | Adapter-based unlearning | Keeps the base model frozen and merges requests in closed form [arxiv:2310.20150 §3.2]. It is a different goal from single-answer editing, so it is not a substitute for the editors above |
| Choosing which weights to edit | MLP blocks, not attention | MEND edits attention worse than MLP on all four large models [arxiv:2110.11309 app. E]. Geva et al. find early MLPs enrich the subject [arxiv:2304.14767 §6.3] |

Latency and memory costs also differ. For SERAC on QA, a routed input takes
31 ms against 87 ms for the base model, and an input sent to the base model
takes 92 ms. Memory rises from 2.96 GB to 3.47 GB [arxiv:2206.06520 tab. 6].
On FC the scope pass takes 19 ms against 7 ms for the base model
[arxiv:2206.06520 tab. 6].

## Implementation sketch

Every variant reduces to the same evaluation: apply the edit, then measure
the two properties. The variants differ in where the edit lives. In MEND it is
a weight update [arxiv:2110.11309 alg. 2]. In SERAC it is an entry in the
cache plus a routing decision [arxiv:2206.06520 eq. 1]. In ICE it is a prefix
on the prompt [arxiv:2307.12976 §5.1].

```pseudocode
# Measure one edit: edit success and drawdown  [arxiv:2206.06520 eq. 4-5]
# Input:  edited model, base model, in-scope (x, y) pairs, out-of-scope inputs
# Output: edit success, drawdown

def evaluate_edit(edited, base, in_scope, out_of_scope):
    # Share of in-scope inputs that get the intended answer  [arxiv:2206.06520 eq. 4]
    es = mean(edited(x) == y for x, y in in_scope)
    # Share of out-of-scope inputs whose output moved from the base model  [eq. 5]
    # MEND measures drawdown as an accuracy or perplexity change instead  [arxiv:2110.11309 §4]
    dd = mean(edited(x) != base(x) for x in out_of_scope)
    return es, dd

# Measure whether implied facts follow the edit  [arxiv:2307.12976 §3.1]
# Simplified: the paper also drops tests that the unedited model fails  [§5.1]
def evaluate_ripples(edited, implied):
    # implied: (question, expected answer) pairs for facts the edit should change
    return mean(edited(q) == a for q, a in implied)
```

## Contradictions

- **Drawdown means different things.** MEND reports accuracy or perplexity
  change on unrelated data [arxiv:2110.11309 §4]. SERAC reports the share of
  out-of-scope outputs that change, and uses KL for ConvSent
  [arxiv:2206.06520 eq. 5, eq. 7]. The two DD columns are not comparable.
- **Cross-paper numbers disagree.** MEND reports ES 0.88 and DD 0.001 on
  zsRE with T5-XL [arxiv:2110.11309 §4.1]. SERAC reports MEND at ES 0.823 and
  DD 0.187 on its QA setting with T5-large [arxiv:2206.06520 tab. 3]. The base
  models, benchmarks and metrics differ. Both sets are author-reported. None
  is resolved here.
- **The large-model claim.** MEND claims it is the only approach that edits
  models above 10B [arxiv:2110.11309 §1, §5]. Its own T5-XXL table shows fine-tuning at
  0.87 ES with DD below 0.001, against MEND's 0.89 [arxiv:2110.11309 §4.1].
- **Batched edits in MEND.** The text reports 96% ES at 25 edits, and the
  table reports 0.89 [arxiv:2110.11309 §4.3].
- **ICE against ROME depends on the subset.** RippleEdits reports ICE as best on
  average [arxiv:2307.12976 §5.2]. On the Recent subset, ROME on GPT-NeoX
  averages 71.4 and ICE 69.9 [arxiv:2307.12976 tab. 3]. ICE leads on Random
  (81.0 against 58.3) and Popular (74.1 against 59.7) [arxiv:2307.12976 tab. 4]
  [arxiv:2307.12976 tab. 5]. The average hides this.
- **Metrics differ across the editing and unlearning papers.** Unlearning is
  judged by forget-set accuracy and a membership attack, and it has no ES or DD
  [arxiv:2310.20150 §4.2] [arxiv:2310.20150 §4.6]. Its numbers cannot be read
  against the ES and DD figures above.
- **Which layer extracts the attribute.** Geva et al. say upper attention
  heads do it in §1 and §7, and that MLP sublayers do it "eminently" in §9
  [arxiv:2304.14767 §1] [arxiv:2304.14767 §7] [arxiv:2304.14767 §9]. The card
  follows §7.
- **SERAC's own tables.** Table 3 and Table 5 do not match row by row for QA-hard
  and FC, and the text's 0.99 for QA does not match the table's 0.986
  [arxiv:2206.06520 tab. 3] [arxiv:2206.06520 tab. 5] [arxiv:2206.06520 §5.2].

## Pitfalls

- **MEND's update sign.** The paper writes the edit as a subtraction
  [arxiv:2110.11309 §3.1]. The released code's update, as the card reads it,
  moves against the descent direction unless the learned step sizes turn
  negative. Check the sign on a small case before using it. See the
  [MEND card](/papers/2110.11309) for the code citations.
- **Generality is tested only lightly.** The MEND paper uses backtranslated
  paraphrases and a prefix heuristic. It does not test whether an edit implies
  other facts [arxiv:2110.11309 §5]. RippleEdits tests implied facts, but only
  for relations it encodes, and only within two hops of the edited entity
  [arxiv:2307.12976 §3.1].
- **Overgeneralisation happens.** MEND's edits sometimes change outputs for
  related but different inputs [arxiv:2110.11309 §5, app. F].
- **SERAC's routing threshold is fixed at 0.5.** The paper does not calibrate it
  per task [arxiv:2206.06520 eq. 1].
- **SERAC's memory is unbounded in continual editing.** The authors propose
  periodic distillation as future work [arxiv:2206.06520 §7].
- **Released code and paper differ for RippleEdits.** The generation budget is
  stated as 20 tokens in the paper [arxiv:2307.12976 §5.1], but the released
  code's limit is not that. The preservation check requires every existing
  object in the released code, while the paper says one match suffices. The
  ICE prefix also differs. Check the [RippleEdits card](/papers/2307.12976)
  before reproducing its numbers.
- **Unlearning code is not in the official repository.** At the pinned commit
  the repository has no unlearning code, so the method must be implemented from
  the equations [arxiv:2310.20150 §3.1]. See the
  [unlearning card](/papers/2310.20150).
- **Geva et al. analyse correct predictions only.** Their rates are
  conditional on the model already answering correctly [arxiv:2304.14767 §3].
  Extraction counts a query only when the top-1 token exactly matches the answer
  [arxiv:2304.14767 §7.1].
- **Attribute rate is a proxy.** It counts top tokens that appear in
  Wikipedia-derived candidate sets, so frequent words from those paragraphs also
  count [arxiv:2304.14767 app. C.1].

## Papers

- [Fast Model Editing at Scale](/papers/2110.11309): MEND, a learned gradient editor that makes one-step weight edits, with fine-tuning and KL baselines.
- [Memory-Based Model Editing at Scale](/papers/2206.06520): SERAC, a memory-based editor with a scope classifier and a counterfactual model, leaving the base weights unchanged.
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](/papers/2304.14767): an analysis of where factual recall happens in GPT-2 XL and GPT-J. It does not run an editor.
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](/papers/2307.12976): RippleEdits, six criteria for implied facts, comparing ROME, MEMIT, MEND and in-context editing.
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](/papers/2310.20150): adapter-based unlearning on a frozen T5 model for deletion requests. A neighbouring goal, not a single-answer editor.
- [Agent memory survey (Zhang et al.)](/papers/2404.13501): a survey that places editing among parametric agent memories. It runs no experiments.