Evaluating the Ripple Effects of Knowledge Editing in Language Models

In plain words

Fixing one fact in a language model can leave the model inconsistent. Tell it that Jack Depp is the son of Johnny Depp, and it should also come to believe that Jack Depp is the sibling of Lily-Rose Depp. The usual tests for editing methods check the edited fact, its rewordings, and some unrelated facts. They do not check the facts that should follow from the edit.

The paper defines six kinds of these follow-on checks. They cover logical consequences (siblings are symmetric), two-step questions that pass through the new fact, alias names of the same person, and facts that must stay put. It turns them into a benchmark of edits, with questions for each, built from Wikidata. Editing methods that change the model's weights pass the direct check but often miss the follow-on facts. The simplest method, which puts the new fact into the prompt, scores best on this benchmark.

For an engineer, the takeaway is that an edit can look correct on its own test and still leave the model inconsistent elsewhere.

TL;DR

• A factual edit is treated as a change to a knowledge graph; its "ripple effect" is the set of other facts the model should then also change or keep §3.
• Six criteria test this: logical generalization, two compositionality tests, subject aliasing, preservation, and relation specificity §3.1.
• RippleEdits is built from three Wikidata-derived subsets (Recent, Random, Popular) with templated natural-language queries §4.1 tab. 1.
• Author-reported: ROME, MEMIT and MEND get consistent scores on aliasing but low scores on logical generalization and composition; an in-context editing (ICE) prompt scores best on average §5.2 tab. 3 tab. 4 tab. 5.
• The official code is MIT-licensed and covers benchmark data, generation scripts and evaluation; several numbers in the paper do not match the released code (see Paper vs code).

Problem

A factual edit replaces an object for a subject and relation, (e, r, o) → (e, r, o*) §2. Standard evaluation checks that the model generates o* for paraphrases of the query, and that unrelated facts are unchanged §1. The authors argue this misses implied changes: if z is the mother of e, the children of z are e's siblings, so an edit to the mother should also change the sibling answers §1. Their running example is injecting "Jack Depp is the son of Johnny Depp", which should also make "Jack Depp is the sibling of Lily-Rose Depp" true [abstract].

The paper assumes the ripple effects that matter are the modified facts within a 2-hop distance of the edited subject or object §3.1. It states two limits. The benchmark relies on Wikidata, which may be incomplete or outdated. It does not test the edit's own paraphrases or subject specificity, and it does not check distant facts §6, Limitations. Editing popular entities is expected to produce more ripple effects (higher severity) §3.

Method

Evaluation criteria §3.1 fig. 2

For an edit δ: (e, r, o) → (e, r, o*), the criteria are:

1. Logical generalization (LG). Relations obey constraints. Sibling is symmetric, and Location is transitive. Check the implied facts about e, o and o* after the edit §3.1.
2. Compositionality I (CI). For a fact (o, r′, z) about the old object, and (o*, r′, z*) about the new object, expect the composed relation r∘r′ of e to change from z to z* §3.1.
3. Compositionality II (CII). For a fact (e′, r′, e) with e as object, expect the composed relation r′∘r of e′ to change from o to o* §3.1.
4. Subject aliasing (SA). For each alias e′ of e, expect (e′, r, o) → (e′, r, o*) §3.1.
5. Preservation (PV). If r is one-to-many, the other existing objects (e, r, o′) with o′ ≠ o* must remain §3.1.
6. Relation specificity (RS). Facts (e, r″, ·) for relations not depending on r must not change §3.1.

Relations are split into a set of N_rel = 54 basic relations, each with a manual phrase, a set D_r of dependent relations, and logical rules §4.1 §4.2. For the mother example, D_r includes sibling, sister, brother, aunt and uncle §4.1. The code encodes the same dependency lists in its relation table relation.py.

Benchmark construction §4.1 fig. 3

The pipeline has four steps.

1. Collect triplets from Wikidata. Three types:
   • Recent: triplets modified in a window of 250 days after July 2022, used as injection edits §4.1.
   • Random: 10 buckets of entities by number of triplets; N_ent entities are sampled per bucket, then one triplet per entity. These are modification edits §4.1.
   • Popular: subjects among the top-1000 most viewed Wikipedia pages in at least one month of 2020–2022 §4.1.
2. Generate edits. Recent: the target is the triplet itself. Random and Popular: the target object o′ ≠ o is sampled from N_cand candidate objects of the same relation §4.1 §4.2.
3. Generate tests. For each criterion, collect test triplets from Wikidata neighbourhoods. The sets are S(e) (triplets with e as subject) and T(e) (triplets with e as object) §4.1.
4. Phrase in natural language. One template per relation, for example "The date of birth of is". The PV query for a multi-valued relation is phrased to exclude the new object, for example "The award received by which is not <o*> is" §4.1.

Statistics are in tab. 1. Each edit has 18.8 to 26.8 queries on average tab. 1 §4.2. A manual check of 200 random queries found 100% sound and 98.5% grammatical §4.3.

Editing methods §5.1

• ROME: a rank-one update to MLP weights of a Transformer layer §5.1.
• MEMIT: an extension of ROME for editing many facts at once §5.1. In the released code each evaluated edit is applied as a single-request batch, then the weights are restored modeleditor.py modeleditor.py.
• MEND: a network that transforms gradients to produce local edits §5.1.
• ICE (in-context editing): no weight change. The prompt is prefixed with the edit, "Imagine that <o*> would have been <P_r>", where P_r is a manual proposition for r, such as "The mother of " §5.1 fig. 5. The released code uses a different prefix (see Paper vs code).

Evaluation protocol §5.1

An edit is kept only if (a) the model generates the original objects for the test queries before the edit, and (b) it generates o* for (e, r) after the edit. Each test case is scored on its queries. Per example, the score is the mean accuracy over its executed test queries; the result is then averaged over examples §5.1 evaluation.py.

A generation counts as correct if one alias of the target appears in the text. Generation is capped at 20 tokens §5.1.

# Score one editing method on one RippleEdits subset  [§5.1]
# Scoring follows the released code  [code: src/testrunner.py#L26-L66]
def score_example(model, editor, example):
    # Keep only test cases whose condition queries the unedited model answers  [§5.1]
    executable = []
    for test_case in example.test_cases:
        if all(answers_correctly(model, q) for q in test_case.conditions):
            executable.append(test_case)
    # Apply the edit; if it does not take, the example is discarded  [§5.1]
    editor.edit(model, example.fact)
    if not answers_correctly(model, example.fact.query):
        editor.restore(model)
        return None
    # Score each executed test case on the edited model  [§5.1]
    passed = []
    for test_case in executable:
        passed.append(all_queries_pass(model, test_case))
    editor.restore(model)
    if not passed:
        return None
    return sum(passed) / len(passed)  # mean per example  # [0, 1]

The per-axis average then runs over examples where the edit succeeded and at least one test was executed evaluation.py.

In-context editing prompt §5.1 fig. 5

# In-context editing (ICE): prepend the edit to the query  [§5.1]
# Paper template: "Imagine that <o*> would have been <P_r>"  [§5.1]
# Released code: "Imagine that " + full edit sentence + newline  [code: src/modeleditor.py#L29-L35]
def ice_prompt(edit_sentence, query_prompt):
    context = "Imagine that " + edit_sentence + "\n"
    return context + query_prompt

The edit sentence is the relation phrase for the subject followed by the new target and a period, for example "The name of the mother of X is Y." fact.py relation.py.

Hyperparameters

Benchmark construction

Editing methods

ROME hyperparameters are the official ones of Meng et al., per model file, except LLaMA which the authors adjusted §5.1. Values from the released files:

Sources: gpt2-xl.json EleutherAI_gpt-j-6B.json llama-7b.json EleutherAI_gpt-neox-20b.json

MEMIT (only GPT-2 XL and GPT-J have files in the release; the paper reports MEMIT on those two only tab. 3 tab. 4 tab. 5):

Sources: gpt2-xl.json EleutherAI_gpt-j-6B.json

MEND (GPT-2 XL and GPT-J): lr_scale 1.0, n_toks 10, counterfact false, zsre false, mini false gpt2-xl.json EleutherAI_gpt-j-6B.json. The paper does not state these values §5.1.

Inference

Results

Author-reported. Setup: RippleEdits subsets Recent, Random and Popular §4.2; edits and queries kept only when the pre-edit and post-edit conditions hold §5.1 tab. 2. Scores are accuracies in percent. Models: GPT-2 (= GPT2-XL, 1.5B), GPT-J, GPT-NeoX, LLaMA-7B, and GPT-3 for ICE §5.1. In the compiled tables, the labels are macros: "GPT-Neo" in the tables is GPT-NeoX-20B in the text.

Table numbers are assigned by source order in the LaTeX, since no compiled PDF was available.

Edits and tests kept after filtering tab. 2:

Averaged over ROME, MEMIT and MEND for the parametric rows tab. 2 caption.

Dataset statistics tab. 1:

Recent subset accuracy tab. 3:

Random subset accuracy tab. 4:

Popular subset accuracy tab. 5:

Parametric methods on GPT-2 XL, by criterion, averaged over the three subsets tab. 6:

Error types on 200 failures tab. 7:

Headline claims, author-reported §5.2:

• Parametric editors are effective on the edited fact but often fail the follow-on criteria; their average accuracy is reported as 38–66 §5.2. The per-subset averages in tab. 3–tab. 5 for ROME, MEMIT and MEND span 38.8–71.4.
• ICE is best on average across subsets. Against ROME it is more than 10 points better on GPT-NeoX and 29 points better on LLaMA §5.2 (subset averages of the tables give +11.9 and +29.2).
• Aliasing (SA) is high for all methods: at least 86.8 in every setting §5.2 tab. 3–tab. 5.
• LG for ROME on GPT-J is 53.8 on Random and 5.5 on Popular tab. 4 tab. 5. Popular entities produce more ripple effects §5.2.
• Editing performance grows with model size for ROME (GPT2-M, GPT2-L, GPT2-XL, GPT-J, LLaMA, GPT-NeoX) §5.2 fig. 6.
• Failures of ROME are mostly noise; for ICE on GPT-NeoX and LLaMA, abstention is the most common failure §5.3 tab. 7.
• GPT-3 abstains on 49% of 100 sampled LG queries, versus 28% for LLaMA under ICE §5.3.

Paper vs code

Code paths are relative to the repository at code_commit. The repository is github.com/edenbiran/RippleEdits (MIT, copyright Eden Biran); the paper's abstract footnote links it §1.

• Generation budget. Paper: at most 20 tokens §5.1. Code: execute_query passes len(prompt) + 30, where len counts characters, as the length for Hugging Face generate, which counts tokens queryexecutor.py queryexecutor.py. The GPT-3 path passes the same value as max_tokens queryexecutor.py. The effective budget is not 20 tokens.
• Preservation matching. Paper: correct if the generation matches at least one of the objects §5.1. Code: _verify_answer requires every target group to match queryexecutor.py. The PV query passes all existing objects as target groups build_benchmark_tests.py. So the code requires all preserved objects to appear.
• ICE template. Paper: "Imagine that <o*> would have been <P_r>" §5.1. Code: "Imagine that " + relation phrase + target + ".", then a newline modeleditor.py fact.py.
• Recent window. Paper: 250 days after July 2022 §4.1. Code: day offsets from the run date, start_in_days_ago=i recently_modified_facts.py, with k = 250 and 4 per day recently_modified_facts.py.
• Subset sizes. Paper: 2,000 Recent, 1,000 Random, 1,000 Popular edits tab. 1; the abstract says 5K §1. The table sums to 4,000. Released files contain 1948 Recent, 1922 Random and 885 Popular examples (counted from "example_type" in data/benchmark/*.json). The code's main also builds Random with 2000 edits and Popular with 1000 but commented out build_benchmark.py.
• Evaluation sample. Paper: edit counts up to 989 per model tab. 2. Code's main samples 200 examples per subset evaluation.py. The paper's reported counts cannot come from one 200-example run; sample size is not stated in the paper.
• Relation count. Paper: N_rel = 54 §4.2. The enum in src/relation.py lists about 53 relations.
• Decoding. Paper: not stated. Code: temperature=0 in both Hugging Face and OpenAI calls queryexecutor.py queryexecutor.py.
• MEMIT. Paper: "editing many facts at once" §5.1. Code: each edit is a batch of one request, then the weights are restored modeleditor.py modeleditor.py.
• MEMIT hyperparameters for LLaMA and GPT-NeoX. Not in the release; the paper's tables omit those rows tab. 3–tab. 5. Code would fail on hparams/MEMIT/{model}.json for those models.
• Sibling logic. Code's LG for Sibling uses the children of the subject's mothers and fathers build_logical_constraints.py, consistent with the paper's rule §3.1.

No other discrepancies were found after checking the criteria, the ROME and MEND hyperparameters, and the answer matching.

Pitfalls

• Matching is substring-based and case-sensitive. Code checks alias in generation for every alias queryexecutor.py. Single-character aliases are dropped unless numeric query.py.
• The match runs on prompt plus completion. Only the ICE context is removed queryexecutor.py; the query text stays. So words in the prompt can count as matches. For example, the PV prompt contains "which is not <o*>".
• Generation post-processing. Leading newlines, spaces, colons, commas and semicolons are stripped utils.py. Other output is not normalised.
• Filtering changes the denominator. Only edits that apply and test queries answered before the edit are scored §5.1. Coverage ranges from 29% to 94% of tests tab. 2, so per-subset averages cover different queries per model.
• ICE does not change weights §5.1. Its scores measure conditioning on the prompt, not a stored edit. Compare the two families with that in mind.
• Restore and working directory. ROME, MEMIT and MEND editors call os.chdir('./memit') and restore original weights afterwards modeleditor.py. Running from a different directory breaks the hyperparameter paths, and a failed restore carries edits into later examples.
• Model names are labels. The paper's macros print GPT-2 for GPT2-XL and GPT-Neo for GPT-NeoX-20B. Use the HF ids from the Hyperparameters section.
• Averages in the text. The text says parametric methods average 38–66 §5.2. The tables show per-subset averages of 38.8–71.4 for these methods.
• Numbering. Table and figure numbers here follow source order and may differ from the published PDF.
• Version. This card uses arXiv v2 (updated 2023-12-20). The licence was recorded by wikillm fetch from the arXiv record.
• Where weight editors fail. MEND has the lowest scores on CI and CII on GPT-2 XL (17.0 and 13.6) tab. 6, and the highest LG (39.1) on the same model. A method chosen by one criterion can be weak on another.

Lineage

• Builds on: MEND Mitchell et al.; arXiv 2110.11309. ROME and MEMIT are cited as Meng et al. (2022, 2023); the paper's bibliography gives no arXiv ids for them, so no link is given §5.1.
• Compared against: ROME, MEMIT and MEND §5.1; in-context editing (ICE), the authors' baseline §5.1.
• Related benchmarks: zsRE and CounterFact §3.2; MQuAKE, a concurrent multi-hop editing benchmark §3.2.
• Superseded by: not stated.