wikillm

Paper · 2026-10-18 · cs.AI

Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing

NCE teaches an LLM agent to edit its own context with special-token edits (drop, rewrite, append) that a task-agnostic runtime applies between turns. Training labels come from an offline release analysis and an online teacher. Author-reported: +5.6 average success points over harness compaction on LongCat-Flash-Lite-Sparse at 256K, with 36% fewer tokens.

LLM agentAgent memoryTool useKnowledge distillationagentscontext-managementdistillation

arXivPDFCodeMarkdown17 min read

AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

A coding agent keeps a long running log of everything it has done: file contents, command output, test failures and its own notes. The log grows until it no longer fits, and the usual fix is to let the tool around the model decide when to squash it into a summary. That decision is made from the size of the log alone, so it can throw away something the agent still needs.

This paper lets the model make those cleanup decisions itself. While it writes, it can also write a few special marker tokens that say "drop this old block", "replace that block with this shorter text" or "add this note". The surrounding program carries out the edits between turns and the model continues from the tidied log. Because the markers are part of the model's own vocabulary, the same trained model can work in different agent tools without a tool-specific editing interface.

Training needed examples of good cleanups, and the recorded agent logs contain none. The authors create them with two helper passes. One reads the whole log afterwards and decides which pieces are safe to tidy. The other stands in for the agent at the moment of a cleanup and writes the edit using only what it could see then. The authors report that the trained models solve more tasks while using fewer tokens than the built-in summarising of the harness.

TL;DR

  • NCE gives the model a small grammar of 70 special tokens: two-token block addresses, three operators (drop, rewrite, append), two envelope delimiters and a lock marker §3.1 tab. 3.
  • Edits run at the end of a turn and are never kept in the context. Training data is built from two teachers and by replaying each edit before the next segment is cut §3.2 §3.3.
  • Author-reported on four long-horizon benchmarks at a 256K budget: average success 53.9 vs 48.3 for harness compaction on LongCat-Flash-Lite-Sparse (+5.6), with 36% fewer tokens tab. 1.
  • The official repository ships the grammar, runtime and annotation pipeline only. It has rule-based teachers in place of the paper's LLM teachers and no training code [README] rules.py.
  • Relevant if you need a learned, harness-independent way to shrink an agent's working context. Budget the extra annotation work and read the Pitfalls before reproducing.

Problem

Long-horizon agents accumulate code, tool outputs, feedback and decisions until they exceed a fixed window §1. The two prevailing answers both live outside the model. Rule-based harnesses such as Claude Code and Codex summarise once a fixed token threshold is crossed, which fires on size alone and cannot tell what the task still needs §1. Tool-based methods expose editing as harness-declared calls, which ties the learned policy to one harness's interface §1 §2.

The paper's assumption is that a context edit can be an atomic capability of the model when its meaning is carried by the model's own vocabulary, in the way an end-of-sequence token is understood without a prompt §1. It further assumes the agent sees the context in a rendered form with stable addresses, so that edits can name blocks §3.1.

Method

The method has three parts: the grammar and its runtime semantics §3.1, the construction of supervision from recorded trajectories §3.2, and distillation on contexts that evolve as edits are applied §3.3. Inference reuses the same loop §3.3.

Grammar. At turn t the state is S_t = (B_t, A_t), the block history and the action history. A block is a pair of a persistent address and its text. The model reads Render(S_t), a listing of blocks and actions in the order they happened, each block tagged with its address, and emits zero or more edits. The state then evolves as S_{t+1} = Execute(S_t, e_t) ⊕ (a_t, b_t) §3.1 eq. 1. With no edits this is ordinary context growth §3.1.

Each edit is one envelope around one operator §3.1:

  • drop removes one or more blocks and retires their addresses for good;
  • rewrite replaces the text of one block and keeps its address;
  • append adds an editable note block under a fresh address.

A turn may carry several edits, each preceded by a one-sentence plain-text rationale §3.1. The edit instructions are not kept in the context, and the KV cache of the next prompt is computed from the rendered text alone §3.1.

Release analysis (offline teacher). Block b is in use up to the final turn whose action reads it, r_b = max{t : a_t uses b} §3.2 eq. 2. Recorded trajectories contain no edits, so the offline teacher reads the whole trajectory. It treats every block as kept unless it names the event that ended the block's use, which fixes the turn k_b §3.2. A literal check then looks for the block's rare literals (paths, long integers, hashes, precise decimals, line-number tokens, underscore identifiers) in later actions. If one reappears at turn lambda_b, the release point is postponed §3.2 app. B.1:

r_hat_b = max(k_b, lambda_b) §3.2 eq. 3

A second pass revisits kept blocks, including those from abandoned branches. A block leaves the kept set only after a cited event and the literal check both pass app. B.1.

Online teacher. At a decision point the online teacher stands in for the agent at turn t. It receives the trigger, the surviving blocks and an editable/off-limits marker per block (editable when t > r_hat_b). It does not see release points, token counts or later turns. It rewrites by default, drops only blocks without information, and writes one consolidated note when several blocks reach the same conclusion §3.2 app. B.1. Decision points come from the budget: soft at 70% of a budget W, hard at 95%, or a forward-pass milestone app. B.1.

Distillation on evolving contexts. Training trajectories are rebuilt by replaying the recorded actions and observations forward through S_{t+1}, inserting the online teacher's edits where they occur. A segment is cut at every turn whose edits execute. The following segment starts from the rendered state after those edits §3.3. The loss is next-token on model outputs (actions, rationales and edits) and zero on rendered prefixes, observations and reminders §3.3 app. C.

Inference. The model writes its own edits. The runtime extracts the envelopes, executes them in order, strips them from the output, and renders the next prompt from the new state if any edit ran. If no edit ran, the next prompt keeps the old prefix, so the cached computation is reused §3.3 app. A.3.

Pseudocode
# One turn of NCE: S_{t+1} = Execute(S_t, e_t) (+) (a_t, b_t)  [§3.1] [eq. 1]
def run_turn(state, model, environment):
    # Render blocks with their addresses, in interaction order  [app. A.2]
    prompt = render(state)
    # The model writes an action and zero or more edit envelopes  [§3.1]
    output = model(prompt)
    # Apply the envelopes in the order they were written  [app. A.3]
    for edit in parse_envelopes(output):
        execute(state, edit)
    # Keep the rationale and action, without the envelopes  [app. A.3]
    state.add_action(strip_envelopes(output))
    # The observation enters as an ordinary block  [§3.1]
    state.add_block(environment(output))
Pseudocode
# Execute one edit; a failed check leaves the state unchanged  [app. A.3]
def execute(state, edit):
    # Drop: every named address must be live and unlocked  [app. A.3]
    if edit.op == "drop":
        if not all(is_editable(state, a) for a in edit.addresses):
            return
        # Dropped addresses are retired and never reused  [§3.1]
        state.remove(edit.addresses)
    # Rewrite: a block that is gone keeps its text as a new note  [app. A.3]
    elif edit.op == "rewrite":
        if not is_live(state, edit.address):
            state.append_note(edit.text)
        elif is_editable(state, edit.address):
            # The rewritten block keeps its address  [§3.1]
            state.replace_text(edit.address, edit.text)
    # Append: a note block under a fresh address  [§3.1]
    elif edit.op == "append":
        state.append_note(edit.text)
Pseudocode
# Offline release point: r_hat = max(k_b, lambda_b)  [§3.2] [eq. 3]
def release_point(block, cited_turn, actions):
    # k_b: turn of the cited event; T if the teacher cites none  [§3.2]
    k = cited_turn
    # lambda_b: last action that repeats a rare literal of the block  [app. B.1]
    lam = last_reappearance(rare_literals(block), actions, after=k)
    # A reappearing literal postpones release, it never advances it  [§3.2]
    if lam is not None:
        return max(k, lam)
    return k
Pseudocode
# Trigger check at turn t; W is the trajectory's budget tier  [app. B.1]
def trigger_at(t, tokens, prev_tokens, W, last_fired):
    # Hard fires at 0.95 W, soft the first time 0.70 W is reached  [app. B.1]
    if tokens >= 0.95 * W and cooled_down(last_fired, "hard", t):
        return "hard"
    if crosses(prev_tokens, tokens, 0.70 * W):
        if cooled_down(last_fired, "soft", t):
            return "soft"
    # Milestones, when due, need the last edit two or more turns back  [app. B.1]
    return None
Pseudocode
# Gate and filter one online-teacher decision  [app. B.1]
def decide(view, trigger, tokens, teacher):
    # Soft and milestone points: skip if editable blocks hold under 8%  [app. B.1]
    editable = editable_tokens(view)
    if trigger != "hard" and editable < 0.08 * tokens:
        return "not_called"
    # Hard point: skip only if nothing is editable  [app. B.1]
    if trigger == "hard" and editable == 0:
        return "not_called"
    # The teacher sees only what the agent could see at turn t  [app. B.1]
    edits = teacher(view)
    # Soft and milestone edits that free under 8% are discarded  [app. B.1]
    if trigger != "hard" and freed_share(edits) < 0.08:
        return "discarded"
    return edits
Pseudocode
# Cut training segments on the evolving context  [§3.3]
def build_segments(trajectory):
    state = initial_state(trajectory)
    segments = []
    current = None
    for turn in trajectory.turns:
        # A segment opens with the rendering of the current state  [§3.3]
        if current is None:
            current = new_segment(render(state))
        # Output (action, rationale, edits) is trained; the prefix is not  [§3.3]
        current.add(turn.output, trained=True)
        step = run_output(state, turn.output)
        # Observations only condition later turns  [§3.3]
        for obs in turn.observations:
            state.add_block(obs)
        # A turn whose edits execute closes its segment  [§3.3]
        if step.executed:
            segments.append(current)
            current = None
    if current is not None:
        segments.append(current)
    return segments

Hyperparameters

Values are as stated in the paper. "Not stated" means the paper does not give the value, and the official repository does not contain training code to supply it [README].

NameValueWhereNotes
Dedicated tokens70 in total: 64 address tokens plus 6 control tokenstab. 3 app. A.1the control tokens are the operators, the envelope delimiters and the lock marker §3.1
Address capacity1,024 (h = floor(n/32), l = n mod 32)app. A.1addresses are never reused in a session
Evaluation budget W256K for all four benchmarks; Terminal-Bench also at 128K, 64K, 48K§4.1 tab. 2
Harness compaction thresholdequal to Wapp. CTerminus-2 also compacts NCE runs, because it counts its own copy
Soft reminderat 70% of Wapp. A.4 app. Ctext contains no numbers
Hard reminderat 95% of Wapp. A.4 app. C
Reminder pacingwait 4 turns after a reminder; 8 after a declined soft one; no more soft reminders after 2 declinesapp. A.4
Base modelsLongCat-Flash-Lite-Sparse (69B-A3B); Qwen3.6-35B-A3B; both mixture-of-experts with 3B active§4.1starts from instruction-tuned checkpoints
Training datanearly 180K recorded trajectories on software engineering and terminal tasks, annotated and split into segments§4.1 app. Cdownsampled from data LongCat was already trained on
TeachersGLM 5.3 as both offline and online teacher§4.1 app. B.2teacher sampling settings not stated
Epochs3 for both models§4.1 app. C
Qwen optimiserAdamW, β1 0.9, β2 0.95, weight decay 5e-5app. C
Qwen learning ratepeak 1e-5; 3% linear warmup; cosine decay to 1e-6app. C
Qwen packing and batchsequences packed to 98,304 tokens; global batch of 128 packed sequencesapp. C
Qwen clipping and precisiongradient norm clip 1.0; bfloat16app. C
Qwen steps2,628 over three epochsapp. C
LongCat trainingMegatron on a large GPU cluster; optimiser, learning rate, batch and clipping not statedapp. C
Lossnext-token on assistant tokens only, both modelsapp. C §3.3
Decodingtemperature 1.0; top-p 0.95; top-k 50; at most 32,768 output tokens per call; thinking offapp. Call methods
HarnessesClaude Code 2.1.7 (SWE-bench variants); Terminus-2 on Harbor (Terminal-Bench 2.0)app. Cneither harness changed
Metrica run that stops without a result is a miss; tokens are a 10% trimmed mean§4.1 app. D.1
Tool-call baselineMemAct-style; three tools mirror the operators; LongCat only; same data and recipe§4.1 app. C
Literal (annotation)at least 4 characters and at most 8 occurrences in the trajectoryapp. B.1
Second-pass literal taila literal must stop reappearing at least 3 turns before the endapp. B.1
Budget tiers (annotation)W in {32K, 48K, 64K, 128K}, the largest tier where the peak crosses 0.70 W and the floor fitsapp. B.1
Trigger cool-down2 turns per trigger kindapp. B.1milestone also needs the last edit 2 or more turns back
Milestonesat most 2 per trajectoryapp. B.1
Teacher gatingskip at soft or milestone if editable blocks are under 8% of context; discard edits freeing under 8%app. B.1hard point discards nothing
Final auditdiscard an edit freeing under 3% of context; discard a trajectory with more than 6 edit turns and more than 15% of turns with edits; discard if a hard reminder ends without an editapp. B.1about 4% of annotated trajectories discarded
Re-prefill rate0.105 s per 1K re-prefilled tokens on Claude Code (about 9.5K tokens/s); applied to both harnessesapp. D.4estimated from traces; no time to first token recorded

Results

Author-reported. Setup: four long-horizon benchmarks at a 256K budget, 89 Terminal-Bench 2.0 tasks, 500 SWE-bench Verified issues, 300 SWE-bench Multilingual issues and 731 public SWE-bench Pro issues §4.1. Compared with harness compaction (Comp.) of the same base model and with tool-call editing (Tools), which is only run on LongCat §4.1.

Table 1 (main results, 256K) tab. 1

BenchmarkComp. successTools successNCE successComp. tokens (M)Tools tokens (M)NCE tokens (M)
LongCat: Terminal-Bench 2.032.632.638.26.626.384.60
LongCat: SWE-bench Verified65.262.470.04.073.402.70
LongCat: SWE-bench Multilingual56.054.362.310.8310.936.12
LongCat: SWE-bench Pro39.439.744.99.508.466.42
LongCat: average48.347.353.97.767.294.96
Qwen3.6: Terminal-Bench 2.038.2not run43.82.98not run3.04
Qwen3.6: SWE-bench Verified69.8not run74.02.92not run2.19
Qwen3.6: SWE-bench Multilingual58.0not run64.35.12not run4.13
Qwen3.6: SWE-bench Pro43.6not run45.35.82not run4.92
Qwen3.6: average52.4not run56.94.21not run3.57

Gains over harness compaction, from the table: LongCat success +5.6 average (+4.8 to +6.3 per benchmark over the best baseline), tokens −36% average (−31% to −44% per benchmark). Qwen3.6 success +4.5 average, tokens −15% average tab. 1 §4.2.

Table 2 (Terminal-Bench 2.0, LongCat, by budget) tab. 2

BudgetComp. successTools successNCE successComp. tokens (M)Tools tokens (M)NCE tokens (M)
256K32.632.638.26.626.384.60
128K25.830.334.86.084.963.70
64K27.029.232.63.423.582.52
48K20.227.030.33.403.172.15

At 64K, NCE matches both 256K baselines with under 40% of their tokens §4.2 tab. 2.

Table 5 (context managers, Terminal-Bench 2.0, LongCat, 128K) tab. 5 (Appendix D.1)

MethodSolved (of 89)Success (%)Tokens (M)Mean context (K)Peak context (K)ReductionsTurns
Harness compaction2325.86.0838.379.70.6119
Sliding window2325.84.6243.671.51.792
Fixed memory state2629.25.5944.072.91.995
Prompt compression2932.64.6838.366.014.095
NCE3134.83.7030.561.13.199

Table 6 (supervision defects on 40 trajectories) tab. 6 (Appendix D.2)

Trajectories (%)One passOriginal historyNCE
Uses future information30.012.55.0
Deletes evidence used later72.540.037.5
Edits a removed block0.095.00.0
Restates removed text7.585.02.5
Any defect80.0100.037.5
Defective edits (%)4.522.51.1
Edits3,2134,2292,484

Other reported results, all author-reported:

  • Tool-call editing, trained on identical edits, scores below harness compaction on average and cuts tokens by only 6%, against 36% for NCE §4.2.
  • At 256K on LongCat, 99.1% of the 73,148 edits the model wrote ran unchanged. Most of the rest target blocks an earlier edit had already removed app. D.3.
  • Re-prefill after an edit adds 0.2 to 0.4% to a task's wall-clock time without overlap. The authors estimate that the re-prefill fits in idle time for 98% of editing turns with Terminus-2 and 81% with Claude Code §4.3 tab. 8 app. D.4.
  • Under Claude Code at 256K, the harness compacts in 3 of 1,531 NCE SWE-bench tasks (0.2%) and in 14 of 1,531 tool-call runs (0.9%) app. D.3.

Paper vs code

The paper links the repository github.com/dengyl20/native-context-editing as its code link, so it is treated as the authors' repository (MIT licence, copyright "The NCE Authors", commit 86804e16bd98a64c26fa57e986aac2a3e2afa7bf). The repository contains a reference implementation of the grammar, runtime and annotation pipeline, with examples and 299 unit tests [README].

  • Scope. The repository is a reference implementation of the grammar, runtime, annotation pipeline and examples. It trains nothing, calls no model and makes no network calls [README]. It has no LongCat or Qwen training code, no Megatron or ms-swift configuration, no benchmark harness and no Tools baseline. The training hyperparameters in Hyperparameters and all Results are therefore unverified by code.
  • Teachers. Paper: two GLM 5.3 teachers with the prompts in app. B.2 §4.1. Code: deterministic rule-based offline and online teachers, with the LLM teacher as an interface that the repository does not implement rules.py rules.py release.py online.py.
  • Triggers and thresholds match. Soft 0.70, hard 0.95, cool-down 2, milestone gap 2, 8% gates, budget tiers 32K/48K/64K/128K online.py online.py matches app. B.1. Audit thresholds (3%, more than 6 edit turns and more than 15%) match audit.py audit.py.
  • Reminder policy matches. Texts, soft and hard levels, 4-turn wait, 8-turn wait after a declined soft reminder and the two-decline stop match runtime.py app. A.4.
  • Literal check matches, with a gap in path detection. Minimum length 4, maximum 8 occurrences and the number kinds match literals.py literals.py. But the path pattern requires at least one / separator literals.py, so a bare file name such as setup.py is not a path literal in the code, while the paper's wording ("paths of at least four characters") does not require one app. B.1.
  • Release point has an extra clamp. The code also clamps k_b to the block's entry turn release.py. The paper does not state this. The file-read and blueprint clamps are in the code release.py and in app. B.1, but not in Eq. (3) §3.2.
  • "Distinctive lines" is undefined in the paper and all non-blank lines in the code. The supersession check treats every non-blank line of the candidate as distinctive release.py.
  • Drops are all-or-nothing. The code rejects a whole drop if any named address is unknown, retired, not live or locked runtime.py. The paper's Execute description does not say so §3.1.
  • Dead-block rewrite. The code turns a rewrite of a block no longer live into an append and records not_live runtime.py. This matches app. A.3; Method §3.1 is silent.
  • Address exhaustion. The pool raises AddressExhausted after 1,024 addresses and appends are rejected with address_exhausted state.py runtime.py. The paper says addresses are never reused and gives no behaviour at exhaustion.
  • Distillation loss matches. Model outputs are trained, rendered prefixes, observations and reminders are not distill.py. A segment closes only when edits execute distill.py.
  • Evaluation literal definition is not in the repository. app. D.2 uses a different literal set: at least 5 characters, a frequency filter against 1,000 reference trajectories, inner-capital identifiers and IPv4 addresses. The repository contains only the annotation version literals.py.

Pitfalls

  • The published supervision cannot be regenerated from the repository. The paper's labels come from GLM 5.3 teachers with prompts shown in abridged form app. B.2. The repository's teachers are rule-based and will produce different release points and edits. Expect different data, not a drop-in replication.
  • Literal reappearance counts only actions. The literal check scans later actions, including tool-call arguments, not later observations literals.py. A literal that reappears only in a tool result does not postpone release. The paper's text says "later actions" and is consistent with this, but it is easy to get wrong when reimplementing.
  • Budget mismatch. Annotation assigns each trajectory a budget tier from 32K to 128K app. B.1. Evaluation uses 256K for the headline results and sets the harness threshold to W app. C. The paper does not say which W drives the reminders in the 256K runs.
  • Aggregates that do not match the table columns. The text says editing turns are "1.7% of all turns" and "8.1% of output tokens", "45 operations per task", "71% of operations are drops" and "12.7K tokens per editing turn" in both harnesses app. D.3. The per-benchmark columns of tab. 7 give 1.5–2.9%, 3.8–9.0%, 32–56 operations, drop shares of 59–75%, and tokens removed per editing turn of 7.9K–16.7K. The 71% is the Claude Code drop share and 12.7K is the Terminal-Bench value. The aggregation is not stated. Use per-benchmark values.
  • Token savings are not uniform. Qwen3.6 Terminal-Bench uses 2% more tokens with NCE (2.98M to 3.04M) tab. 1. The 15% headline is an average.
  • Tool-call comparison is LongCat only. The Tools baseline is run only on LongCat, so the "native beats tools" claim has no Qwen counterpart §4.2.
  • Re-prefill timing is an estimate. The 98% and 81% "hidden" figures depend on a regressed prefill rate of about 9.5K tokens/s, applied to both harnesses, and on decode-time estimates from output tokens app. D.4. The measured cost assumes no overlap and is an upper bound app. D.4.
  • Edit granularity differs between annotation and inference. Annotation edits are all or nothing across editable blocks app. B.1. At inference each envelope is atomic runtime.py, so a teacher-style "dispose of everything" behaviour is not enforced by the runtime.
  • A rewrite of a dead block creates a new block. It consumes a fresh address and adds to the context app. A.3. The paper reports that 0.9% of written edits fail, mostly because they address a block an earlier edit already removed app. D.3, but does not say how many of those are kept this way. Track address use in long runs.
  • Annotation data size. §4.1 says nearly 180K trajectories were annotated. app. B.4 describes one batch of about 50K. The relation between the two is not stated.
  • Only supervised fine-tuning. Limitations state that RL on task outcomes could refine when and how the agent edits, and that only software-engineering and terminal tasks were tested § Limitations.
  • Figure and table numbers were inferred from source order. The compiled PDF was not checked. Numbers are as they would be if floats numbered in source order.

Lineage

  • Contrasts with: harness compaction and rule-based summarising of Claude Code and Codex §1 (no card); tool-based context editing, Sculptor and Context-Folding §1 (no card).
  • Builds on the tool-call editing of MemAct, re-implemented here as the Tools baseline §4.1 (no card).
  • Compared against in ablations: a sliding window, a fixed-size memory block and compression of older tool results app. D.1.
  • Superseded by: not stated.

Source and license

Adapted from “Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing” by Yunlong Deng, Yan Li, Xing Hu, Yida Zhu, Sean Du, Weiyang Liu, Guangyi Chen (arXiv:2610.25550), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/dengyl20/native-context-editing at commit 86804e16bd98 (MIT). Any quoted code is shown under that license.