Autogenesis: A Self-Evolving Agent Protocol

In plain words

An AI agent built from a language model usually has several moving parts: the instructions it follows, the code of the tools it calls, the rules that decide which tool to use, a simulated environment it works in, and a memory of past steps. This paper argues that when such an agent improves itself, every one of these parts should change the same way. Each change is proposed, checked against some feedback, and only kept if it passes; the previous version stays on record so it can be brought back.

The paper calls this shared bookkeeping a protocol. It keeps the list of registered parts, their versions and their histories apart from the strategy that suggests a change. The authors also build a multi-agent system on top of it, where a planner hands work to specialist agents. In their experiments the self-changing version does better than the same system with fixed parts on most reported reasoning, tool-use and coding settings, but it also spends more model calls and tokens. These are the authors' own reported numbers, and the paper says its comparisons do not separate the protocol from the update strategy.

TL;DR

• AGP separates a registry of versioned resources (prompts, agents, tools, environments, memory) from the strategy that proposes changes to them §3.1 §3.2.
• Each change goes through candidate proposal, assessment, and commit; a rejected candidate leaves the active resource as it was §3.2 app. G.
• The AGS system with evolution beats its non-evolving configuration in most reported settings on GAIA, GPQA-Diamond, AIME and LeetCode (author-reported) tab. 1 tab. 2 tab. 3.
• Gains come with extra model calls and output tokens, and the paper does not run matched-budget comparisons §5.2 app. B.
• No model parameters are trained. Restoring an old version and recovery after a bad version went live are not demonstrated §3.2 §6.

Problem

An LLM agent adapts by changing things at run time: a prompt, a tool implementation, a decision rule, or what memory keeps after compaction §1. Each kind of change needs an identity for the thing being changed, a way to judge a candidate, a rule for when it becomes active, and a stored history. Communication standards such as MCP and A2A cover how an agent talks to tools and other agents, but not this update lifecycle §1 §2. Existing optimizers (Self-Refine, Reflexion, TextGrad, GEPA, AFlow, EvoAgentX, Alita) each keep their own search state, and none defines a common contract across resource types §2 app. E.1.

Method

Layer 1: Resource Substrate Protocol Layer (RSPL)

RSPL represents five resource types: Prompt (instructions), Agent (decision logic), Tool (callable capabilities, including local code, MCP-connected tools and skills), Environment (executable environment model), and Memory (storage and retrieval policy) §3.1 app. D. Each resource is a tuple of a name unique within its type, a description, an invocation mapping, an evolvability marker, and metadata §3.1 app. D. Each registration record adds a version, an implementation descriptor, instantiation parameters, and the exported schemas or text that consumers use §3.1 app. D.

Each type gets a management triple: the set of active records, a context manager that handles registration, construction, execution, update and restoration, and a server interface that exposes these to consumers §3.1 app. D. The operation vocabulary (init, register, get, list, update, restore and others) is conceptual; each resource type maps it to its own native methods, and optional operations need explicit adapter support app. D.

Four infrastructure services sit under RSPL: a model manager (routes calls by model name or role), a version manager (version identifiers and lineage), a dynamic manager (loads implementations and converts schemas), and a trace manager (links execution observations to update decisions) §3.1 app. D.

Layer 2: Self-Evolution Protocol Layer (SEPL)

SEPL is a loop over candidate states and messages. Its operators map a state and an incoming message to a new state and an outgoing message. Each proposal and assessment step leaves the active resource and its bindings unchanged, and only an accepted commit publishes a new version §3.2 app. G.

Editable targets are fields of registration records (for example an implementation descriptor or a prompt template), selected by the evolvability marker and the manager's permitted fields. Task-local outputs (the current answer or code) can be edited as well, but they are not persistent resources §3.2 app. G. Traces are evidence only and are never edited §3.2.

The reflection instantiation used in the experiments composes five operators: Reflect (turns traces, the active state and prior assessments into hypotheses), Select (picks editable targets and proposes modifications), Improve (builds a provisional candidate), Evaluate (scores the candidate under a task feedback policy), and Commit (records an accepted version or updates the task-local solution) app. G. A candidate is kept only if its assessment is accepted. Otherwise the active state stays the same and the rejection is recorded so that later proposals can use it §3.2 app. G.

Reflection loop

# Reflection-driven resource update  [alg. 1]
# Input:  system, feedback policy, update budget T
# Output: retained state (resources and task-local solution)

def update_resources(system, policy, budget):
    # Expose editable resources and run the task once  [alg. 1]
    state = expose_resource_state(system)
    state, trace = execute(system, state)
    # Keep candidate assessments apart from the execution trace  [app. G]
    assessments = []
    for step in range(budget):
        # Stop before more work if the stop rule holds  [alg. 1]
        if should_stop(state, trace, assessments, policy):
            break
        # Turn the trace and past assessments into hypotheses  [§3.2] [alg. 1]
        hypotheses = reflect(trace, state, assessments)
        # Select only editable targets, respecting markers  [app. G] [alg. 1]
        edits = select(state, hypotheses)
        # Build a provisional candidate; the active state is untouched  [alg. 1]
        candidate = improve(state, edits)
        # Assess the candidate under the feedback policy  [§3.2] [alg. 1]
        verdict = evaluate(candidate, policy)
        # Commit only accepted candidates  [app. G] [alg. 1]
        if accept(verdict):
            state = commit(candidate, verdict)
        assessments.append(verdict)
        # Stop after the last update, or when the rule says so  [alg. 1]
        if should_stop(state, trace, assessments, policy) or step == budget - 1:
            break
        # Re-run the task when the edits need fresh execution  [alg. 1]
        if needs_execution(state, edits, verdict):
            state, trace = continue_task(system, state, trace)
    return state

The budget T counts update iterations, not model calls. Candidate evaluation, proposal and continuation calls all count toward cost and must be accounted for separately app. G.

AGS system

AGS instantiates AGP with a planner and five specialist roles (deep researcher, browser use, deep analyzer, tool calling, coding). The planner breaks a task into subtasks without executing them, dispatches independent subtasks in parallel, and re-plans after collecting results. Specialists read their prompt and tool resources from RSPL and write results and traces to shared memory. An "agent as tool" option exposes a specialist through an RSPL tool schema §4. The paper describes an Agent Bus for dispatch and a plan file; neither was found in the released repository (see Paper vs code) §4.

Evolution in AGS uses the reflection loop above. Edits can be made to prompts, tools and code (Agent-Evo variant) §5.2. Textual-feedback and reward-based editing sketches are described in the appendix as unevaluated adapters and are not used for any reported result app. G.

Experimental setup

Benchmarks: GPQA-Diamond (198 multiple-choice questions), AIME24 and AIME25 (30 integer-answer problems each), no external retrieval §5.1. GAIA (three difficulty levels) and Humanity's Last Exam (HLE; judge is o3-mini) §5.2. 100 recently released LeetCode problems in Python3, C++, Java, Go and Kotlin, scored by an online judge §5.3. Accuracy requires the exact option or integer; reference answers are used only for scoring §5.1 app. B.

Multi-model reasoning: up to three reflection rounds without external tools; the final output is scored after termination §5.1. Single runs are reported in tab. 1. Repeated GPT-4o comparison: three complete runs against Vanilla, Self-Consistency, Best-of-3, Simple Reflection and PS-Joint-Evo under at most three primary generation or refinement opportunities app. B.

Coding: Solution-Evo allows an initial submission plus up to two revisions using judge feedback; Vanilla uses only the initial submission §5.3 app. F.

Resource-update cases

Three illustrative cases, all using google/gemini-3.1-pro-preview and all accepted updates: (1) a GAIA spreadsheet task where a new Tool reads the workbook, a new Agent performs auditable aggregation because the built-in Agent is frozen, and a Prompt update separates verification from answering app. A; (2) a GAIA map task where the generated Environment's transition rules are corrected app. A; (3) a Memory policy that returns constraint records after compaction. On a 45-record workload, returned constraints rise from zero to eight, with 25 recent records preserved. The added storage is a possible confound §5.4 app. A. These cases show resource behavior, not task accuracy gains §5.4.

Hyperparameters

Results

Author-reported. All runs are single runs unless stated otherwise. Setups are described in the Method section.

Multi-model reasoning, one complete evaluation tab. 1

Accuracy with correct/total counts. Evolution variants: Prompt-Evo edits prompts; Solution-Evo refines the task solution; PS-Joint-Evo edits both.

Gain of PS-Joint-Evo over Vanilla, in percent, per model (GPQA / AIME24 / AIME25): gpt-4o 21.05 / 25.00 / 100.00; gpt-4.1 3.88 / 71.43 / 66.67; grok-4.1-fast 7.27 / 0.00 / 7.41; claude-sonnet-4.5 3.87 / 13.04 / 22.73; gemini-3-flash-preview 2.29 / 12.00 / 12.00 tab. 1. For gpt-4.1 on GPQA-Diamond, PS-Joint-Evo (67.68) scores below Solution-Evo (68.69) tab. 1.

GAIA tab. 2

Task-completion accuracy (%) by level and split. Rows marked as external use their own backbones, tools and budgets, and are context only. Alita's validation figure is pass@3 app. B.

Within-AGS gain of Agent-Evo over Vanilla: validation 4.09 / 5.26 / 0.00 / 4.05 (L1 / L2 / L3 / Avg.); test 8.23 / 10.56 / 33.34 / 12.61 tab. 2. Gains are computed from the displayed percentages; external rows are not a controlled comparison app. B.

HLE: AGS scores 59.6 on the full HLE set; the paper's leaderboard comparison is shown as a figure §5.2 fig. 2.

LeetCode, 100 problems, Gemini-3-flash-preview tab. 3

Vanilla and Solution-Evo per language. PR = problems accepted; TLE, WA = counts of time-limit and wrong-answer submissions; AR = mean runtime (ms) over accepted solutions; AM = mean memory (MB); ARB and AMB = mean platform runtime and memory percentiles over accepted solutions. Paper reports these as a single run per setting tab. 3 app. F.

The paper's Gain column (only for the Gemini Vanilla vs Solution-Evo pair) lists PR gains of 10.1 (Python3), 17.9 (C++), 16.7 (Java), 15.9 (Go) and 26.7 (Kotlin) percent tab. 3. Java TLE rises from 0 to 1 and has no defined percentage app. F.

Repeated GPT-4o evaluations (three runs) app. B fig. 3

Mean accuracy ± sample SD (%), pooled correct/total, and approximate output tokens per problem.

Per-run counts: GPQA-Diamond PS-Joint-Evo 115, 118, 118 of 198; AIME24 5, 4, 6 of 30; AIME25 4, 3, 5 of 30. Joint editing uses about 52 to 61 percent more output tokens than the matched controls, as the paper reports app. B. The paper states these runs are not an equal-compute comparison.

Additional reported facts

• Memory case: constraints returned rise from 0 to 8 on a 45-record workload; 25 recent records kept §5.4.

Paper vs code

Repository checked: https://github.com/DVampire/Autogenesis at commit 5b070d5cd497e951a188a01409138e8b530eaf4a, MIT license LICENSE. The paper links this repository on its title page p. 1. Results below are what the repository contains at that commit.

• Planner and Agent Bus not present. The paper describes a planner, an Agent Bus and a plan file for dispatch §4. A search of the repository found no plan.md writer or bus module. The HLE config states that the old bus/planner pipeline is not part of this repository and that the answering pipeline is a general single agent hle.py.
• Step limits differ. The paper states planner m=30 and specialist m=20 §5.2. The repository's general agent uses max_step 50 general_agent.py, and the code and browser agents use 30 code_agent.py browser_agent.py. The value 20 appears only for evaluator configs agent_evaluate_agent.py. No planner config with m=30 was found.
• Backbone matches. Default model is google/gemini-3.1-pro-preview base.py, consistent with §5.2.
• Output ceiling matches. max_tokens = 32768 base.py, consistent with the 32,768 ceiling in app. B.
• GPQA option shuffle matches. Options are shuffled with a seeded random generator, seed 42 GPQA.py GPQA.py, as stated in app. B.
• Evolution loop not located. The variant names (Prompt-Evo, Solution-Evo, PS-Joint-Evo), the Reflect/Select/Improve/Evaluate/Commit operators, and the two-revision coding loop were not found by name in the repository. The general agent config has enable_evolving = False general_agent.py. Optimizer agents exist for agents, tools, skills, memory and environments, but their connection to the paper's reported runs is not verifiable from this commit agent_optimize_agent.py.
• Repeated-run sampling settings not located. The temperature 0.7 and frequency penalty 0.3 from app. B were not found in the directories searched (autogenesis/data, autogenesis/benchmark, configs/base.py, configs/agents/general_agent.py, code_agent.py, browser_agent.py).
• Data release. The paper says the public release excludes LeetCode-derived problem statements, tests, traces and submission tools §6 app. F. The repository does contain autogenesis/data/leetcode.py; whether it includes problem content was not checked, and the paper's exclusion statement governs.

Pitfalls

• Self-assessment is not correctness. Reflection's acceptance uses the same backbone's judgement of whether a candidate looks satisfactory, with reasoning. Reference answers are used only for final scoring. A positive assessment is not a correctness check, and acceptance does not guarantee monotonic improvement §3.2 app. B app. G.
• Rejection and restoration differ. Rejection keeps the active state; restoring a retained version is a separate operation. The paper does not demonstrate recovery after a faulty version goes live §3.2 §6 app. A.
• Cases show accepted updates only. The three resource cases are all accepted, so they do not test rollback app. A.
• Memory case is not a task-accuracy result. The added persistent storage may account for part of the retrieval gain §5.4 app. A.
• Single runs. Table 1 entries are single complete evaluations, and no repeat-run variance is reported for GAIA or LeetCode tab. 1 tab. 2 tab. 3. Small count differences may be within run-to-run noise; the repeated GPT-4o runs show spread of one to three questions on GPQA-Diamond per run app. B. On AIME, one correct answer equals 3.33 points §5.1.
• Repeated GPT-4o runs are small. Three runs over the same questions give SDs that are not confidence intervals; the three AIME sets have 30 questions each app. B.
• Gain values. Gains use integer counts before rounding. For LeetCode, a zero-to-nonzero change has no percentage and is shown as a dash app. F.
• Cost is high. Per-task model steps and minutes are reported for GAIA, HLE and coding, with API totals of about USD 2,000, 6,000 and 200 respectively. The comparisons are not matched on cost app. B.
• Opportunity control is not equal compute. The repeated comparison caps primary opportunities at three, but auxiliary reflection, judging and editing calls are extra app. B. The paper's own table shows PS-Joint-Evo at about 52 to 61 percent more output tokens than the matched controls app. B.
• Joint editing is not uniformly best. For gpt-4.1 on GPQA-Diamond, PS-Joint-Evo (67.68) is below Prompt-Evo and Solution-Evo (68.69); the paper says the benefit depends on backbone and task tab. 1 §5.1.
• Budget T is not stated per experiment. The algorithm's T counts update iterations; the experiments' value is not given. Sub-calls for proposals, assessments and continuation are counted separately app. G.
• Stop and continuation. A solution-only edit does not trigger a new unconstrained answer; continuation runs only when edits need execution app. G.
• Sandboxing is out of scope. Evaluation code and executable edits need deployment-specific isolation and permissions; the paper does not provide them app. G §6.
• Evolvable markers are not OS permissions. The marker restricts targets only through the manager; frozen implementations can still execute app. G.
• Contamination. The LeetCode problems were chosen by release date. The paper says recency does not show the problems were absent from training data app. F.
• Feedback boundary for label-free runs. In the reasoning runs the optimizer sees the question, its own reasoning and the label-free reflection, but no reference answer or correctness label app. B.
• Figures are not reproduced. Architecture, HLE leaderboard, case and LeetCode figures are described in words here only fig. 1 fig. 2 fig. 4.

Lineage

• Builds on: AutoGen (multi-agent conversation) and planner-specialist frameworks for the AGS organisation §4. Also ReAct and chain-of-thought style reasoning and tool use §1.
• Related refinement methods compared: Self-Refine, Reflexion, TextGrad, MIPRO, GEPA, AFlow, EvoAgentX and Alita. Alita is arXiv:2505.20286 (per the paper's bibliography). The paper states these compare mechanisms, not matched-budget performance §2 app. E.1.
• Communication standards it complements: MCP and A2A §2 app. E.
• Superseded by: not known from the paper or the repository.