Paper · 2026-10-01 · cs.AI
Federated Agent Optimization
Position and survey paper that defines Federated Agent Optimization (FAO): LLM agents improve each other by exchanging typed, protected updates for policy, memory, tools, rewards and skills or rules, while utility, privacy leakage and communication cost are traded off. Gives a multi-objective formulation, a taxonomy of existing methods, and a gate with a Hoeffding-based guarantee. No experiments.
Federated learningLLM agentAgent memoryfederated-learningllm-agentsprivacysurvey
AI-compiled by claude-haiku-5-5 on 2026-11-21, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Companies that run AI agents learn a lot from their daily work: which steps succeed, which tools break, which business rules matter. That knowledge stays inside each company, because the raw logs contain customer data and trade secrets. Federated learning already lets many parties improve one model without pooling data, but it only shares model weights. An agent is more than its weights: it also has a memory, a set of tools, a way of judging its own results and a body of written procedures.
This paper asks how agents could help each other across those boundaries. Each agent shares distilled pieces of what it learned, such as a lesson, a procedure, a rule or a judging standard, instead of logs. A shared step merges these pieces, and each agent adopts only those that help it. The paper treats this as a three-way trade-off between how well the agents perform, how much private information leaks and how much data has to be sent. It also sorts the existing work by which part of the agent is shared, and lists open problems. It is a framing and survey paper: it reports no new experiments.
TL;DR
- Defines Federated Agent Optimization (FAO), a federation of agents that share typed updates over policy, memory, tools, rewards and structured knowledge (skills, rules, ontologies) §3.1 §3.2 eq. 5.
- Scores a protocol on three objectives: local task utility, privacy leakage, and communication cost §3.3 eq. 10.
- Gives a gate that accepts a shared item only when its estimated gain clears a Hoeffding error margin. This bounds the chance that the agent ends up worse off §5.2 eq. 30 eq. 31 [prop. 2].
- No experiments. Its only numbers are payload sizes quoted from cited works eq. 24.
Problem
Conventional federated learning averages parameters, so it cannot express sharing of memories, tools, rewards or skills §1. The paper formalises K clients, each with a private task distribution, a utility and a private dataset of logged trajectories and raw records. None of the raw data may leave the client §3.1 eq. 1.
An agent is a tuple of five components: policy, memory, tools, reward (a local evaluator) and structured knowledge §3.1 eq. 2. Its utility is the expected trajectory score under the client's task distribution, with scores in [0, 1] eq. 3 eq. 4. The reward is only an estimate of the utility, because the true utility is seen only through outcomes and human feedback §3.1.
Method
The paper's contribution is a formulation and a taxonomy, not an algorithm. The main objects are as follows.
Protocol. A coordinator fixes who takes part in each round, the local update, aggregation and adaptation rules, the protection of uploads and the message schedule §3.2. Each round has three steps: a typed local update that has one block per component (blocks may be empty) eq. 5, an aggregation that produces one view per client eq. 6, and a local adaptation that merges the view into the agent eq. 7. Federated averaging is the special case where all clients get the same view. Personalised methods give each client its own view. Protocols without a coordinator are covered by summing over all messages app. D eq. 32.
Objectives. Utility is measured on each client's own tasks eq. 10. Leakage is a property of the randomised mechanism that turns private data into uploads. It can be a differential privacy bound eq. 9, or as the edge that the most capable attacker, under a declared threat model, gains from seeing the uploads. A record counts as recovered when a similarity score passes a threshold eq. 9. Communication cost sums the size of every upload and every returned view over all rounds and participants eq. 8.
Trade-off. Protocols are compared by Pareto dominance eq. 12. The paper gives two scalarised forms: a weighted sum with positive weights eq. 13, and a constrained form that caps leakage per client and total communication eq. 14. Proposition 1 shows that every weighted maximiser is Pareto optimal, and that every Pareto-optimal protocol solves the constrained form with budgets set to its own leakage and cost [prop. 1] eq. 15. Only the constrained form is guaranteed to reach every Pareto-optimal protocol, because the decision space includes text and rules and is not convex §3.3.
Collaboration gain. A client should join only if the federation improves its utility over its local agent eq. 16. The paper treats this as a participation constraint rather than a fourth objective §3.3.
Components. Policy updates are adapter weights or textual prompts merged either by weighted averaging eq. 18, first form or by a language model eq. 18, second form. Memory updates are distilled items that the coordinator condenses and each client retrieves at inference eq. 19. Tool updates are documentation plus call and success counts, merged by summing counts and computing a pooled success rate eq. 20. Reward updates are local heads trained on top of a shared evaluator, with only the shared part leaving the client eq. 21. A cheaper variant sends only one correlation score per candidate reward and picks the candidate with the highest summed score eq. 22. Structured-knowledge updates average entity embeddings over the clients that know an entity, and merge skill patches into each client's library eq. 23 §4 Structured knowledge.
Improvement loop. Knowledge moves from sender j to receiver k through abstraction at a chosen level, protection, aggregation and adaptation eq. 25. Protection releases an artifact only if at least n_min local evidence items support its pattern. Otherwise it is dropped. Sanitisation removes identifiers or adds noise eq. 26. Transfer is favoured when three conditions hold together: the task distributions are close enough, the item is novel relative to what the receiver already has, and the item's concepts and tools exist at the receiver eq. 29. The paper states these as heuristics rather than necessary conditions §5.3.
Gate. The receiver tries the candidate items one at a time. It keeps an item only if its held-out gain is at least twice an error margin e_k eq. 30. With n_val validation tasks, m items and failure probability eta, the margin is eq. 31. Proposition 2 then gives collaboration gain at least zero with probability at least 1 − eta, provided the items are fixed before validation and the validation tasks are independent of them [prop. 2].
Algorithm: one FAO round. This is a reading of eq. 5–eq. 7 in pseudocode. The paper gives no code.
# One FAO round [eq. 5] [eq. 6] [eq. 7]
# Input: agents (one per client), private data, protocol Phi
# Output: agents after adaptation
def fao_round(agents, private_data, participants, protocol):
# Each participant builds a typed update, one block per component [eq. 5]
uploads = {}
for client in participants:
uploads[client] = build_typed_update(
agents[client], private_data[client], protocol)
# Coordinator aggregates the uploads into one view per client [eq. 6]
views = aggregate(uploads, protocol)
# Each client adapts its agent with its own view and local data [eq. 7]
for client in all_clients:
agents[client] = adapt(agents[client], views[client],
private_data[client])
return agentsAlgorithm: gated adoption. This follows eq. 30 and the bound in eq. 31.
# Gated adoption of m shared items [eq. 30] [eq. 31]
# Input: local agent, items, validation tasks per client, failure prob.
# Output: agent after accepting the items that pass the gate
def gated_adopt(local_agent, items, n_val, eta):
# Error margin for held-out estimates [eq. 31]
item_count = len(items)
error_margin = sqrt(
((item_count + 1) * log(2) + log(1 / eta)) / (2 * n_val))
agent = local_agent
for item in items:
# Accept only if the held-out gain clears twice the margin [eq. 30]
candidate = integrate(agent, item)
gain = estimate_utility(candidate) - estimate_utility(agent)
if gain >= 2 * error_margin:
agent = candidate
return agentHyperparameters
Not applicable: the paper trains no system and reports no runs. The table lists the free parameters that appear in the formulation. None of them has a value in the paper.
| Name | Value | Where | Notes |
|---|---|---|---|
| lambda_p, lambda_c | not stated | eq. 13 | positive weights on leakage and communication |
| epsilon-bar_k, B | not stated | eq. 14 | per-client leakage cap and total communication budget |
| epsilon, delta (differential privacy) | not stated | eq. 9 | guarantee parameters; threat model not fixed |
| similarity threshold gamma | not stated | eq. 9 | recovery threshold for an attack |
| n_min | not stated | eq. 26 | support threshold for releasing an artifact |
| Sanitize noise and identifier rules | not stated | eq. 26 | described only in words |
| div_max, nu | not stated | eq. 29 | relatedness and novelty thresholds |
| eta, m, n_val | not stated | eq. 31 | failure probability, item count, validation tasks |
| number of rounds T, participants S^t | not stated | §3.2 | protocol parameters left to the instantiation |
Results
No results. The paper contains no experiments and no benchmark numbers. Its only quantitative content is a comparison of payload sizes, which the authors take from cited works. These are author-reported, and the works measure them at different granularities (per round or per run), as the appendix notes app. B:
| Component | Reported payload | Source |
|---|---|---|
| policy, adapters of rank 8 | about 76 MB per round | eq. 24 (cites FedSE) |
| tools, tool knowledge | 0.08 to 20 MB | eq. 24 (cites FICAL, typed artifacts) |
| structured knowledge, skill patches | 0.1 to 0.3 MB | eq. 24 (cites FederatedSkill) |
The paper also notes that weight updates can shrink to kilobyte scale under compression eq. 24 app. B.
Paper vs code
No code is linked in the paper, its abstract or its source files. A GitHub
search was not run in this session, because the web search tool was denied
permission. The search therefore remains unverified. The paper is a position
and survey paper, so an official implementation is unlikely, but this is not
confirmed. No code field is set.
Pitfalls
- The three objectives are not all set by the paper. The formulation fixes the shape of the trade-off but no weights or budgets. An implementer must pick lambda_p, lambda_c or the caps, and the paper says only that these express soft preferences or regulatory limits §3.3.
- Only the constrained form reaches all Pareto-optimal protocols. Positive weights reach only supported points on the front, which lie on the convex hull §3.3 [prop. 1].
- The gate's guarantee is conditional. It needs fixed items, independent validation tasks and one sampled trajectory per task [prop. 2]. It is a probability statement, not a bound on each run. It also covers only the participation constraint, not pairwise mutual improvement, which the paper says is not implied §5.2.
- The error margin grows with the number of items. Its term for the 2^m reachable agents makes the margin widen with m, so the paper advises filtering or batching items §5.2 eq. 31.
- Shapley-style contribution estimates are costly and not specified. The paper defines pairwise contribution by withholding one client's updates and cites Shapley values as a way to average such marginal effects over subsets. It gives no estimator or cost §5.2 eq. 28.
- Privacy notions are not comparable. The differential privacy guarantee and the attack advantage measure different things. The paper says an instance must pick one scalar metric and add other notions as objectives or constraints §3.3 eq. 9.
- Text leaks more directly than weights. Readable artifacts can quote records verbatim, while weights must be inverted first. The paper treats text and rules as having a wider leakage surface and no formal guarantee in the cited works §3.3 §4 Memory app. B.
- Ratio estimates are under-specified. The pooled success rate in eq. 20 is a ratio of summed counts. The paper does not say how a zero denominator is handled, and says secure aggregation can protect the sums §4 Tool use.
- Some merge steps are only named. Memory distillation (Distill), the language-model merge of prompts, the reward-correlation selection and the skill-patch merge are described by name only. Their prompts, models and thresholds are not stated eq. 18 eq. 19 eq. 22 eq. 23.
- Heuristic transfer conditions. The three conditions in eq. 29 are explicitly heuristics. A distant task can still share a useful strategy, and a known item can add robustness. Heterogeneity can be both the obstacle and the source of value §5.3.
- Mislabelled heading in the appendix. The appendix section titled "The Four Types" lists five types: experience, skills, rewards, ontologies and rules app. C. The main text says five §5.2 tab. 2.
- Preprint sources. Table 1 marks several methods with a dagger, which means they are preprints tab. 1. Treat their results as unreviewed.
- Scope of the literature claim. Claims that no method federates an evaluator for agent trajectories, or that no method trades the objectives off across components, rest on the authors' search up to 29 September 2026 §4 §4 Comparison. This card did not re-run that search.
- Payload numbers are not like for like. The main text says the reported payloads differ by about three orders of magnitude eq. 24. The appendix notes that the studies measure them at different granularities app. B.
Lineage
- Builds on: federated learning and federated fine-tuning §2, agent architectures with five components (CoALA, cited in §3.1), and federated knowledge graph embeddings for the ontology type §4 Structured knowledge.
- Compared against: federated averaging as a special case eq. 17; experience pooling without privacy; single-agent optimisation with DSPy and TextGrad eq. 17. The paper also contrasts itself with constrained multi-objective federated learning §3.3.
- Superseded by: not applicable.
Source and license
Adapted from “Federated Agent Optimization” by Qiang Yang, Zhiqiang Kou, Xueyi Zhang, Dong-Dong Wu, Hanlin Gu, Jing Guo, Yang Liu, Di Jiang, Qian Xu (arXiv:2610.01195), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.