Multi-agent systems

Overview

A multi-agent system here is a group of language-model agents that work toward one goal. Each agent has a role, its own prompt and often its own tools. A coordinator, a shared plan or a debate joins them, and their results are combined 2505.10468 §3.1.2.

The agents differ mainly in how they interact:

• a fixed pipeline that passes documents between roles 2308.00352 §3.1 2308.00352 §3.2;
• dialogue between one agent per robot, ending in a plan that rules check 2307.04738 §3.1 2307.04738 §3.2;
• a coder and a reviewer in a group chat, optionally behind a planner, that write robot code from a spoken request 2411.16723 §3.1;
• a panel of expert agents that votes and revises a shared report 2311.10537 §2.4;
• a panel of expert agents that debates a model design under a critic, then a coding agent writes, runs and repairs the training code 2508.02276 § Discussion protocol 2508.02276 § Code generation and repair;
• reviewer, author and area-chair agents that run a simulated review, each reviewer with written traits 2406.12708 §2.1 2406.12708 §2.2;
• an orchestrator that routes questions to specialists and checks their answers 2503.10265 §3.2 2503.10265 §3.3;
• a supervisor that generates, ranks and evolves ideas 2502.18864 p. 29;
• a planner over specialists whose prompts, tools and code are edited during the run 2604.15034 §4 2604.15034 §3.2;
• a 7B planner, trained with reinforcement learning, that writes a short plan for a pool of worker models: which worker runs each step, the subtask text, and which earlier outputs each step may read 2512.04388 §3.1 2512.04388 app. E;
• a tree search whose moves are agent actions: retrieve, analyse, filter, analyse the candidates, select, or stop. Rollouts are scored and their values are backed up the tree 2507.07426 §2.1 2507.07426 §2.2;
• a population of persona agents that vote and message over signed ties. The group is studied as a dynamical system, not as a team that solves a task 2608.16578 §2 2608.16578 §3.2;
• two chat agents, an assistant and a user, that take turns under fixed system prompts after a task-specifier agent makes the idea concrete 2303.17760 §3.1 2303.17760 §3.2;
• copies of one model that answer, read each other's answers and revise over a few rounds 2305.14325 §2.1;
• a framework in which agents exchange messages, and a group manager or a program decides who speaks next 2308.08155 §2.2;
• two debates in sequence: one picks a statistical causal-discovery algorithm and runs it, and a second refines the graph with text and metadata 2407.15073 §3.4 2407.15073 §3.5;
• a society of agents with traits and memory that farm, rob, trade or donate, and answer each other's requests 2406.14373 §3.3 2406.14373 §4;
• a fixed sequence of prompted stages for a robot team, with no messages between agents 2309.10062 §4;
• a supervisor and workers that share a provenance store, where each tool call is checked before it runs 2507.14267 § Methods: Safety guard.

A conceptual framework describes agents as tuples of model, role, state and permissions, with a stateless oracle and a supervisor that halts loops 2306.03314 §2.1 2306.03314 §3.4 2306.03314 §3.5.

The review draws one line that the cards use. A single AI Agent is a language model with tools that handles one bounded task. Agentic AI adds specialized agents, planning, persistent memory and an orchestration layer 2505.10468 §3.1.1 2505.10468 §3.1.2. The authors call this a convention, not a formal definition 2505.10468 §3.0.1.

The evidence is thin and mostly author-reported. One source is a narrative review with no experiments 2505.10468 §1.1. One is a conceptual framework with no experiments 2306.03314 §6. Another is a perspective on biomedical agents that reports no experiments at all 2404.02831 §1. The others report experiments, several in single runs or on small samples 2307.04738 tab. 2 2502.18864 p. 11 2604.15034 tab. 1. The peer-review simulation compares each setting with its own baseline, not with real review outcomes. Its authors call that comparison unreliable, because reviewer traits are unknown 2406.12708 §2.4. The debate paper uses 100-item sets for its arithmetic and GSM8K results 2305.14325 app. A.2. The framework paper's group-chat comparison is 12 tasks with one run per setting 2308.08155 app. A5. The drone survey reports no experiments at all 1805.00881 §I. The robot-code study has 63 scored samples and reports no statistical test in its text 2411.16723 §4. The distributed MPC paper's timings come from one machine 2412.19669 tab. III. The three newest surveys report none either 2503.23037 §1.2 2504.19678 §III 2506.04133 §1. The causal-discovery paper's ten-seed tables overlap across methods: on Auto with Gemini, its F1 is 0.61 ± 0.08 and the LLM-SCD baseline's is 0.58 ± 0.07 2407.15073 tab. 2. The simulation's 85-run means come from a proprietary model whose snapshot and temperature are not stated 2406.14373 §4.1 2406.14373 §7.2. The planner paper's baselines are each model's best score across the authors' own runs, private code and online leaderboards, so its comparison is not controlled 2512.04388 §4.2.

Variants

Agent tuple with creation and halting rights (conceptual)

Each agent holds a model, a role, a state, a flag for whether it may create agents, and the set of agents it may halt. Plugins supply tools. The paper gives no algorithm and no numbers, and it describes existing systems in these terms 2306.03314 §2.1 2306.03314 §4. Card: Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents.

Software roles with a shared message pool

Five roles, product manager, architect, project manager, engineer and QA engineer, follow a fixed procedure and pass structured documents, not chat. Each role reads only the messages it subscribed to. The engineer writes and runs tests and debugs on failure 2308.00352 §3.1 2308.00352 §3.2 2308.00352 §3.3. Author-reported: 85.9% HumanEval and 87.7% MBPP Pass@1 2308.00352 §4.2. Card: MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework.

Per-robot dialogue with ordered plan checks

Each robot has its own LLM agent. The agents speak in turns and end with one sub-task per robot. Ordered checks (task rules, reach, inverse kinematics, collisions, waypoint spacing) return failures as text, and the dialogue starts again. A centralized multi-arm planner then produces the trajectories 2307.04738 §3.1 2307.04738 §3.2 2307.04738 §3.3. Zero-shot with GPT-4 on six simulated tasks, author-reported success is 0.44 to 0.95 per task 2307.04738 tab. 2. Card: RoCo: Dialectic Multi-Robot Collaboration with Large Language Models.

Coder and reviewer in a group chat (robot code from speech)

A coder agent writes Python for a legged robot from a spoken request. In the two-agent set, a reviewer checks the code for correctness, task fit and safety, and sends it back with feedback or passes it on. A group-chat manager picks the next speaker. The three-agent set puts a planner first, which writes a plain-language plan for the coder 2411.16723 §3.1. Seven prompts, three runs each, one observer per run 2411.16723 §3. The two-agent set has the fewest first-attempt code errors: 9 points below the single agent and 14 below the three-agent set. The three-agent set is about 5 points above the single agent 2411.16723 §4 2411.16723 fig. 6. Observer scores show no significant difference across configurations 2411.16723 §4 2411.16723 fig. 8. Card: Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction.

Expert panel: analyse, summarise, vote, revise

Question experts and option experts each write an analysis. A summary report goes to a yes-or-no vote. Any objection brings a revision, for up to a fixed number of rounds. A last prompt picks one answer. No examples are given in any prompt 2311.10537 §2.1 2311.10537 §2.2 2311.10537 §2.3 2311.10537 §2.4 2311.10537 §2.5. Author-reported average over nine medical datasets: 86.7 for GPT-4 and 72.1 for GPT-3.5 2311.10537 tab. 2. Card: MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning.

Orchestrator, specialists and a rule-and-rating panel

A top-level orchestrator sends visual questions and knowledge questions to different agents. Visual action answers pass a panel that checks a knowledge-graph table of instrument-action pairs and two 1-to-5 ratings, with up to three reruns. Knowledge questions get retrieved text first 2503.10265 §3.2 2503.10265 §3.3. Zero-shot on a surgical video benchmark, GPT-4o reaches 61.73% overall with the workflow and 31.44% without it 2503.10265 tab. 3. Card: SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis.

Expert panel debates a model design, then code is written and repaired (AI scientist)

The experts share one LLM backend and differ only by system prompt. They cover data modelling, single-cell biology and deep learning, and a critic moderates them. Each round, the experts write plans, peers score each other's plans, the critic writes a meta-review with a score, and the plans are revised. Each expert's confidence is a weighted sum of its previous value, the critic's score and the mean peer score. The loop stops when every confidence is at least 0.8 and the spread is under 0.03, or after a round limit of 20. The critic then picks one plan 2508.02276 § Discussion protocol 2508.02276 eq. 1. A coding agent writes training code from that plan, runs it, reads the errors and patches the code, for up to five attempts 2508.02276 § Code generation and repair. The paper says the confidence is not a calibrated probability or a performance measure [arxiv:2508.02276 § Critic-mediated expert panel]. Author-reported on six perturbation datasets, averaged over three runs: PCC 0.9883 for CPA-X on Adamson CRISPRi, and MSE_DE 0.1736 for scGen-X on Norman CRISPRa 2508.02276 § Discussion 2508.02276 § Overview. Most per-dataset baselines appear only in figures 2508.02276 fig. 3 2508.02276 fig. 4. Card: AI Scientists for Building Virtual-Cell Models.

Supervisor, generation, ranking and evolution

A supervisor turns a research goal into a plan and runs six agent types from a task queue: generation, reflection, ranking, proximity, evolution and meta-review. Ranking is an Elo-style tournament of hypotheses. Meta-review feedback is added to every agent's prompt, so the system changes without fine-tuning 2502.18864 p. 29 2502.18864 p. 31 2502.18864 p. 34 2502.18864 p. 35. On 11 goals, expert preference ranks give an average of 2.36, against 2.45 for the best baseline 2502.18864 p. 11. No public code. Card: Accelerating scientific discovery with Co-Scientist.

Planner and specialists with self-editing resources

The AGS system has a planner and five specialists: deep research, browser use, deep analysis, tool calling and coding. Its prompts, agents, tools, environments and memory are registered as versioned resources. A reflect, select, improve, evaluate and commit loop proposes edits, and a candidate is kept only if it passes assessment 2604.15034 §3.1 2604.15034 §3.2 2604.15034 §4 2604.15034 app. G. On GAIA test, the non-evolving configuration scores 79.07 and the evolving one 89.04 2604.15034 tab. 2. These numbers measure the edits, not the team structure. Card: Autogenesis: A Self-Evolving Agent Protocol.

Tree search over agent actions (drug repurposing)

The search drives the run, and the agents are its moves. Five agents and an end action make up six actions. Four of the five agents call the same LLM. The retrieval agent calls no model 2507.07426 §2.3. Monte Carlo tree search picks which move to expand next by UCT 2507.07426 §2.2. A rollout's reward is the mean of a relative score, how often the model picks the same target, and an absolute score, the share of yes answers 2507.07426 eq. 4 2507.07426 eq. 5 2507.07426 eq. 6. The setup uses Qwen2.5-7B-Instruct without fine-tuning and 12 rollouts 2507.07426 §3.2. Author-reported DrugBank and KIBA recall is 0.4466 and 0.4224 with the ground-truth count, and 0.5534 and 0.4924 with that count plus three 2507.07426 tab. 1. Card: DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree Search.

Trained planner over a worker pool (learned orchestration)

The planner is a 7B model, starting from Qwen2.5-7B. It writes one to five steps as three equal-length lists: worker ids, subtasks and access lists. An access list says whether a step sees no earlier context or all of it 2512.04388 §3.1 2512.04388 app. E. Training uses GRPO with a reward of 1 for a correct final answer, 0.5 for a wrong one that parsed, and 0 for an unparsable plan 2512.04388 §3.1. The paper trains for 200 iterations of 4 questions with 64 rollouts each 2512.04388 app. A. Workers are named only by ordinal id, so the planner does not see model names 2512.04388 app. E. Author-reported, unconstrained, the planner beats the three listed models on all seven benchmarks; on LiveCodeBench v6 it scores 83.93, against 82.90 for GPT 5 2512.04388 tab. 1. Under constraints (4096 output tokens, minimal reasoning), it averages 72.35 on four tasks, against 64.14 for the best single model and 62.13 for a mixture-of-agents baseline with eight model calls 2512.04388 tab. 7. A recursive variant lets the planner call itself again, with a cap on inner calls 2512.04388 §3.2. On BigCodeBench it scores 40.0 against 37.8 without recursion 2512.04388 tab. 2. The recursion fine-tune reuses questions seen in training, so these results do not show generalisation to new questions 2512.04388 §4.4. The paper names no code. Card: Learning to Orchestrate Agents in Natural Language with the Conductor.

Staged prompts for a robot team, with no dialogue (task planning)

No agents talk to each other. Four stages run in order: decompose the instruction into sub-tasks, form a robot or a team for each sub-task, write code that allocates the sub-tasks and runs independent ones in parallel, and execute the code through an interpreter. Each stage is one prompt built from Python-style worked examples, and no stage checks the one before it 2309.10062 §4 2309.10062 §4.4. Author-reported GPT-4 success on a 36-task AI2-THOR benchmark is 1.00 on elemental tasks and 0.62 to 0.71 on the other three categories 2309.10062 tab. 1. The released script makes three model calls, not four, and accepts only three GPT model names 2309.10062 § Paper vs code. Card: SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models.

Hierarchical planner with a provenance guard (materials simulation)

A planning supervisor splits the goal into plan steps, each naming a worker. Workers handle structures and input files, cluster jobs and convergence. All tool outputs go into a shared store with a provenance record. Before a tool runs, each referenced value is checked against that store, each parameter is judged by an LLM, and some tools get a cross-parameter judge. A report judge walks the provenance graph before a result is accepted [arxiv:2507.14267 § Methods: Safety guard] 2507.14267 fig. 2 2507.14267 fig. 5. The paper says a warning still registers the value and that the guard fails open on infrastructure errors 2507.14267 § Methods: Safety guard. Author-reported on periodic DFT: lattice constants within about 1% MAPE of expert DFT, and a CO/Pt(111) adsorption-energy difference within 2 meV of the expert value 2507.14267 tab. 2 2507.14267 tab. 4. Card: DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulation.

Agent population on a signed social graph (simulation)

Not a task team. Thirty-two persona-conditioned LLM agents vote on binary questions over eight rounds and exchange messages over signed ties 2608.16578 §2 2608.16578 §3.2. A logistic update with separate concordant, discordant and unsigned couplings predicts each agent's next vote 2608.16578 §4 2608.16578 tab. 1. Author-reported one-step balanced accuracy on held-out questions is 75 to 86 percent, and multi-step rollouts score lower 2608.16578 §5.1. The measure is agreement with observed votes, not correctness on a task. Card: Physics of Agents: Statistical Mechanics Predicts Collective Behavior of AI Agents.

Agent society with traits, memory and scarce resources (simulation)

Nine gpt-3.5-turbo agents share food and land. Each has sampled traits (aggressiveness, covetousness, strength), a memory of its recent actions, and one initiative action per day: farm, rob, trade or donate [arxiv:2406.14373 §3.2] 2406.14373 §4. A robbery target either resists or concedes. Conceding makes the robber a permanent superior. A contest gives a sigmoid win probability for the strength gap, and how that becomes one outcome is not stated 2406.14373 §3.3 2406.14373 eq. 1. Author-reported over 85 runs: robbery falls from 0.391 to 0.037 once a sovereign exists, and farming rises from 0.477 to 0.737 2406.14373 §7.2. The run stop rule and the pooling of runs are not explained 2406.14373 §7. Donation was offered and never chosen 2406.14373 §8.6. The paper's covetousness prior conflicts with its prompt range, so the trait values are not settled 2406.14373 §3.2 2406.14373 app. A. Card: Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory.

Review panel with trait-labelled agents (peer-review simulation)

Three reviewer agents, an author and an area chair run five phases on one paper: reviews, rebuttal, reviewer–AC discussion, meta-review and a batch decision. Each reviewer's traits come from prompt text. Each run changes one trait and is compared with a run that has none 2406.12708 §2.1 2406.12708 §2.2 2406.12708 §2.4. Runs use one AC and a fixed 32% acceptance rate 2406.12708 §2.2. Author-reported: one malicious reviewer changes 37.1% of decisions. Table 2 gives 62.91% agreement with baseline for that row, and the paper does not show how the 37.1% was derived 2406.12708 tab. 2. The paper gives about USD 2,780 of API spend for all experiments 2406.12708 app. A.2. The authors advise against LLM reviewers in real review 2406.12708 §2.4. Card: AgentReview: Exploring Peer Review Dynamics with LLM Agents.

Copies of one model that debate over rounds

N copies of one chat model answer the question. In each later round, every copy reads the other copies' latest answers and is asked to check them and update its own. The method needs only text prompts, so it works with any chat model 2305.14325 §1 2305.14325 §2.1. The paper's table uses 3 agents and 2 rounds. Its code counts the first answers as round 1, so the same setting is one debate update 2305.14325 tab. 1. Author-reported, against a single agent: arithmetic 81.8 against 67.0, GSM8K 85.0 against 77.0, and biography factuality 73.8 against 66.0 2305.14325 tab. 1 2305.14325 tab. 2. The paper gives no aggregation rule in the text. Its code takes a majority vote over the final answers 2305.14325 §2.1. Card: Improving Factuality and Reasoning in Language Models through Multiagent Debate.

Two-agent role-play with a task specifier (data generation)

A task-specifier agent turns an idea into a concrete task. An assistant and a user, each with a fixed system prompt, then take turns. The user gives one instruction per turn, the assistant answers, and the user ends the chat with a stop token. The prompts fix the roles, forbid role flips and fix the answer format 2303.17760 §3.1 2303.17760 §3.2. Author-reported: CAMEL solutions win 76.3% of human votes against single-shot gpt-3.5-turbo on AI Society, and a LLaMA-7B tuned on the generated data scores 14.0 pass@1 on HumanEval, against 10.5 for the base model 2303.17760 tab. 1 2303.17760 tab. 3. The paper caps conversations at 40 messages 2303.17760 §4. Card: CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society.

Conversation programming with auto-reply agents (framework)

Agents are conversable: each sends and receives messages, and each can be backed by an LLM, a human, a tool or a mix. A conversation runs by auto-reply until a termination condition holds. Control can sit in Python, in a system prompt, or in both 2308.08155 §2.1 2308.08155 §2.2. Built-in patterns include a two-agent loop and a group chat run by a manager that picks the next speaker 2308.08155 §2.2 2308.08155 §3 A5. Author-reported: MATH full test set 69.48% against 55.18% for vanilla GPT-4 2308.08155 app. A1, and ALFWorld success of 69 for a three-agent design against 54 for ReAct 2308.08155 app. A3. The chat-loop limits in the library, such as 100 auto-replies and 10 group rounds, are not stated in the paper. They come from a later library release, read through summaries, and are unconfirmed line by line (see the card's Paper vs code section). Card: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation.

Debate on an algorithm, then debate on the graph (causal discovery)

Both stages use one three-role debate. An affirmative agent proposes, a negative agent proposes a plausible alternative, and a judge picks the winner 2407.15073 §3.3 2407.15073 alg. 1. Stage one has the debate choose a statistical causal-discovery algorithm from a pool, and a coding agent runs it on the full data to give a graph 2407.15073 §3.4. The pool is only partly named (PC, FCI and GES) 2407.15073 app. A.2. Stage two turns the graph into sentences, adds them to the metadata, and runs the debate again on graphs 2407.15073 §3.5 2407.15073 alg. 3. Author-reported: averaged over datasets and backbones, the method ranks first on every metric, and with Gemini-2.0-Flash it is first on 10 of 15 evaluation points 2407.15073 §4.2. The Earthquake result is matched by the Exact Search baseline 2407.15073 tab. 2. Card: Multi-Agent Causal Discovery Using Large Language Models.

Federated reinforcement learning (not an LLM team)

Listed here because its card carries this concept, but it is a different setting. In vertical federated RL, agents may have no action or reward, and states cannot be shared. The paper says the closest existing RL setting is multi-agent RL, and that the two differ on these points 2108.11887 §4.3. Card: Federated Reinforcement Learning: Techniques, Applications, and Open Challenges. Do not transfer its results or methods to LLM agent teams.

Decentralised patrol with a neighbour agreement (no language model)

Each drone patrols one perimeter segment. When a neighbour agrees, the two meet at their shared border and move in the set direction. A drone turns back at the perimeter end 1805.00881 alg. 1 1805.00881 §X. The algorithm comes from a study the survey reviews, and the survey reports no experiments of its own 1805.00881 §I. It is a multi-robot pattern, not an LLM team, so do not transfer it to LLM agents. Card: Civil UAV applications survey.

Distributed MPC with learned policies (no language model)

Each robot has an actor that maps its own and its neighbours' formation errors to a control, and a critic that estimates the cost. Both are trained online in each prediction interval, and the robot applies the actor's output 2412.19669 §IV-B 2412.19669 eq. 14. A check against a baseline cost resets the weights when it fails 2412.19669 §IV-C 2412.19669 eq. 18. Author-reported, on collision avoidance with two robots, the learned policy's per-step cost is 13.5 against 5.0 for the IPOPT-solved baseline. Its per-step deploy time is 2e-5 s, against 0.03 to 0.05 s for the baseline 2412.19669 tab. III. The safe variant with barrier terms relies on steps and proofs in an auxiliary file that is not in the source package 2412.19669 §V-B. This is not a language-model team, so do not transfer its results to LLM agents. Card: Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPC.

Survey views: role teams, agent protocols and risk controls (no experiments)

Three surveys add no team design of their own and no measured result. The agentic-LLM survey files role-based teams and open-ended societies under interacting agents, and its table rows give no scores 2503.23037 §4.2 2503.23037 §4.3 2503.23037 tab. 3. Card: Agentic Large Language Models, a survey.

The agent review sets out a loop of planning, acting with tools and assessing the result. The paper gives no stop rule or retry budget for it 2504.19678 §IV.A. For agent-to-agent messages it describes A2A, where agents publish capabilities as Agent Cards and tasks share an ID, and ACP, where a session ID keeps state and retries fall to the client 2504.19678 §IV.C. It calls protocol security largely unexamined 2504.19678 §V.F. Card: From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review.

The TRiSM review applies trust, risk and security management to LLM-based multi-agent systems. It lists six threats, among them tool misuse, memory contamination and multi-agent collusion or drift, and sorts risks into four families 2506.04133 §3 2506.04133 fig. 9. Its five pillars are explainability, ModelOps, security, privacy and lifecycle governance 2506.04133 §9. Its Component Synergy Score sums, over ordered agent pairs, the impact of one agent on another times the quality of the second agent's output, then divides by the number of pairs 2506.04133 §5 2506.04133 eq. 4. The paper gives one worked link (0.35 times 0.92) and no estimator for impact or quality, and the score is not validated on real systems 2506.04133 §5. Card: Review of trust, risk and security for LLM multi-agent systems.

Collaboration schemes for biomedical discovery (perspective, no experiments)

A perspective on AI agents for biomedical research names five ways a team can work together. In brainstorming, agents pool many ideas, and volume counts for more than quality at first. In expert consultation, one agent or a human gathers evidence and feeds it back to the others. In a research debate, one team argues for a position and another critiques it. In a round-table, agents exchange and revise proposals over several rounds. In a self-driving laboratory, the whole discovery loop runs under a scientist's broad direction, without step-by-step oversight 2404.02831 §3.2 2404.02831 fig. 3. It argues that heterogeneous teams, with ML tools and human experts as well as language models, suit this work better than one model playing several roles 2404.02831 §3. It names three ways to assign roles: fine-tuning, in-context prompts and automatic role generation 2404.02831 §3.1. No scheme is measured. Card: Empowering Biomedical Discovery with AI Agents.

Single AI Agent (the contrast case)

One language model with tools, one bounded task. It runs a reason, act, observe loop and has no coordinator 2505.10468 §6.1. The multi-agent designs above add coordination on top of this loop. The clearest contrast is in the medical cards. With GPT-4, MedAgents averages 86.7, against 80.6 for zero-shot prompting and 85.4 for few-shot chain-of-thought with self-consistency 2311.10537 tab. 2. The gain over the strongest few-shot baseline is 1.3 points.

What the ablations show

Only some papers remove one step at a time. The rows below are the ones that do.

The largest measured effects in these rows come from prompting or first-stage structure: chain-of-thought in SurgRAW, the analysis stage in MedAgents, and feedback in MetaGPT and RoCo. The Conductor rows fit that pattern: in its constrained setting, dropping the few-shot examples costs about 9.4 points on LiveCodeBench, and dropping the subtask text about 5.7 (our arithmetic from the table). The later interaction steps give mixed results. DrugMCTS is different: removing both analysis agents costs about 0.10 DrugBank recall in the first cell, and dropping the absolute reward costs about 0.01 2507.07426 tab. 2. MedAgents' consultation adds 2 points on MedQA, and SurgRAW's panel adds 3.09 points on the visual average. RoCo's history matters on average but not on every task. Effects of 2 to 3 points on 300 items are inside the noise: one binomial estimate there is about 2.8 points (our arithmetic from the card) 2311.10537 §3.1. In the causal-discovery paper, removing the negative agent and the judge costs a lot of F1 on Survey and Auto, but not on Cancer with GPT-4o 2407.15073 tab. 6.

When to use which

The cited results support the rows below. No paper gives a rule for how many agents to use, and the review says so 2505.10468 §6.1.

Implementation sketch

The sources do not give one multi-agent algorithm. The sketch below combines their common parts: a coordinator, specialists, ordered checks, shared memory and a round cap. Stop rules, memory schema and checks are choices the implementer must make.

# Plan, act, check and revise in bounded rounds  [arxiv:2307.04738 §3.2]
# Input:  goal (text), roles (name -> agent), checks, max_rounds
# Output: combined answer (text), or None when no round passes

def run_team(goal, roles, checks, max_rounds):
    # Shared memory of messages and results  [arxiv:2308.00352 §3.2]
    memory = []
    # Coordinator splits the goal into one sub-task per role
    #   [arxiv:2307.04738 §3.1] [arxiv:2503.10265 §3.2]
    plan = coordinator(goal, roles, memory)
    for _ in range(max_rounds):
        # Each specialist works on its own sub-task  [arxiv:2308.00352 §3.1]
        results = {r: roles[r](plan[r], memory) for r in plan}
        memory.append(results)
        # Checks run in order. They may be rules (RoCo) or model ratings
        # (SurgRAW); the first failure becomes the feedback
        problem = first_failure(checks, results)
        if problem is None:
            return combine(goal, results)
        # Feedback goes back to the coordinator for a new plan
        plan = coordinator(goal, roles, memory + [problem])
    # Round cap reached: report failure, do not guess
    #   [arxiv:2311.10537 §2.4]
    return None

Contradictions

• Team versus one central planner. In RoCo the centralized planner, which sees the full scene, averages 0.81 over six tasks. The dialogue averages 0.75. The dialogue is higher on two tasks, Sort Cubes (0.93 vs 0.70) and Move Rope (0.65 vs 0.50) 2307.04738 tab. 2. RoCo is the only card that compares a team with a centralized planner. Its own text says the dialogue is sometimes comparable to the central planner 2307.04738 §5.1.
• Medical gains do not carry over in one comparison. MedAgents on GPT-4 averages 86.7 across nine medical question sets 2311.10537 tab. 2. In the SurgRAW table, the run of MedAgents on GPT-4o scores 31.96% overall, and MDAgents scores 32.88%. Both sit close to zero-shot GPT-4o at 31.44% 2503.10265 tab. 3. The backbones, data and tasks differ, so these do not refute each other. They do show that the team design did not add to the single model on the surgical benchmark.
• Paper and code disagree on basic settings. The MedAgents paper uses temperature 1.0 and up to five rounds; its repository uses 0 and three 2311.10537 §3.1. MetaGPT's paper allows three retries; its repository allows five test rounds 2308.00352 §3.3. Reproduced numbers may differ. See the cards' Paper vs code sections: /papers/2311.10537, /papers/2308.00352.
• Team structure is not checkable in the released code. The Autogenesis paper describes a planner and specialists. Its repository's HLE config says the old planner pipeline is not part of the release, and the answering pipeline is a single agent 2604.15034 §4. See the Autogenesis card.
• Search versus team. DrugMCTS beats single-shot models with and without RAG in its headline table 2507.07426 tab. 1. Its ablation rows do not include one agent run inside the same search. Its relative reward alone comes close to the full method 2507.07426 tab. 2. So the gain mixes search, agent steps and the scoring rule, and cannot be credited to the team alone.
• Single agent versus debate, by dataset (causal discovery). The text says the single-agent variant is "competitive" on most datasets, and that Earthquake and Cancer are comparable 2407.15073 §4.4. The table shows the single agent at F1 1.00 on Cancer with GPT-4o, against 0.83 for the full method 2407.15073 tab. 6. The table also shows large gaps on Survey and Auto 2407.15073 tab. 6. The text and table do not agree on Cancer.
• Simulated message scheme. The Physics of Agents paper says each agent writes one message per round 2608.16578 §2. Its code says two of the four models were generated with one message per recipient. The card's Paper vs code section records this. See the card.
• Review versus measured results. The review lists collaborative medical decision support as an application area. It gives no measurement for it 2505.10468 §4.0.2. MedAgents reports measured gains on medical multiple-choice questions 2311.10537 tab. 2. Do not cite the review as evidence for the MedAgents result.
• Text and table differ in the peer-review paper. The AgentReview text gives 69.8% agreement for authoritarian area chairs. Its table gives 69.41%, with the same Cohen's κ of 0.266 in both 2406.12708 §3.2 2406.12708 tab. 2.
• Debate helps in one paper and not in another. The debate paper reports arithmetic accuracy of 81.8 with three agents against 67.0 for one agent 2305.14325 tab. 1. The AutoGen paper says its authors did not run multi-agent debate on the full MATH set, because on the smaller set it did worse than vanilla GPT-4 2308.08155 app. A1. The models, tasks and prompts differ, so neither result refutes the other. Do not quote either as settling whether debate helps.
• Does a team beat the best single model? Under caps, the learned planner averages 72.35 against 64.14 for the best single model [arxiv:2512.04388 tab. 7]. MedAgents adds 1.3 points over its strongest few-shot baseline on medical questions 2311.10537 tab. 2, and in SurgRAW the MedAgents run on GPT-4o scores 31.96% against 31.44% for zero-shot GPT-4o 2503.10265 tab. 3. The planner's setting uses minimal worker reasoning and a 4096-token cap, which the other two papers do not share. The three results therefore do not settle the question between them.
• Agents reading each other: a gain or a risk. Copies of one model that read each other's answers reach 81.8 on arithmetic, against 67.0 for one agent 2305.14325 tab. 1. The same paper says the consensus can settle on a wrong answer 2305.14325 §5. The TRiSM review lists group-level errors, where agents reinforce each other, as a risk family 2506.04133 §3 2506.04133 fig. 9. The review measures nothing for that risk, so the two sides are not tested against each other.
• Two agents beat three, or the gap is not significant. The robot-code paper concludes that the two-agent set performs better than the three-agent one 2411.16723 §5. Its results text says observer scores do not differ significantly across configurations 2411.16723 §4. The Discussion also calls the effect significant, and the paper reports no test 2411.16723 §5. The card treats the conclusion as tentative.
• Cost metrics disagree in the distributed MPC paper. Per-step cost is lower for the learned policy with no obstacles 2412.19669 tab. III. The cumulative cost at two robots favours the solver, 8.9 against 16.9 2412.19669 §VI, and the collision-avoidance per-step cost at two robots favours it too, 5.0 against 13.5 2412.19669 tab. III.
• Paper and repository disagree on the design loop. The paper sets 20 discussion rounds and a confidence update with weights 0.3, 0.4 and 0.3 2508.02276 § Discussion protocol 2508.02276 eq. 1. The repository's pipeline runs at most six rounds and uses other weights. The paper names OpenHands as the code agent, and the repository defaults to Codex. The card's Paper vs code section lists these differences: /papers/2508.02276. Results from the paper's design cannot be assumed to follow from the released code.

Papers

• AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges: narrative review that separates single-entity AI Agents from multi-agent Agentic AI, with qualitative comparisons and a list of challenges and mitigations.
• Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents: conceptual framework of agent tuples, plugins, an oracle and a supervisor; no experiments.
• MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework: five software roles that pass documents through a message pool, with executable feedback.
• RoCo: Dialectic Multi-Robot Collaboration with Large Language Models: one LLM agent per robot in a dialogue, with ordered plan checks and a centralized motion planner.
• MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning: role-play experts analyse, vote and revise a report for zero-shot medical multiple-choice questions.
• SurgRAW: Multi-Agent Workflow with Chain of Thought Reasoning for Robotic Surgical Video Analysis: orchestrator, specialists and a rule-and-rating panel for surgical video questions.
• Accelerating scientific discovery with Co-Scientist: supervisor and six agent types that generate, rank and evolve research hypotheses.
• Autogenesis: A Self-Evolving Agent Protocol: registry of versioned agent resources and a commit-on-accept update loop, used with a planner and specialists.
• Learning to Orchestrate Agents in Natural Language with the Conductor: a 7B planner trained with GRPO writes a short plan of worker models, subtasks and visibility lists; no code is named.
• DrugMCTS: a drug repurposing framework combining multi-agent, RAG and Monte Carlo Tree Search: five LLM-backed agents as moves in a tree search for drug-target recall, without fine-tuning.
• AgentReview: Exploring Peer Review Dynamics with LLM Agents: simulated five-phase review with trait-labelled reviewers, an author and an area chair; one trait varied at a time.
• Physics of Agents: Statistical Mechanics Predicts Collective Behavior of AI Agents: persona agents voting over signed ties, with a fitted one-step update; a simulation study.
• Federated Reinforcement Learning: Techniques, Applications, and Open Challenges: survey of federated RL, including a vertical variant that the paper separates from multi-agent RL.
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society: two role-playing chat agents with a task specifier and fixed prompts, used to generate training conversations.
• Improving Factuality and Reasoning in Language Models through Multiagent Debate: copies of one chat model answer, read each other and revise over rounds, with measured gains on reasoning and factuality.
• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation: a framework of conversable agents, with auto-reply, a group chat manager and several example applications.
• Civil UAV applications survey: drone survey with a decentralised patrol rule and no experiments; not an LLM team.
• Artificial Leviathan: Exploring Social Evolution of LLM Agents Through the Lens of Hobbesian Social Contract Theory: nine chat agents in a food-and-land world, with traits, memory and robbery or contracts; a simulation with no public code.
• Multi-Agent Causal Discovery Using Large Language Models: two debate stages that choose a statistical causal-discovery algorithm, then refine the graph with metadata.
• Empowering Biomedical Discovery with AI Agents: perspective with four autonomy levels for biomedical agents, five collaboration schemes and a module list; no experiments.
• Agentic Large Language Models, a survey: survey that sorts agentic LLMs into reasoning, acting and interacting; role-based teams and societies sit under interacting, with no experiments.
• From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review: survey of agent benchmarks, frameworks and the MCP, ACP and A2A protocols; no experiments.
• Review of trust, risk and security for LLM multi-agent systems: review of trust, risk and security for LLM multi-agent systems, with a threat list and an unvalidated synergy score.
• Two Heads Are Better Than One: Collaborative LLM Embodied Agents for Human-Robot Interaction: coder and reviewer agents, with an optional planner, write robot code from spoken requests; a small study with no significant observer difference.
• Toward Scalable Multirobot Control: Fast Policy Learning in Distributed MPC: learned actor and critic per robot replace a per-step solver in distributed MPC; not a language-model team.
• AI Scientists for Building Virtual-Cell Models: expert agents debate a perturbation-model design under a critic, then a coding agent writes and repairs the training code; author-reported results on six datasets.
• SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models: four prompted stages turn one instruction into code for a robot team in AI2-THOR; no agent dialogue, and no retry between stages.
• DREAMS: Density Functional Theory Based Research Engine for Agentic Materials Simulation: planner and DFT, HPC and convergence workers with a provenance store, a per-call guard and a report judge; author-reported DFT results.