Improving Factuality and Reasoning in Language Models through Multiagent Debate

In plain words

Language models sometimes state wrong facts with full confidence, and they can slip on multi-step reasoning. A common fix is to check one answer on its own, or to sample several answers and keep the most common one.

This paper's idea is to let several copies of the same model argue. Each copy answers the question first. Then each one reads the other copies' answers and is asked to update its own. This repeats for a few rounds. The copies usually settle on a shared answer, and the shared answer is often better than any single copy's first try. It is like colleagues checking each other's working.

The method only needs plain text prompts, not access to the model's internals, so it works with any chat model. In the paper's own tests it helped on maths, chess move choice and factual biographies. The price is several model calls for every question.

TL;DR

• Several instances of one chat model answer a question, then revise their answers after reading the others' answers, for a few rounds §2.1.
• Author-reported gains over single answers, self-reflection and majority voting on arithmetic, GSM8K and chess (pawn score), with 3 agents and 2 rounds tab. 1.
• Author-reported gains in factuality on biographies, MMLU and chess move validity tab. 2. Debate is the only method that beats the baselines on all three.
• Use it when a single model answer is unreliable and several calls per question are affordable. Expect more cost, a dependence on the wording of the prompts, and confident consensus on wrong answers §5.

Problem

Large language models hallucinate facts and make mistakes in reasoning, and existing remedies such as chain-of-thought prompting, verification and self-consistency act on one model instance at a time §1. The authors want a complementary method that needs only black-box access to generated text, with the same procedure and no model internals such as likelihoods or gradients §1. They also build a benchmark of factual biographies of computer scientists for the factuality part §1 §3.2.

Method

The method has a generation step and a debate step, repeated over rounds §2.1 fig. 2.

1. Each of N agents, which are copies of the same model, answers the question independently §2.1.
2. Each agent then receives the other agents' latest answers, concatenated as context, together with an instruction to check them and produce an updated answer. Each agent therefore both verifies the others' answers and refines its own §2.1.
3. Step 2 repeats for several rounds. The debate prompt is what changes between rounds. The starting prompt is the task prompt §2.1 fig. 15.
4. The final answer is the consensus that the agents reach. The paper does not define one aggregation rule in the text. The code takes a majority vote over the agents' final answers for arithmetic and GSM8K, and scores each agent's biography separately gen_math.py eval_gsm.py eval_conversation.py.

# Multiagent debate, as the code runs it  [§2.1] [code: gsm/gen_gsm.py#L47-L61]
# Input:  question; num_agents; num_rounds (round 0 is the independent
#         answers, so num_rounds = 2 means one debate update)
# Output: each agent's final answer

def debate(question, num_agents, num_rounds):
    # Each agent keeps its own transcript, starting with the question  [§2.1]
    histories = [[question] for _ in range(num_agents)]
    # Answers from the previous round, one per agent  [§2.1]
    previous_answers = [None] * num_agents
    for round_index in range(num_rounds):
        current_answers = []
        for agent_index in range(num_agents):
            if round_index > 0:
                # Show the other agents' previous answers and ask for an update
                # [§2.1] [code: math/gen_math.py#L40-L55]
                others = [
                    previous_answers[other]
                    for other in range(num_agents)
                    if other != agent_index
                ]
                histories[agent_index].append(
                    debate_prompt(others, question)
                )
            # One chat call per agent per round  [code: gsm/gen_gsm.py#L55-L58]
            answer = chat_model(histories[agent_index])
            histories[agent_index].append(answer)
            current_answers.append(answer)
        previous_answers = current_answers
    return previous_answers

# Final answer by majority vote  [code: math/gen_math.py#L73-L83]
# Ties go to the earliest agent that reaches the top count.
def final_answer(answers):
    best_answer = answers[0]
    best_count = 0
    for candidate in answers:
        count = answers.count(candidate)
        if count > best_count:
            best_answer = candidate
            best_count = count
    return best_answer

The paper reports that agents are fairly agreeable, so the wording of the debate prompt controls how long a debate runs §2.2. Two prompts, one short and one long, differ only in how much they invite the agents to stick with their own answer fig. 3. The long version gives slower convergence and a better final consensus on the correct answer §3.3 fig. 12. The short version gives faster consensus fig. 14.

Two combinations are reported. First, debate can run on top of zero-shot chain-of-thought prompting, which the authors apply in their evaluations §2.1 fig. 6. The exact trigger phrase is not stated. Second, for more than four agents, the responses of the other agents can first be summarised by a model before being shown to each agent §3.3 fig. 13.

Each task has its own starting and debate prompts, listed in fig. 15. The abstract and the introduction describe the procedure and prompts as identical across tasks, but the prompt text differs per task fig. 15.

The factuality metric for biographies works as follows. Ground-truth bullets for each person come from Wikipedia. Each generated bullet biography is checked against each ground-truth bullet by asking the same chat model whether the biography is consistent with that fact, and the answer is yes, no or uncertain app. A.2 eval_conversation.py.

The paper has no numbered equations or algorithm boxes. Its procedure is in prose, in fig. 2, and in the prompts fig. 3 fig. 15.

Hyperparameters

Results

Author-reported throughout. Each cell is over the subset in app. A.2: 100 items for arithmetic, GSM8K, MMLU and chess validity, and 300 games for chess. Every row uses zero-shot prompting with the same chat model §3.1.

Reasoning tab. 1. Values are mean ± the paper's spread. The paper does not define the ± (see Paper vs code).

Factuality tab. 2. The majority vote is omitted here, because the authors say individual biographies are not easy to compare §3.2.

The table gives no unit for the chess column tab. 2.

Other results from the text and figures:

• Arithmetic accuracy rises with the number of agents, from 2 agents up §3.3 fig. 10. With 3 agents, accuracy rises with the number of rounds, and more than 4 rounds gives about the same result as 4 §3.3 fig. 10.
• Long debate prompts converge more slowly to the correct answer and reach a better final consensus §3.3 fig. 12. Short prompts reach consensus faster fig. 14.
• Debate with zero-shot chain-of-thought improves GSM8K over debate without it fig. 6.
• Summarising peer answers improves debate performance in the authors' experiments §3.3 fig. 13.

Paper vs code

The official repository is composable-models/llm_multiagent_debate, at commit 9846749350eb917ae5bfaaff4c645fc705b8d3af. The paper's abstract links the project website, and that page links this repository [README.md#L3]. The code has no LICENSE file, so code_license is not stated. This card uses the code only for pseudocode and line citations, not verbatim.

• No chess code. The repository has folders for math, gsm, biography and mmlu only [README.md#L20-L23]. The chess reasoning and chess validity results have no code here.
• Round counting. The paper calls the debate setting "two rounds of debate" tab. 1. The code loops over rounds starting from the initial answers, so rounds = 2 is one initial answer plus one debate update gen_gsm.py. For arithmetic, the same loop is run with rounds = 3 in the default __main__ gen_math.py.
• Default agent and round counts for arithmetic. The paper's setting is 3 agents and 2 rounds tab. 1. The code's arithmetic __main__ uses 2 agents and 3 rounds gen_math.py. The code is probably a leftover default, not the paper's setting. The paper's values are the ones reported.
• Arithmetic operands. The paper says two-digit numbers in §3.1 and 0 to 30 in §A.2. The code draws randint(0, 30), which excludes 30 and allows repeats gen_math.py.
• Biography sample size. The paper uses 524 people §3.2. The code takes the first 40 of a shuffled list gen_conversation.py.
• Biography metric wording. §A.2 says the score is the percentage of ground truth bullets where the judge returns yes or no. The code averages only yes and no answers, so the score is the share of yes among yes and no answers eval_conversation.py. This is the only definition this card can implement.
• Biography judge prompt. §A.2 quotes a prompt without the sentence the code adds an instruction to check dates and locations precisely against the fact eval_conversation.py.
• Biography debate prompts. The paper gives one debate prompt fig. 15. The code uses two: a middle-round prompt that asks for an updated bio, and a final-round prompt that asks the agent to compare its bio with the others' gen_conversation.py.
• Reflection baseline. The paper has a separate reflection row tab. 1. The code has no reflection script. With one agent, the code's peer list is empty and it sends a verify-your-answer prompt instead gen_math.py. Treating this as the reflection baseline is an inference.
• Majority baseline. The paper has a separate majority-vote row tab. 1. The code has no such script. The majority function in the code is applied to the debate's final answers gen_math.py. Running the generator with one round would give independent samples, but that is an inference the paper does not state.
• Output file names do not match between generator and evaluator. The GSM generator writes gsm_3_2.json gen_gsm.py, but the evaluator reads gsm_debate_3_3.json eval_gsm.py. The biography generator writes biography_3_2.json gen_conversation.py, but its evaluator reads biography_1_2.json eval_conversation.py. The MMLU generator writes mmlu_3_2.json gen_mmlu.py, but the evaluator reads mmlu_personalities_3_2.json eval_mmlu.py. The persona prompt does not appear in the MMLU generator. Rename or pass the files explicitly.
• Model name. The text says "chatGPT" §3.1. The code and §A.2 use gpt-3.5-turbo-0301 gen_gsm.py.
• Identical prompts. The abstract and §1 say the procedure and prompts are the same for all tasks. The code and fig. 15 show different prompts for each task.
• Error scores in the GSM evaluator. If no agent gives a parseable answer, the evaluator returns 1, which counts the item as correct eval_gsm.py. See Pitfalls.
• Table spread. The paper does not define the ± in the tables. The evaluators print the standard error, std divided by √n eval_gsm.py. The paper may have used the same quantity, but this is an inference.

Pitfalls

• Count rounds carefully. The paper's "rounds" and the code's rounds differ by one if the paper counts only debate updates. Check which one you implement and label it. Cost follows from it: the default GSM setting (3 agents, rounds = 2) makes 6 model calls per question, because each agent answers once in each round gen_gsm.py.
• Context grows with every round. Each agent receives all other agents' full answers. The authors note that long debates strain the model, which focuses on the most recent parts, and that summarising can help §5 §3.3. The paper's limitations also say debate is more expensive than other prompting methods §5.
• Majority vote is the scoring rule, not a single answer. For arithmetic and GSM8K, the code scores the majority of the agents' final answers. Reporting one agent's answer will not match the table gen_math.py eval_gsm.py.
• Ties go to the first agent. The majority function keeps the first answer that reaches the highest count, so an all-different set returns agent 0's answer gen_math.py.
• Number extraction is fragile. The arithmetic parser replaces every comma with a dot before reading the last token that parses as a float. "1,000" becomes 1.0 gen_math.py. The GSM parser looks for a {...} or \boxed{...} pattern, then falls back to the last number in the text eval_gsm.py.
• Missing answers count as correct in the GSM evaluator. The fallback return of 1 for a missing answer means a run with no parseable outputs scores as correct eval_gsm.py. Check the number of unparsed items before quoting an accuracy.
• The yes/no parser matches substrings. It checks for "uncertain", then "yes", then "no", as substrings of the whole reply. "no" matches "not", "cannot" and "know" eval_conversation.py. Parse the first word or match whole words.
• The biography judge is the same family as the generator. The authors use chatGPT to judge chatGPT's output, so the judge and the generator may share blind spots. The biography metric checks consistency with the ground-truth bullets only. Facts outside those bullets are not checked §A.2.
• Silent data drops. The biography generator stops a debate once a reply has only one bullet, which the code reads as "the model does not know this person" gen_conversation.py. The evaluator skips one-bullet replies shorter than 400 characters eval_conversation.py.
• Confident consensus on wrong answers. The paper reports that debates often converge on one answer, and that the answer can still be wrong, with the models confidently affirming it §5. Debate does not measure whether the consensus is right.
• Reflection is not uniformly helpful. The text says reflection "generally gives a modest boost" §3.1. In tab. 1, reflection is below the single agent on GSM8K (75.0 against 77.0). In tab. 2, it is below on MMLU (57.7 against 63.9).
• Reproducibility. The code seeds question sampling but not model sampling, and it sets no temperature. Two runs will differ, and the snapshot may not be available.
• Licence of the code is unknown. No LICENSE file means no permission to copy code. Implement from the pseudocode above and the cited lines only.

Lineage

• Builds on: "Society of Mind" as the framing §1. It compares against zero-shot chain-of-thought §1, self-reflection (Reflexion and Madaan et al. 2023) §1 and majority voting over samples §3.1. The related-work section names an earlier debate procedure for AI safety, by Irving et al. (2018), which uses human judges rather than model agents §4.
• Compared against: single-agent answers, reflection, and majority voting tab. 1 tab. 2.
• Superseded by: not stated in this wiki.