Multi-Agent Causal Discovery Using Large Language Models

In plain words

Working out which variables cause which is hard. Statistical methods look only at the numbers, so they can miss the right answer when their assumptions fail. A single language model can read the variable names and descriptions, but it may simply repeat what it memorised, and nothing checks it.

MAC uses several language-model agents that argue instead of one agent that answers. The first stage has agents choose a standard causal-discovery algorithm that suits the data, then runs that algorithm to get a first graph. The second stage turns the graph into plain sentences and adds them to the background notes about the variables. A debate follows: one agent proposes a graph, a second proposes a plausible alternative, and a judge picks the better one. The authors test it on five benchmark datasets and report that it does best on average. These are the authors' own results.

TL;DR

MAC is a two-stage, multi-agent LLM framework for causal discovery, evaluated on five benchmark datasets §4.1. Stage 1 (DCM) debates which statistical algorithm to run on the data and executes it §3.4 alg. 2. Stage 2 (MDM) refines the resulting graph using metadata in an Affirmative–Negative–Judge debate, and Meta Fusion passes the data graph between them as text §3.3 §3.5 alg. 3. The authors report the best aggregate performance; with Gemini-2.0-Flash it comes first on 10 of 15 evaluation points §4.2. No official code was found, so the prompts (only in figures) must be written from the text.

Problem

Causal discovery asks which variables have a direct causal effect on which others, given observational data and optionally metadata §3.1. Statistical causal discovery (SCD) methods use only the data. Their validity depends on assumptions such as linearity, Gaussianity, hidden confounders and sample size, which differ between problems, and no single SCD algorithm is best across datasets §3.4 §1.

LLM-based methods use metadata but usually treat the model as one agent. Such a model can repeat memorised or biased associations, which the paper calls "causal parrots", and a single pass gives no way to challenge a wrong but confident answer §3.2 §3.3. Combining the two is not direct, because the data-derived graph is a numeric adjacency matrix while metadata and agents work in natural language §3.5.

Method

Inputs and outputs. The data is an observation matrix O with rows as observations and columns as variables. The variables are V, the metadata is an optional text I, and a small subset S of rows is used only for cheap reasoning. The final graph is always estimated on the full O §3.1.

1. Meta-Debate Module (MDM) alg. 1 §3.3. One operator with three agents. The Affirmative agent proposes a candidate output. The Negative agent proposes a plausible alternative under the same evidence, aiming at disputed decisions. The Judge scores the disagreements against evidence-derived criteria and returns the winner app. A.1. The operator runs in two modes. In graph mode the context is (V, I) and the output is a DAG over V. In algorithm mode the context is (V, I, S) and the output is one algorithm from a pool 𝒜 §3.3 alg. 1.
2. Causal principles given to the agents app. A.1. Under causal sufficiency and no cycles, two nodes are directly connected if and only if no set of other nodes makes them conditionally independent. A node B is a collider in A → B ← C if A and C are marginally independent but become dependent when B is conditioned on. Other edge directions are set by acyclicity and by a causal order taken from the metadata.
3. Debate-Coding Module (DCM) alg. 2 §3.4. Step (i): MDM runs in algorithm mode, with S, to choose an SCD algorithm A from 𝒜. Step (ii): a Causal Coding Executor writes, runs and debugs code that applies A to the full O with an existing causal-discovery library (causal-learn is named), and returns the data graph G_C §3.4.
4. Meta Fusion §3.5. Converts the adjacency matrix of G_C into human-readable directed edges and implied orderings. These become causal metadata I_c, appended to I. The example is a 4-variable binary matrix over Hot, Sales, Swim and Attack, which becomes Hot→Sales, Hot→Swim and Swim→Attack §3.5.
5. Refinement alg. 3 §3.5. MDM runs in graph mode on (V, I ∪ I_c) with S empty, and returns the final graph G_D.
6. Rounds §4.3. The number of Affirmative–Negative exchanges in MDM is varied from one to five in the experiments. A single round still runs the full Affirmative–Negative–Judge exchange §4.3.

The overall procedure is in pseudocode below. Its three functions follow the paper's algorithms 1 to 3.

# MAC, end to end  [alg. 3] [§3.5]
# Input:  variables; data (N, d); metadata text; subset; algorithm pool
# Output: final directed graph (n, n) over the variables

def mac(variables, data, metadata, subset, algorithm_pool):
    # Stage 1: data-grounded initial graph  [alg. 2] [§3.4]
    initial_graph = debate_coding_module(
        variables, data, metadata, subset, algorithm_pool)
    # Turn the data graph into text constraints  [§3.5]
    causal_constraints = meta_fusion(initial_graph)
    # Stage 2: refine with metadata plus constraints  [alg. 3] [§3.5]
    final_graph = meta_debate_module(
        variables,
        metadata + causal_constraints,
        subset=None,
        output_type="graph",
    )
    return final_graph

# Debate-Coding Module  [alg. 2] [§3.4]
# Input:  variables; full data (N, d); metadata; subset; algorithm pool
# Output: initial graph (n, n), built from data only

def debate_coding_module(variables, data, metadata, subset, pool):
    # Choose an SCD algorithm by debate over (V, I, S)  [alg. 2] [§3.4]
    algorithm = meta_debate_module(
        variables, metadata, subset, output_type="alg", pool=pool)
    # Run that algorithm on the full data via a coding agent  [§3.4]
    initial_graph = causal_coding_executor(algorithm, data)
    return initial_graph

# Meta-Debate Module, one operator for two output types  [alg. 1] [§3.3]
# Input:  variables; metadata; optional subset; output type; pool
# Output: a DAG over the variables, or one algorithm from the pool

def meta_debate_module(variables, metadata, subset, output_type, pool=None):
    # Context: graph mode uses (V, I); algorithm mode adds S  [alg. 1]
    if output_type == "graph":
        context = (variables, metadata)
    else:
        context = (variables, metadata, subset)
    # Affirmative proposes a candidate  [alg. 1] [§3.3]
    affirmative = propose(context, pool)
    # Negative proposes a plausible alternative to the candidate  [alg. 1]
    negative = propose_alternative(context, pool, affirmative)
    # Judge scores the disagreements and returns the winner  [alg. 1]
    return judge(context, pool, affirmative, negative)

Hyperparameters

Results

Author-reported. Setup: five benchmark datasets, three backbone LLMs, ten seeds, against nine baselines (five statistical, four LLM-based) §4.1 app. D.

Headline numbers:

• Averaged over datasets and backbones, MAC is first on every metric. Its mean rank is 1.4 for F1, 1.6 for NHD and 2.0 for SHD. The next best are LLM-based and statistical baselines fig. 2 §4.2.
• Across the 15 evaluation points (five datasets × three metrics), MAC with Gemini-2.0-Flash is first on 10 and second on 4. MAC with DeepSeek-R1 is first on 9 and second on 5. MAC with GPT-4o is first on 7 and second on 7 §4.2.
• Earthquake with Gemini-2.0-Flash: SHD 0, NHD 0, F1 1.00 §4.2 tab. 2. Exact Search has the same values on Earthquake for all three LLMs tab. 2.
• The paper says GES has the lowest SHD on Auto and Cancer, LiNGAM on Survey, and Exact Search and ReAct tie on NHD for Child §4.2.

Ablations §4.3 tab. 3 tab. 4 tab. 5:

• MAC beats DCM-only and MDM-only in most cases. On Earthquake, MDM alone is best. MDM alone beats DCM alone in general §4.3.
• Rounds §4.3 fig. 4: performance is stable after one round for most datasets. On Survey and Auto it falls as rounds increase.

Single-agent variant (one MDM agent, no Negative agent or Judge) §4.4 tab. 6:

• The single-agent variant is "competitive" on most datasets at about half the cost. Full MAC is much better on harder graphs: Survey with GPT-4o F1 0.36 → 0.79, Auto with Gemini F1 0.06 → 0.61, and Child with DeepSeek-R1 F1 0.49 → 0.65. Earthquake and Cancer are comparable §4.4.

Cost §4.4 fig. 5: the full system uses 16× to 25× more tokens than standard baselines. Coding and debugging account for up to half of that. GPT-4o is the most token-efficient backbone. The dollar figures are in fig. 5 only and are not given in the text.

Overall table tab. 2, mean ± std over 10 seeds. The source's bold (best) and underline (second) marks are omitted; read the ranks from the source table if needed.

Single-agent ablation tab. 6, mean ± std over 10 seeds:

Paper vs code

No official code.

Searched, in order, with no official repository found:

1. Links in the paper, abstract and arXiv page (arxiv.org/abs/2407.15073): none. The paper's own source has no repository link. The only URLs are AutoGen, causal-learn and model-provider docs §4.1.
2. GitHub repository search (via the GitHub API): "multi-agent causal discovery", "MAC causal debate", "causal discovery LLM debate", "causal discovery autogen", "Meta-Debate causal" and "Hao Duong Le causal". Results were unrelated (for example a multi-agent drug-discovery debate app and a federated causal-discovery experiment repo, neither by the authors).
3. The first author's GitHub repositories: unrelated (a campus RAG assistant and an e-commerce agent demo).

Pitfalls

• Prompts are not in the text. The full agent prompts (Role, Goal, Causal Principles, Procedure, Output Format) are only in fig. 6 and fig. 7, which are images. The text gives the intent of each part but not the wording, and there is no code to check against. Implementers must write their own prompts and output parser app. A.1 app. A.2.
• Judge scoring is underspecified. The live text says only that the Judge uses "a quantifiable metric" over the disagreements in skeleton, colliders and edge directions app. A.1. The source has commented-out text describing a point-per-difference rule. That text is not part of the paper and should not be used as the specification.
• Algorithm pool. Only PC, FCI and GES are named as examples app. A.2. The text does not say which algorithms form the pool 𝒜, nor which algorithm each dataset selected app. E. The statistical baselines (PC, FCI, Exact Search, GES, DirectLiNGAM) are a separate list §4.1.
• Subset S. Its size is not stated ("a few sampled rows", §3.1). The algorithm-selection prompt receives S, so the choice of S changes what the Negative agent can argue.
• Debate rounds. The default number of rounds in the main tables is not stated. R3 tests one to five rounds and finds one sufficient §4.3.
• Cyclic or invalid outputs. The method targets DAGs §6. The text does not say what happens when an LLM returns a cycle, a bad edge format or a non-parsable answer.
• Meta Fusion format. The text gives an example with direct edges and "implied orderings" §3.5, but does not say how orderings are derived from the graph (for example transitive closure) or the exact text format.
• SHD counting. The definition sums over ordered pairs i ≠ j of directed adjacency entries app. B. Taken literally, a reversed edge counts as 2 differences. The paper does not say whether it uses that convention or counts a reversal once.
• NHD on directed graphs. NHD is defined for undirected graphs, normalised by the number of node pairs app. B. The text does not say how the directed output is converted to an undirected comparison.
• The headline Earthquake result is shared. Exact Search also has SHD 0, NHD 0 and F1 1.00 on Earthquake for every backbone tab. 2. The abstract's "perfect reconstruction" is true for MAC with Gemini, but it is not unique to MAC.
• Win counts. The 10/4, 9/5 and 7/7 counts are the authors' own §4.2. The text does not define how ties are counted, and several cells in tab. 2 are tied or near-tied.
• Determinism claim. The paper says temperature 0 makes outputs deterministic §4.1, but it still reports std over 10 seeds for LLM methods tab. 2. Plan to repeat runs.
• Small graphs. The benchmarks are 5 to 20 nodes §6. The paper says MAC's metadata results may partly reflect memorised knowledge of canonical networks §6. The DCM stage is meant to reduce that risk, but this is not tested on novel graphs §6.
• Costs. The 16× to 25× token figure and "about half" single-agent cost are in the text. The dollar values are only in fig. 5, so no USD figures are given here.
• Source version. This card uses arXiv v4 (updated 2026-05-26), as recorded in the arXiv metadata. Earlier versions may differ.
• Figure and table numbers. These were counted from the source in order of appearance (Table 1 is the comparison table in §1; Figure 1 is the MAC overview in §2). They were not checked against the compiled PDF.

Lineage

• Builds on: multi-agent debate, which the paper cites as Du et al. (2023) for debate improving reasoning and factuality §3.3; the causal-learn library §3.4; AutoGen §4.1.
• Compared against: statistical baselines PC, FCI, Exact Search, GES and DirectLiNGAM; LLM baselines Zero-shot prompting, ReAct, LLM-SCD (Ban et al., 2023) and ILS-CSL (Takayama et al., 2024) §4.1. None of these has a card in this wiki.
• Superseded by: not stated.