---
kind: concept
slug: causal-discovery
title: Causal discovery
summary: >-
  Learning which variables have a direct causal effect on which others, from observational data and optional metadata. One source so far: a multi-agent LLM debate that picks a statistical algorithm, runs it, then refines the graph with metadata. Author-reported wins on five benchmarks; 16 to 25 times the tokens of standard baselines. Trade-off: accuracy against token cost.
tags: [causality, multi-agent, llm-agents]
papers: ["2407.15073"]
related: [causal-inference, multi-agent-systems, llm-as-a-judge]
status: draft
updated_at: 2026-10-19
---

## Overview

This page covers only what the wiki's sources say about causal discovery. There
is one source: a paper that proposes Multi-Agent Causal discovery (MAC) with
large language models and compares it with statistical and LLM baselines
[arxiv:2407.15073 §1] [arxiv:2407.15073 §4.1]. The task is to decide which
variables have a direct causal effect on which others, given observational data
and, optionally, a text description of the variables [arxiv:2407.15073 §3.1].

Statistical methods use only the data. Their output is valid only when their
assumptions hold, such as linearity, Gaussian noise, no hidden confounders and
enough samples, and the paper finds that no single statistical algorithm is best
across its datasets [arxiv:2407.15073 §3.4]. Single-model LLM methods can read the
metadata, but the paper argues that one model may repeat memorised or biased
associations, which it calls causal parrots, and that a single pass has no way
to challenge a confident but wrong answer [arxiv:2407.15073 §3.2] [arxiv:2407.15073
§3.3]. MAC is an attempt to combine the two: the data-derived graph is a numeric
adjacency matrix, while metadata and agents work in natural language, so the
graph has to be converted before the agents can use it [arxiv:2407.15073 §3.5].

The core idea is debate. Two LLM agents argue over a candidate output, and a
third agent judges the disagreements against criteria derived from the evidence
[arxiv:2407.15073 §3.3] [arxiv:2407.15073 app. A.1]. The same debate operator
serves two jobs: choosing an algorithm and refining a graph
[arxiv:2407.15073 alg. 1].

## Variants

### Single-model LLM graph proposal

The paper's single-agent baselines ask one model to propose the graph. Zero-shot
prompting and ReAct are named as LLM baselines [arxiv:2407.15073 §4.1]. In the
ablation, a one-agent MAC (one MDM agent, no Negative agent and no Judge) is
"competitive" on most datasets at about half the cost, but is much weaker on
harder graphs [arxiv:2407.15073 §4.4] [arxiv:2407.15073 tab. 6]. Card:
[Multi-Agent Causal Discovery Using Large Language Models](/papers/2407.15073).

### Two-stage debate: data first, then metadata (MAC)

Stage 1 runs a debate that chooses one statistical algorithm from a pool. A
coding agent then writes and runs code that applies it to the full data and
returns a graph [arxiv:2407.15073 §3.4] [arxiv:2407.15073 alg. 2]. Stage 2
turns that graph into plain-text constraints, adds them to the metadata, and
runs a second debate that returns the final graph [arxiv:2407.15073 §3.5]
[arxiv:2407.15073 alg. 3]. The paper names PC, FCI and GES as examples of
algorithms in the pool, and does not state the full pool [arxiv:2407.15073
app. A.2]. Card: [Multi-Agent Causal Discovery Using Large Language Models](/papers/2407.15073).

## When to use which

The only comparison in the wiki is the paper's own, so the table below uses the
constraints that the paper's results bear on. All numbers are author-reported.

| Situation | Use | Why |
|---|---|---|
| Variable names and descriptions are available and the variables are few | Two-stage debate (MAC) | On average MAC is first on all three metrics across the five datasets and three backbones [arxiv:2407.15073 §4.2] [arxiv:2407.15073 fig. 2]. |
| Token budget is tight | Single-model proposal, or one-agent MAC | The full system uses 16 to 25 times the tokens of standard baselines, and the single-agent variant costs about half [arxiv:2407.15073 §4.4]. The single-agent variant loses most on harder graphs [arxiv:2407.15073 tab. 6]. |
| Graph is small and close to a canonical network (for example Cancer or Earthquake) | Either; expect similar results | On Cancer and Earthquake the single-agent variant is comparable to full MAC [arxiv:2407.15073 §4.4]. Exact Search also reaches zero SHD on Earthquake for every backbone [arxiv:2407.15073 tab. 2]. |
| Data are the only trustworthy input and the variables have no usable description | Run statistical methods, and treat LLM output with caution | The paper's MAC is built on metadata. Its ablation results without metadata are not stated in the wiki's sources. |
| Model access is limited to one backbone | Check that the chosen backbone is among those tested | Results differ by backbone. DeepSeek-R1 has the best F1 on Child, Cancer and Survey among the tested models [arxiv:2407.15073 §4.2]. |

## Implementation sketch

The paper gives its procedure as three functions. The sketch below keeps their
structure; it is not the authors' code, because none is published
[arxiv:2407.15073 §Paper vs code]. Agent prompts and the output parser must be
written from scratch [arxiv:2407.15073 app. A.1].

```pseudocode
# MAC, end to end  [alg. 3] [§3.5]
# Input:  variables; data (N, d); metadata text; algorithm pool
# Output: final directed graph (n, n) over the variables

def mac(variables, data, metadata, pool):
    # Stage 1: data-grounded graph  [alg. 2] [§3.4]
    graph_data = debate_coding_module(variables, data, metadata, pool)
    # Turn the adjacency matrix into text constraints  [§3.5]
    constraints = meta_fusion(graph_data)
    # Stage 2: refine with metadata plus constraints  [alg. 3] [§3.5]
    return meta_debate_module(variables, metadata + constraints, "graph")
```

```pseudocode
# Debate-Coding Module  [alg. 2] [§3.4]
# Input:  variables; full data; metadata; algorithm pool
# Output: graph built from the data alone

def debate_coding_module(variables, data, metadata, pool):
    # Debate over a small row subset S picks the algorithm  [§3.4]
    algorithm = meta_debate_module(variables, metadata, "algorithm", pool)
    # Code agent applies it to all rows, with debugging  [§3.4]
    return causal_coding_executor(algorithm, data)
```

```pseudocode
# Meta-Debate Module: one operator, two output types  [alg. 1] [§3.3]
# Output: a DAG over the variables, or one algorithm from the pool

def meta_debate_module(variables, metadata, output_type, pool=None):
    # Affirmative proposes a candidate  [alg. 1]
    candidate = propose(variables, metadata, output_type, pool)
    # Negative argues for a plausible alternative  [alg. 1] [§3.3]
    alternative = counter(variables, metadata, output_type, pool, candidate)
    # Judge scores the disagreements and returns the winner  [app. A.1]
    return judge(variables, metadata, candidate, alternative)
```

Points where variants differ: the subset size S is not stated [arxiv:2407.15073
§3.1]; the number of debate rounds in the main tables is not stated, and one
round is said to suffice [arxiv:2407.15073 §4.3]; the judge's scoring formula
is not stated [arxiv:2407.15073 app. A.1]; and the handling of cyclic or
malformed LLM outputs is not described [arxiv:2407.15073 §6].

## Contradictions

Only one source, so there is no disagreement between papers. Within the paper,
the results are not uniform. The authors' headline Earthquake result (SHD 0,
F1 1.00 with Gemini-2.0-Flash) is matched by Exact Search, a statistical method,
on the same dataset [arxiv:2407.15073 §4.2] [arxiv:2407.15073 tab. 2]. The
paper's ranks also show ties or near-ties in several cells [arxiv:2407.15073
§4.2] [arxiv:2407.15073 tab. 2]. The paper says temperature 0 gives
deterministic outputs, yet it reports standard deviations over ten seeds for the
LLM methods [arxiv:2407.15073 §4.1] [arxiv:2407.15073 tab. 2].

## Papers

- [Multi-Agent Causal Discovery Using Large Language Models](/papers/2407.15073): a two-stage debate framework that picks a statistical algorithm, runs it, and refines the graph with metadata; author-reported gains on five benchmarks.