Causal discovery

Overview

This page covers only what the wiki's sources say about causal discovery. There is one source: a paper that proposes Multi-Agent Causal discovery (MAC) with large language models and compares it with statistical and LLM baselines 2407.15073 §1 2407.15073 §4.1. The task is to decide which variables have a direct causal effect on which others, given observational data and, optionally, a text description of the variables 2407.15073 §3.1.

Statistical methods use only the data. Their output is valid only when their assumptions hold, such as linearity, Gaussian noise, no hidden confounders and enough samples, and the paper finds that no single statistical algorithm is best across its datasets 2407.15073 §3.4. Single-model LLM methods can read the metadata, but the paper argues that one model may repeat memorised or biased associations, which it calls causal parrots, and that a single pass has no way to challenge a confident but wrong answer 2407.15073 §3.2 [arxiv:2407.15073 §3.3]. MAC is an attempt to combine the two: the data-derived graph is a numeric adjacency matrix, while metadata and agents work in natural language, so the graph has to be converted before the agents can use it 2407.15073 §3.5.

The core idea is debate. Two LLM agents argue over a candidate output, and a third agent judges the disagreements against criteria derived from the evidence 2407.15073 §3.3 2407.15073 app. A.1. The same debate operator serves two jobs: choosing an algorithm and refining a graph 2407.15073 alg. 1.

Variants

Single-model LLM graph proposal

The paper's single-agent baselines ask one model to propose the graph. Zero-shot prompting and ReAct are named as LLM baselines 2407.15073 §4.1. In the ablation, a one-agent MAC (one MDM agent, no Negative agent and no Judge) is "competitive" on most datasets at about half the cost, but is much weaker on harder graphs 2407.15073 §4.4 2407.15073 tab. 6. Card: Multi-Agent Causal Discovery Using Large Language Models.

Two-stage debate: data first, then metadata (MAC)

Stage 1 runs a debate that chooses one statistical algorithm from a pool. A coding agent then writes and runs code that applies it to the full data and returns a graph 2407.15073 §3.4 2407.15073 alg. 2. Stage 2 turns that graph into plain-text constraints, adds them to the metadata, and runs a second debate that returns the final graph 2407.15073 §3.5 2407.15073 alg. 3. The paper names PC, FCI and GES as examples of algorithms in the pool, and does not state the full pool [arxiv:2407.15073 app. A.2]. Card: Multi-Agent Causal Discovery Using Large Language Models.

When to use which

The only comparison in the wiki is the paper's own, so the table below uses the constraints that the paper's results bear on. All numbers are author-reported.

Implementation sketch

The paper gives its procedure as three functions. The sketch below keeps their structure; it is not the authors' code, because none is published 2407.15073 §Paper vs code. Agent prompts and the output parser must be written from scratch 2407.15073 app. A.1.

# MAC, end to end  [alg. 3] [§3.5]
# Input:  variables; data (N, d); metadata text; algorithm pool
# Output: final directed graph (n, n) over the variables

def mac(variables, data, metadata, pool):
    # Stage 1: data-grounded graph  [alg. 2] [§3.4]
    graph_data = debate_coding_module(variables, data, metadata, pool)
    # Turn the adjacency matrix into text constraints  [§3.5]
    constraints = meta_fusion(graph_data)
    # Stage 2: refine with metadata plus constraints  [alg. 3] [§3.5]
    return meta_debate_module(variables, metadata + constraints, "graph")

# Debate-Coding Module  [alg. 2] [§3.4]
# Input:  variables; full data; metadata; algorithm pool
# Output: graph built from the data alone

def debate_coding_module(variables, data, metadata, pool):
    # Debate over a small row subset S picks the algorithm  [§3.4]
    algorithm = meta_debate_module(variables, metadata, "algorithm", pool)
    # Code agent applies it to all rows, with debugging  [§3.4]
    return causal_coding_executor(algorithm, data)

# Meta-Debate Module: one operator, two output types  [alg. 1] [§3.3]
# Output: a DAG over the variables, or one algorithm from the pool

def meta_debate_module(variables, metadata, output_type, pool=None):
    # Affirmative proposes a candidate  [alg. 1]
    candidate = propose(variables, metadata, output_type, pool)
    # Negative argues for a plausible alternative  [alg. 1] [§3.3]
    alternative = counter(variables, metadata, output_type, pool, candidate)
    # Judge scores the disagreements and returns the winner  [app. A.1]
    return judge(variables, metadata, candidate, alternative)

Points where variants differ: the subset size S is not stated [arxiv:2407.15073 §3.1]; the number of debate rounds in the main tables is not stated, and one round is said to suffice 2407.15073 §4.3; the judge's scoring formula is not stated 2407.15073 app. A.1; and the handling of cyclic or malformed LLM outputs is not described 2407.15073 §6.

Contradictions

Only one source, so there is no disagreement between papers. Within the paper, the results are not uniform. The authors' headline Earthquake result (SHD 0, F1 1.00 with Gemini-2.0-Flash) is matched by Exact Search, a statistical method, on the same dataset 2407.15073 §4.2 2407.15073 tab. 2. The paper's ranks also show ties or near-ties in several cells [arxiv:2407.15073 §4.2] 2407.15073 tab. 2. The paper says temperature 0 gives deterministic outputs, yet it reports standard deviations over ten seeds for the LLM methods 2407.15073 §4.1 2407.15073 tab. 2.

Papers

• Multi-Agent Causal Discovery Using Large Language Models: a two-stage debate framework that picks a statistical algorithm, runs it, and refines the graph with metadata; author-reported gains on five benchmarks.