---
kind: concept
slug: causal-inference
title: Causal inference
summary: >-
  A 2021 review finds counterfactual methods ungrounded. A 2021 NLP survey lists text-effect assumptions. A 2022 survey gives the SCM core. A 2020 review says importance scores need assumptions. A 2022 GPT-3 study is mixed on interventions. A 2023 paper finds LLM subjects confounded. A 2024 trial separates access from use. A 2026 method replays agent steps. Trade-off: what-if answers vs grounding.
tags: [explainability, counterfactuals, causality]
papers: ["2007.04131", "2012.05876", "2011.15091", "2103.04244", "2109.00725", "2206.14576", "2206.15475", "2312.15524", "2407.15073", "2410.03017", "2606.08275"]
related: [counterfactual-explanation, explanation-evaluation, invariant-learning, algorithmic-fairness, causal-discovery, llm-evaluation, meta-learning, prompt-engineering, systematic-generalization, llm-agent, credit-assignment]
status: draft
updated_at: 2026-11-01
---

## Overview

This page covers only what the wiki's sources say about causal inference. There
are eleven sources: a systematic review of model-agnostic counterfactual
explanations [arxiv:2103.04244 §1], a survey of causal machine learning
[arxiv:2206.15475 §1], a survey of causal inference in NLP that treats text as a
confounder, an outcome or a treatment [arxiv:2109.00725 §1], a position paper on
neurosymbolic AI that discusses
causal hierarchy [arxiv:2012.05876 §5], a review of pitfalls in
model-agnostic interpretation that has one section on causal interpretation
[arxiv:2007.04131 §9], a study of GPT-3 on classic cognitive-psychology tasks,
one of which asks for interventions after passive observation
[arxiv:2206.14576 §Results / Causal reasoning], a randomised trial of an AI
tutoring tool that estimates effects from random assignment
[arxiv:2410.03017 §Methods], a 2024 preprint on causal discovery with several
LLM agents [arxiv:2407.15073 §3.1], a 2020 position paper that proposes causal
variables as inductive biases for deep learning [arxiv:2011.15091 §3.5], a
2023 paper on using LLMs as simulated subjects, which argues that blinded
simulations confound the treatment with invented covariates
[arxiv:2312.15524 §1, §2], and a 2026 paper that blames a failed LLM-agent run
on a step by intervening on that step [arxiv:2606.08275 §1]. None is a general
textbook on causal inference.

The review's central question is whether a counterfactual explanation can give
causability, which it defines as an understanding that rests on causal
relations rather than on feature correlations [arxiv:2103.04244 §1.2]. The
methods share one core. They look for a nearby input x' that receives a
different (target) prediction, by minimising a distance d(x, x') between x and
x' [arxiv:2103.04244 §4.1 eq. 4]. The review argues that such post-hoc
explainers capture correlations between features rather than the mechanism that
produced the prediction [arxiv:2103.04244 §1.2]. Its conclusion is that the
surveyed model-agnostic methods are not grounded in formal causal theory, so
they may not give causability [arxiv:2103.04244 §9.1] [arxiv:2103.04244 §10].

The causal ML survey supplies the formal vocabulary that the other sources
assume. Its core is a structural causal model (SCM), in which each variable is
a function of its parents and independent noise. In that setting, interventions
and counterfactuals become answerable quantities [arxiv:2206.15475 §2.3]. Its
point is that standard learning assumes test data follow the training
distribution, and that causal assumptions are what make models hold up when it
does not [arxiv:2206.15475 §1].

The neurosymbolic position paper takes a different route to causal questions.
It uses Pearl's three levels (association, intervention, counterfactual) and
argues that a symbolic description of a trained network allows interventions
and counterfactual questions [arxiv:2012.05876 §5]. The paper runs no
experiments, so this is an argument, not a measured result [arxiv:2012.05876
§6].

The pitfalls review makes a narrower point. A model-agnostic importance score
or effect plot describes what the model uses, and a predictive model exploits
associations, so a non-zero importance does not show that a feature is a cause,
an effect or unrelated [arxiv:2007.04131 §9]. The review says a partial
dependence plot can be read as an average causal effect only if the model
predicts well and the remaining variables form a valid adjustment set
[arxiv:2007.04131 §9]. In its linear example, all five inputs get a similar
non-zero effect, although three of them do not cause the outcome [arxiv:2007.04131
§9].

## Formal core

The causal ML survey defines the terms used throughout this page
[arxiv:2206.15475 §2]. The survey gives three layers of questions. The
associational layer asks what seeing a value tells us about another variable.
The interventional layer asks what happens when we set a variable. The
counterfactual layer asks what would have happened to the same case under a
different value [arxiv:2206.15475 §2.2.2] [arxiv:2206.15475 §2.2.3]. The
counterfactual layer sits at the top, because it subsumes the two below it
[arxiv:2206.15475 §2.2.3].

An SCM is a set of structural assignments with independent noise terms, and
each SCM induces a directed acyclic graph [arxiv:2206.15475 §2.3]. An
interventional query keeps the noise prior. A counterfactual query has three
steps: abduction (replace the noise prior by its posterior given the
observation), action (replace the chosen equation), and prediction (rerun the
equations on the same noise) [arxiv:2206.15475 §2.3.2]. The survey stresses
that using the prior for a counterfactual gives the wrong answer
[arxiv:2206.15475 §2.3] [arxiv:2206.15475 §2.3.2].

A causal quantity is identifiable if it can be rewritten as a purely
statistical one. Without hidden confounders and with a known graph, the
backdoor criterion gives an adjustment set [arxiv:2206.15475 §2.6]. The survey
does not detail the other graphical tests.

The NLP survey states the same core for a binary treatment. It defines the
counterfactual outcome of a unit, the average treatment effect (ATE) and the
conditional average treatment effect (CATE) [arxiv:2109.00725 eq. 1]
[arxiv:2109.00725 eq. 2]. Three assumptions make the effect identifiable:
conditional ignorability given the observed covariates [arxiv:2109.00725 eq. 4],
positivity, so that both treatment levels occur at every covariate value
[arxiv:2109.00725 eq. 5], and consistency [arxiv:2109.00725 eq. 6]. Under them,
the counterfactual mean can be written as an expectation of observable
conditional means [arxiv:2109.00725 eq. 7]. The survey adds that a causal DAG,
read with d-separation, decides which variables to condition on
[arxiv:2109.00725 §2.3].

## Variants

### Counterfactual explanations (model-agnostic)

The review groups its algorithms into seven families by mechanism
[arxiv:2103.04244 §5]. The paper's text says six categories in one place and
lists seven, so the list below follows the list.

#### Instance-centric search

Changes the input features directly, under a loss and a distance. These follow
the Lewis and Wachter counterfactual definition [arxiv:2103.04244 §4.1 §6.1].
Algorithms named: WatcherCF, Prototype, FACE, Weighted, TRUCE and DiCE. Card:
[Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Constraint-centric search

Frames the search as a constraint satisfaction problem. MACE encodes the model,
the distance and the constraints as one satisfiability formula, and asks a
solver for a counterfactual within a distance threshold ε [arxiv:2103.04244 §6.2
eq. 20]. GIC and Coherent counterfactuals are also in this family [arxiv:2103.04244
§6.2]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Genetic-centric search

Uses crossover and mutation to search for counterfactuals. CERTIFAI scores
robustness as the expected distance to the closest counterfactual [arxiv:2103.04244
§6.3 eq. 22]. MOCE optimises four objectives (prediction gap, Gower distance,
number of changed features, and likelihood) with NSGA-II [arxiv:2103.04244 §6.3
eq. 23]. LORE generates a neighbourhood with a genetic algorithm and fits a
decision tree to it [arxiv:2103.04244 §6.3]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Regression-centric search

Fits a local linear model to perturbed features, similar to LIME. LIME-C, SED-C
and CLEAR belong here [arxiv:2103.04244 §6.4]. CLEAR scores its regression by a
counterfactual fidelity error [arxiv:2103.04244 §6.4 eq. 24]. SED-C removes the
smallest set of words that flips a linear text classifier [arxiv:2103.04244
§6.4]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Game-theory-centric search

Uses Shapley values, similar to SHAP. SHAP-C ranks features by Kernel SHAP and
then runs the SED-C loop. SHAP-CC splits a contrastive "why P not Q" question
into two Shapley-value questions [arxiv:2103.04244 §6.5]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Case-based reasoning

Retrieves a stored explanation case (a query and its good counterfactual), maps
its features to the new query, and revises until the class flips [arxiv:2103.04244
§6.6]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Probabilistic search

Uses random walks, Markov sampling, VAEs or graphical models. The family
contains PRINCE, C-CHVAE, MC-BRP, ABELE, CRUDS and RECOURSE [arxiv:2103.04244
§6.7]. RECOURSE is the only method the review marks as causal. It defines
recourse actions as do-calculus interventions on a structural causal model that
may be only partly known [arxiv:2103.04244 §6.7 tab. 2]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244).

#### Symbolic extraction for interventional queries

Extract symbolic rules from a trained network, then answer causal questions over
those rules. This is not a counterfactual search. The paper treats it as one
route to explainability, since a rule set can give a proof trace where raw
activations cannot [arxiv:2012.05876 §5]. It proposes that rules carry
confidence values, so a system can signal when it does not know
[arxiv:2012.05876 §5]. It names rule extraction from very large networks as an
open challenge [arxiv:2012.05876 §6]. Card: [Neurosymbolic AI: The 3rd Wave](/papers/2012.05876).

### Causal machine learning (survey view)

The causal ML survey groups its methods into five problem areas. The entries
below are the ones that matter for implementation. Its method summaries point to
the primary papers, and it trains no models of its own [arxiv:2206.15475 §1]
[arxiv:2206.15475 §3]. Card: [Causal Machine Learning: A Survey and Open Problems](/papers/2206.15475).

#### Invariant risk minimization

Learns a feature map and one classifier that is optimal in every training
environment. Its bilevel form and a Lagrangian relaxation are given
[arxiv:2206.15475 §3.2]. The survey says it often performs no better than
empirical risk minimisation, and that it works best with linear causal relations
and enough environment heterogeneity [arxiv:2206.15475 §3.2].

#### Causal generative models

Learns assignments for a causal graph, so that interventional and counterfactual
samples can be drawn. DeepSCM learns each structural equation and inverts them
for abduction; the survey notes it has been applied to graphs of order five
[arxiv:2206.15475 §4.1.1]. Diff-SCM uses a diffusion model for abduction and
classifier guidance for an intervention on the label [arxiv:2206.15475 §4.2].

#### Causal feature attribution and recourse

Explanation methods that use a causal model rather than a predictive one. CXPlain
trains an explanation model on causal influence measures [arxiv:2206.15475
§5.1.1]. Causal Shapley values replace conditioning on observations with
conditioning by intervention [arxiv:2206.15475 §5.1.4]. Causal algorithmic
recourse seeks the minimal-cost set of interventions that flips a prediction,
with an SCM predicting their effects [arxiv:2206.15475 §5.2.2].

#### Counterfactual fairness

A predictor is counterfactually fair if, for an individual, its prediction has
the same distribution in the factual world and in the world where only the
protected attribute is changed, with exogenous variables held fixed
[arxiv:2206.15475 §6.1]. Path-specific variants count only effects along chosen
unfair paths [arxiv:2206.15475 §6.1.1]. See [algorithmic fairness](/concepts/algorithmic-fairness).

#### Causal reinforcement learning

The survey lists eight RL problem types where a causal view is claimed to help,
among them bandits, off-policy evaluation and credit assignment
[arxiv:2206.15475 §7]. It reports these as claims from the cited work, and
gives no comparison of its own.

### Causal inference for text (NLP survey)

This variant is a map of two uses of causal reasoning with text. The first is
estimating an effect when text plays a causal role. The second is making a text
model robust, fair or explainable. The survey is a review and reports no
experiments, so its claims are argued, not measured [arxiv:2109.00725 §1]
[arxiv:2109.00725 §5]. Card: [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](/papers/2109.00725).

Text can play three roles in an effect estimate [arxiv:2109.00725 §3]. As a
confounder, text carries variables that affect both treatment and outcome. The
assumption at risk is that every confounder is measured. The survey says this
cannot be tested from data and must rest on domain knowledge [arxiv:2109.00725
§3.1]. As an outcome, the measured quantity is often the output of an NLP model.
The survey recommends fitting that model on a sample separate from the one used
for the effect estimate [arxiv:2109.00725 §3.2]. As a treatment, the treatment is
a latent property of the text [arxiv:2109.00725 §3.3]. Text can nearly determine
its own treatment, so the overlap that adjustment needs (positivity) can fail in
high-dimensional text [arxiv:2109.00725 §3.1]. The survey also names
consistency failures, such as a hidden version of a treatment that is not
recorded, and interference between units [arxiv:2109.00725 §2.2] [arxiv:2109.00725 §3.2].

For robust and fair prediction [arxiv:2109.00725 §4], the survey discusses
counterfactual data augmentation (CDA). Manual edits are accurate but expensive.
Keyword swaps can miss labels and covariates, and automatic rewriting may add
new spurious correlations [arxiv:2109.00725 §4.1.1]. Distributional invariance
criteria need labels for the nuisance factor or several labelled environments.
The survey says they have been applied only to classification, and that many
environments may be needed [arxiv:2109.00725 §4.1.2]. It also says invariance
criteria are rarely applied to text in fairness work [arxiv:2109.00725 §4.2].
For interpretation, the survey casts an explanation as a comparison between an
input and a counterfactual input, including counterfactual activations inside
the model. It warns that attention weights can mislead and suggests intervening
on internal nodes instead [arxiv:2109.00725 §4.3].

The open problems include partial causal models, noisy proxies and structured
outputs [arxiv:2109.00725 §4.4]. Heterogeneous effects and benchmarks are also
open. The survey notes that no real-world text causal benchmark exists, because
counterfactuals are never observed [arxiv:2109.00725 §3.4]. The survey gives no
estimator for the identity in eq. 7. Regression adjustment, with one outcome
model per treatment level, is the standard reading, and it is not the paper's
method [arxiv:2109.00725 eq. 7].

### Causal discovery with LLM debate (MAC)

Causal discovery asks which variables have a direct causal effect on which,
given observational data and optionally metadata [arxiv:2407.15073 §3.1].
Statistical methods use the data alone, and their validity depends on
assumptions such as linearity, Gaussian noise, hidden confounders and sample
size. No single algorithm is best on every dataset [arxiv:2407.15073 §1]
[arxiv:2407.15073 §3.4].

MAC uses two debate stages. In the first, agents choose a statistical
algorithm from a pool, and a coding agent runs it on the full data to give an
initial graph [arxiv:2407.15073 §3.4] [arxiv:2407.15073 alg. 2]. In the second,
the graph is turned into plain-language constraints, added to the metadata, and
refined by another debate [arxiv:2407.15073 §3.5] [arxiv:2407.15073 alg. 3].
Each debate has an Affirmative, a Negative and a Judge agent
[arxiv:2407.15073 §3.3]. The paper's stated motivation is that a single LLM
can repeat memorised associations and gives no challenge to a wrong answer
[arxiv:2407.15073 §3.2].

The results are author-reported. With Gemini-2.0-Flash, MAC comes first on 10
of 15 evaluation points across five benchmarks [arxiv:2407.15073 §4.2]. On
Earthquake, exact search gets the same perfect scores [arxiv:2407.15073 tab.
2]. The single-agent version is competitive on most datasets at about half the
cost, while the full version is much better on the harder graphs
[arxiv:2407.15073 §4.4 tab. 6]. The full system uses 16 to 25 times the tokens
of the standard baselines [arxiv:2407.15073 §4.4]. The agent prompts appear
only as figures, and the card records no official code, so an implementation
must write its own prompts and output parser [arxiv:2407.15073 app. A.1].
Card: [Multi-Agent Causal Discovery Using Large Language Models](/papers/2407.15073).

### Causal variables as an inductive bias (position paper)

This is a design argument, not a method. The paper proposes that the variables
a learner uses should be high-level and verbalisable, and that these variables
are often causal [arxiv:2011.15091 §3.4] [arxiv:2011.15091 §3.5]. Changes in
the world should be sparse and local, starting at one or a few nodes of the
causal graph [arxiv:2011.15091 §3.6]. An intervention on one mechanism should
leave the others unchanged [arxiv:2011.15091 §4.1]. The paper also argues that
shuffling data into an i.i.d. sample discards causal information that lies in
the changes between sources [arxiv:2011.15091 §4.2].

The only causal-learning example is a meta-learning setting taken from cited
work. That work suggests that parameterising the correct variables lets a model
adapt faster to interventions. The paper gives no numbers for this
[arxiv:2011.15091 §4.3]. It gives no training procedure, and it calls combining
these biases in one system an open question [arxiv:2011.15091 §1]. Card:
[Inductive Biases for Deep Learning of Higher-Level Cognition](/papers/2011.15091).

### Causal reasoning probe for language models (wine casks)

This variant tests whether a language model answers interventional questions
from observed data. It does not estimate an effect. The model sees 20 casks,
each with the presence or absence of substances A, B and C, plus a stated
causal structure. It is then asked how many of 20 new casks will contain C,
under an observation (B is known to be present) or an intervention (B is added
by hand) [arxiv:2206.14576 §Results / Causal reasoning] [arxiv:2206.14576 fig. 5A].

The normative answers for the two questions differ only in the common-cause
condition, where intervention cuts the arrows into B [arxiv:2206.14576 §Results
/ Causal reasoning]. In the causal-chain condition the normative answers are the
same [arxiv:2206.14576 fig. 5C]. The paper reports that GPT-3 (Davinci) matches
the normative intervention answers in the common-cause condition only when B is
present. When B is absent, its intervention answers are not reduced
[arxiv:2206.14576 fig. 5B]. In the causal-chain condition its answers are
identical to those of the common-cause condition [arxiv:2206.14576 fig. 5C].
The paper's headline is that GPT-3 fails the causal task [arxiv:2206.14576
§Discussion]. All results are author-reported, and the causal figure's values
are typed into the plotting script rather than computed from the outputs
[code: CausalReasoning/analyze.py#L12-L18]. Card: [Using cognitive psychology to understand GPT-3](/papers/2206.14576).

### Randomised assignment and instrumental variables (trial analysis)

This family estimates an effect from an experiment, not from a model's
predictions. Tutor CoPilot randomised tutors, not students, to get the tool or
not [arxiv:2410.03017 §Methods]. Its primary model is an intent-to-treat
regression: a session outcome on a treatment indicator, student covariates and
a school-by-grade fixed effect, with the residual clustered by student-tutor
pair [arxiv:2410.03017 §Methods eq. 1]. Under this design the coefficient
estimates the effect of access. The paper reports a 4.0 percentage-point rise in
the unconditional exit-ticket pass rate (standard error 0.015, control mean
0.617, Romano-Wolf adjusted p = 0.010) [arxiv:2410.03017 tab. 1].

For the effect of use, the paper uses assignment as an instrument and estimates
it by two-stage least squares. It reports a 14 percentage-point increase, from
62% to 76% [arxiv:2410.03017 §Results]. The card does not restate the
instrument's exclusion assumptions, so check the paper's supplementary analysis
before relying on this number. The trial ran for about two months, and students
matched to a tutor with access had little exposure. The paper lists this as a
limitation, and it may explain the null survey and end-of-year results
[arxiv:2410.03017 §Limitations]. All results are author-reported.

### Disclosing the randomisation to a simulated subject (LLM simulation)

This variant changes the prompt of an LLM that plays a subject in a price
experiment. The study estimates demand for 40 consumer packaged-goods products
against a human survey, with the price as the treatment [arxiv:2312.15524 §2].
In the usual blinded setup, each simulated customer sees one scenario and is
not told that it is part of a test [arxiv:2312.15524 §5]. When the price
changes, the model also changes the details it invents, such as the last price
paid and a competing price [arxiv:2312.15524 §2, fig. 1]. The simulated demand
curve is then inverted-U, while the human curve slopes downward
[arxiv:2312.15524 §2, fig. 2].

Adding covariates to the prompt was the first remedy the paper tests. Demographic
personas, and then a sequence of twelve stages of behavioural covariates, were
added to the prompt [arxiv:2312.15524 §3, app. E]. The error on the survey was
lowest at stage 2 (MAE 0.0971, against 0.2250 at stage 1) and rose for most
later stages [arxiv:2312.15524 app. E]. The paper reports a cost of the remedy,
which it calls focalism: the model gives the stated details too much weight
[arxiv:2312.15524 §1, §3].

The second remedy is unblinding. Its system prompt says that the price is drawn
at random and uniformly from a range, and that the customer sees one price and
does not know about the draw [arxiv:2312.15524 §5]. The range is not given in the
text; the minimum and maximum are placeholders [arxiv:2312.15524 §5]. Out of the
box, GPT-4o-mini's mean absolute error (MAE) against the survey falls from 0.532
with the blinded prompt to 0.397 with the unblinded one [arxiv:2312.15524 tab. 1].
The text says unblinding lowered MAE for every model tested, with relative gains
from about 1% to over 60%, but the per-model values appear only in a figure
[arxiv:2312.15524 §5, fig. 6]. After fine-tuning on the survey, the best
unblinded run reaches 0.113, against 0.134 for the best blinded run
[arxiv:2312.15524 tab. 1]. Adding 10,000 purchase-only observations from Amazon
browsing data worsened blinded evaluation (0.233) but left unblinded evaluation
near its earlier error (0.120) [arxiv:2312.15524 tab. 2].

The study covers one demand setting and one main model for the covariate
analysis, and the human price range is not stated [arxiv:2312.15524 §2, §3].
All results are author-reported.

### Step-level intervention for agent failures (CAR)

This variant applies the interventional question to a run of an LLM agent, not
to a model's predictions. It asks which step of a failed run caused the
failure [arxiv:2606.08275 §1]. The paper argues that the step which performs
the harmful action is often only carrying out an earlier decision, so blaming
that step is a category error [arxiv:2606.08275 §1]. It describes LLM-judge
attribution as correlational, and cites about 14% step-level accuracy as the
best reported result for it on the Who&When benchmark [arxiv:2606.08275 §1]
[arxiv:2606.08275 §6].

The intervention redraws the action at one step from the same policy and then
reruns the rest of the run. The paper defines this resample as the null
intervention. Four further operations are defined: forcing an action, replacing
a tool result, editing the message history, and swapping the model from that
step on [arxiv:2606.08275 §2]. Because the policy is stochastic, K reruns give
a distribution of outcomes, not one path [arxiv:2606.08275 §2]. The
single-step estimator compares the bad-outcome probability under the
intervention with the observed run, and reports a confidence interval. The
causal locus is the latest step whose interval still excludes zero on the side
where the intervention lowers that probability [arxiv:2606.08275 §4].

When two steps cause the failure only together, the paper splits credit with a
Monte-Carlo Shapley estimator over steps. A coalition holds its steps at their
factual actions, and its value is the bad-outcome probability over K rollouts.
Marginals are not cached across permutations, because caching would make their
variance collapse and give false confidence [arxiv:2606.08275 §4].

The paper validates on synthetic structural causal models whose answers are
planted in advance. There the single-step method recovers the pivotal step, and
the Shapley estimator splits a two-step AND interaction at 0.44 and 0.45, against
an analytic 0.5 each [arxiv:2606.08275 §5]. It reports no run on real agent
traces and no comparison with LLM-judge or other attribution methods
[arxiv:2606.08275 §5] [arxiv:2606.08275 §7].

Two caveats follow from the design. The estimate is a total effect: rerunning
step k also redraws every later random step, so an early and irrelevant step can
look important. The paper names common random numbers as a refinement that it
does not implement [arxiv:2606.08275 §4] [arxiv:2606.08275 §7]. The method
also depends on the recorded state being faithful. Reruns call the environment afresh, so
the side effects of real tools are not modelled, and the demonstrations use
mocked tools [arxiv:2606.08275 §7]. Hosted models vary even at temperature 0, so
the paper measures the rate at which replayed actions match the recorded ones,
rather than assuming exact replay [arxiv:2606.08275 §3].

## When to use which

The review reports no benchmark that compares its counterfactual methods on
speed or accuracy, so the choices below follow the mechanisms the review
describes. Rows from the causal ML survey name the constraint the survey ties to
each method. The survey also reports no head-to-head numbers.

| Situation | Use | Why |
|---|---|---|
| You need a causal grounding and can supply a causal model, even a partial one | RECOURSE [arxiv:2103.04244 §6.7 tab. 2] | Only method the review marks causal. The review also warns that an assumed partial model can yield incorrect counterfactuals [arxiv:2103.04244 §6.7] |
| You need a causal grounding and have no causal model | None in the counterfactual review | The review's position is that current methods lack the causal grounding, so this gap is what its argument rests on [arxiv:2103.04244 §9.1] [arxiv:2103.04244 §10] |
| You want a causal statement from a partial-dependence plot or an importance score | Treat it as a predictive description; state the causal assumptions and use a causal discovery tool [arxiv:2007.04131 §9] | Importance and effect plots do not identify causes by themselves. The review names causal discovery and inference from ML models as open [arxiv:2007.04131 §9] |
| Test data may come from shifted environments, and you have data from several of them | Invariant risk minimisation [arxiv:2206.15475 §3.2] | Aims at predictors that stay optimal across environments. The survey says it often matches plain training, so compare it against that baseline [arxiv:2206.15475 §3.2] |
| You need a fairness criterion in terms of a causal model | Counterfactual fairness [arxiv:2206.15475 §6.1] | Requires the latent exogenous variables and structural equations. The survey gives no procedure to fit or test them [arxiv:2206.15475 §6.1] |
| You can extract symbolic rules from a trained network and want to ask interventional or counterfactual questions over them | Symbolic extraction, as the position paper proposes it [arxiv:2012.05876 §5] | The paper argues this is possible in principle. It reports no experiment, and it names sound extraction from very large networks as open [arxiv:2012.05876 §6] |
| You need a causal graph over variables from data plus text metadata, and can afford many LLM calls per graph | MAC [arxiv:2407.15073 §3] | Refines a statistically derived graph with metadata. Costs 16 to 25 times the baseline tokens. The single-agent variant is cheaper and weaker on harder graphs [arxiv:2407.15073 §4.4 tab. 6] |
| You need one counterfactual under explicit constraints, and the model can be encoded as a satisfiability formula | MACE [arxiv:2103.04244 §6.2 eq. 20] | Constraints and distance are part of one formula. Runtime is not stated |
| You need several different counterfactuals | DiCE [arxiv:2103.04244 §6.1 eq. 18] | Its objective adds a diversity term over k counterfactuals |
| You need counterfactuals that stay plausible for the data | FACE, ε-graph variant [arxiv:2103.04244 §6.1 eq. 17] | The review reports that the ε-graph variant gave the most feasible counterfactuals |
| You want sparse changes | Any method with an L1 distance, such as WatcherCF [arxiv:2103.04244 §4.1 §6.1 eq. 11] | The review says L1 favours sparse changes [arxiv:2103.04244 §4.1] |
| You explain a text classifier at word level | SED-C or LIME-C [arxiv:2103.04244 §6.4] | Removes or ranks words that flip the prediction |
| You already compute Shapley values | SHAP-C or SHAP-CC [arxiv:2103.04244 §6.5] | Reuses Shapley values; SHAP-CC answers contrastive questions |
| You keep a store of earlier explanation cases | CBR [arxiv:2103.04244 §6.6] | Adapts stored counterfactuals to a new query |
| You can randomise who gets access to a tool or treatment, and want the effect of access | Intent-to-treat regression with a treatment indicator and covariates [arxiv:2410.03017 §Methods eq. 1] | Keeps the randomised comparison. The card reports a 4.0 percentage-point effect for access, with tutors as the unit of randomisation [arxiv:2410.03017 tab. 1] |
| You want the effect of actual use, and assignment changes use | Instrumental variables with two-stage least squares [arxiv:2410.03017 §Results] | Estimates use from assignment. The card reports 14 percentage points, larger than the access effect, but does not restate the instrument's assumptions |
| You want to build causal structure into a learner's design, and have no tested method for it | Treat the position paper's biases as hypotheses: verbalisable causal variables, sparse local changes, independent mechanisms [arxiv:2011.15091 §3.5] [arxiv:2011.15091 §3.6] [arxiv:2011.15091 §4.1] | The paper gives no algorithm and no experiments [arxiv:2011.15091 §1]. Test each bias yourself before relying on it |
| You want to check whether a language model handles interventions, not only observations, on a small stated causal structure | Wine-cask intervention-versus-observation probe [arxiv:2206.14576 fig. 5A] | The normative answer is known for each condition, so score both conditions. The GPT-3 results are mixed, so do not rely on the headline alone |
| You simulate a randomised experiment with an LLM and can state the randomisation design | Unblinded prompt: say the treatment is drawn at random from a range, and that the subject does not know it. Fill in the range yourself [arxiv:2312.15524 §5] | Lowered MAE against a human survey for every model the text tests. The paper does not give the range, so choose it and report it |
| You plan to add covariates to a simulated subject's prompt | Add them in small steps, and score each step against human data [arxiv:2312.15524 app. E] | The lowest error came at stage 2, and the curve is not monotone after it, so there is no single safe count |
| You have a failed multi-step agent run that you can replay from recorded state, and want to know which step caused the failure | Contrastive single-step intervention, as CAR defines it [arxiv:2606.08275 §4] | Asks, for each step, whether redrawing it lowers the bad-outcome probability, with a confidence interval. Validated only on synthetic models with known answers [arxiv:2606.08275 §5] |
| Two steps of an agent run only cause the failure together | Monte-Carlo Shapley over steps, with fresh rollouts for each coalition [arxiv:2606.08275 §4] | Splits credit between interacting steps. The paper says Shapley is exponential in the worst case, and its Monte-Carlo variance depends on the budget [arxiv:2606.08275 §7] |
| You estimate an effect and the confounders are in the text | Regression adjustment, reading eq. 7 as one outcome model per treatment level, averaged over observed covariates [arxiv:2109.00725 eq. 7] | The survey gives no estimator. The method is valid only if ignorability, positivity and consistency hold, and ignorability cannot be tested from data [arxiv:2109.00725 §2.2] [arxiv:2109.00725 §3.1] |
| The outcome you measure is an NLP model's output | Fit that model on a sample separate from the one used for the effect [arxiv:2109.00725 §3.2] | The survey's recommendation for the measurement step |
| The treatment is a property of the text | Check positivity before adjusting; high-dimensional text can make it fail [arxiv:2109.00725 §3.1] [arxiv:2109.00725 §3.3] | Adjusting without overlap gives estimates the data cannot support |
| You want robustness to spurious features and can write counterfactual pairs | CDA, with manual edits where possible, and a check for new shortcuts [arxiv:2109.00725 §4.1.1] | Manual edits are the most accurate but costliest; keyword swaps can miss labels |
| You have labelled data from several environments and a nuisance factor | Distributional invariance criteria [arxiv:2109.00725 §4.1.2] | The survey says these were applied only to classification, so expect a gap for other tasks |
| You want to explain a model's use of internal features | Intervene on internal nodes, not on attention weights [arxiv:2109.00725 §4.3] | Attention weights can mislead, according to the survey |
| Your outcome can be written as a rule over the run | A rule-based outcome function [arxiv:2606.08275 §7] | The paper prefers rule-based outcomes, since judge-based outcomes add their own noise |
| Your agent's tools change real state | Replace the tools with mocks before attributing | Reruns call the environment afresh and side effects are not modelled. The paper's demonstrations use mocked tools [arxiv:2606.08275 §7] |

## Implementation sketch

The shared core is the instance-centric search. The review gives no learning
rate, step count, λ schedule or start point for it, so those values are left
open below [arxiv:2103.04244 §6.1].

```pseudocode
# Instance-centric counterfactual search  [§4.1] [§6.1]
# Input:  instance x (p,), model f giving a score, target y'
# Output: counterfactual x' (p,)

def counterfactual(x, target, model, lam, mad, steps):
    # Start point not stated; the instance is used here  [§6.1]
    candidate = x.copy()
    for _ in range(steps):                  # step count not stated  [§6.1]
        # Squared gap between the model score and the target  [eq. 10]
        gap = (model(candidate) - target) ** 2
        # L1 distance, each feature scaled by its MAD  [eq. 11]
        dist = sum(abs(candidate - x) / mad)
        # Trade the two terms off with lambda  [eq. 10]
        loss = lam * gap + dist
        # Optimiser is Adam; learning rate not stated  [§6.1]
        candidate = adam_step(candidate, grad(loss))
    return candidate
```

Variants differ in the loss (prototype, FACE path cost, DiCE diversity), in the
optimiser (FISTA, Nelder-Mead, graph search) and in the distance
[arxiv:2103.04244 §6.1].

For an interventional or counterfactual query on a known SCM, the sketch
above does not apply. Use the three-step procedure in the Formal core section
instead [arxiv:2206.15475 §2.3.2].

## Contradictions

The sources frame causal grounding differently. The counterfactual review says
post-hoc, model-agnostic explainers capture feature correlations rather than the
mechanism behind a prediction, so they may not give causability
[arxiv:2103.04244 §1.2] [arxiv:2103.04244 §9.1]. The causal ML survey places
counterfactual explanations under causal explanations. It presents the smallest
feature change that flips a prediction as one such explanation, and treats
causal algorithmic recourse as the version that uses an SCM
[arxiv:2206.15475 §5.2.1] [arxiv:2206.15475 §5.2.2]. These two papers disagree
on whether the first kind is causal. Both sides are stated here and are not
resolved.

The position paper argues that a symbolic description extracted from a network
supports interventions and counterfactual questions [arxiv:2012.05876 §5]. This
is the same claim as the counterfactual review's causal-grounding argument, from
a different direction, and it is also not resolved. The position paper's causal
claims are argued, not measured [arxiv:2012.05876 §2].

The counterfactual review's own counts do not agree with each other.
It says 26 algorithms were analysed in §5 and 18 in §6, and its summary table
has 25 rows [arxiv:2103.04244 §5] [arxiv:2103.04244 §6] [arxiv:2103.04244 tab. 2].

The review also reports a study by Zheng et al. with mixed results: causal
information helped only users without prior domain knowledge [arxiv:2103.04244
§8.1] [arxiv:2103.04244 §9.2].

The GPT-3 study's headline says the model fails the causal task
[arxiv:2206.14576 §Discussion]. Its own table reports that intervention answers
match the normative answer when B is present in the common-cause condition
[arxiv:2206.14576 fig. 5B]. Both statements come from the same paper, and this
page does not resolve them.

## Papers

- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](/papers/2103.04244): a systematic review of model-agnostic counterfactual explanations, with a seven-family taxonomy and an argument that they lack causal grounding.
- [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](/papers/2109.00725): a survey of causal inference for text, covering text as confounder, outcome or treatment, and causal views of robustness, fairness and interpretation. It runs no experiments.
- [Causal Machine Learning: A Survey and Open Problems](/papers/2206.15475): a survey of causal ML, with the SCM core, five problem areas, a benchmark list and open problems. It runs no experiments.
- [Neurosymbolic AI: The 3rd Wave](/papers/2012.05876): a position paper on combining neural learning with symbolic reasoning, with a view that extracted rules support interventional and counterfactual questions. It runs no experiments.
- [General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models](/papers/2007.04131): a review of ten pitfalls in PDP, ALE, permutation importance and SHAP, with a section on why their output is not causal without assumptions.
- [Using cognitive psychology to understand GPT-3](/papers/2206.14576): classic cognitive-psychology tests run on GPT-3, including a wine-cask task that compares intervention with observation. Results are author-reported.
- [Multi-Agent Causal Discovery Using Large Language Models](/papers/2407.15073): a two-stage LLM debate method that picks and runs a statistical causal-discovery algorithm, then refines the graph with metadata. Results are author-reported.
- [Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise](/papers/2410.03017): a randomised trial of an LLM tutoring assistant, with an intent-to-treat analysis and a 2SLS estimate of use. Results are author-reported.
- [Inductive Biases for Deep Learning of Higher-Level Cognition](/papers/2011.15091): a position paper that proposes causal, verbalisable variables and sparse, local mechanism changes as inductive biases. It gives no algorithm or experiments.
- [Using LLMs as simulated subjects, read through a causal lens](/papers/2312.15524): a study of LLM subjects in a price experiment, which shows confounding by the price and proposes disclosing the randomisation in the prompt. Results are author-reported.
- [Causal Agent Replay: Counterfactual Attribution for LLM-Agent Failures](/papers/2606.08275): attributes a failed agent run to a step by resampling that step and rerunning, with a Monte-Carlo Shapley estimator for steps that interact. Validated only on synthetic models with planted answers.