Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

In plain words

Most language models learn which words go with which labels. That is useful for prediction, but it does not tell you what would happen if you changed one thing about a text. Causal inference is the set of tools for answering "what if we changed this?" questions. This survey explains how those tools meet language data, and it covers two directions.

In the first, researchers want a cause-and-effect answer from text, such as whether a profile icon lowers the number of likes a post gets. Text can be a hidden confounder, the outcome being measured, or the treatment itself, and each role makes the usual assumptions harder to justify. In the second, causal thinking is used to make a model less likely to rely on accidental shortcuts, to make its fairness conditions explicit, and to explain its predictions by comparing them with counterfactual inputs. The authors also list open problems and argue that NLP work should state its causal assumptions openly.

TL;DR

• A survey, not a method: it reviews causal inference for text in two parts, estimation §3 and robust, fair, interpretable prediction §4 §1.
• It states the identification assumptions (conditional ignorability, positivity, consistency) and an identification chain for the average treatment effect §2.2 eq. 4 eq. 5 eq. 6 eq. 7.
• It reports no experiments and no benchmark numbers; the value is the taxonomy and the open problems listed in §3.4 and §4.4.
• Use it as a reading map before designing a causal study on text or a counterfactual robustness test. Do not read its claims as measured results.

Problem

Standard NLP models are judged on predictive accuracy. Causal questions ask about the change that an intervention would bring about, and those questions need assumptions that a plain correlation does not supply §1 §2.2. Two failures are named. First, causal effects estimated from text can be biased when the text carries hidden confounders, when the measured outcome is itself an NLP output, or when the treatment is a latent property of the text §3.1 §3.2 §3.3. Second, correlational predictors can rely on spurious features that break under distribution shift, produce unequal error across groups, or offer explanations that are hard to trust §4 §4.1 §4.2 §4.3.

Method

The paper is a survey and has no algorithm of its own. Its structure is:

1. Background §2. Defines the counterfactual outcome for a unit and the average treatment effect (ATE) and conditional average treatment effect (CATE) eq. 1 eq. 2. It then lists three identification assumptions: conditional ignorability eq. 4, positivity eq. 5 and consistency eq. 6. An identification argument shows that, under these assumptions, the counterfactual mean can be written as an expectation of observable conditional means eq. 7 §2.2. Causal directed acyclic graphs (DAGs) are introduced to derive which variables to condition on, using d-separation §2.3 fig. 1.
2. Estimating causal effects with text §3. Discusses three roles for text: a confounder §3.1, an outcome §3.2 and a treatment §3.3. For each role it lists the prior approaches and the assumption that becomes hard to check. Future work covers heterogeneous effects, representation learning for causal targets, benchmarks and controllable generation §3.4.
3. Robust and explainable prediction §4. Covers:
   • learning robust predictors, through counterfactual data augmentation §4.1.1 and distributional criteria based on invariance across environments §4.1.2;
   • fairness, where counterfactual swaps of protected attributes are discussed and the invariance criteria are said to be rarely applied to text §4.2;
   • interpretation, where explanations are cast as comparisons between an input and a counterfactual, including counterfactual activations inside the model §4.3;
   • open problems: partial causal models, noisy proxies and structured outputs §4.4.
4. Conclusion §5 restates the two directions and calls for stating causal assumptions explicitly.

Working formulas from §2

These are the paper's definitions written as an estimator, for an implementer. The paper gives eq. 7 as an identity and does not provide an estimator. The code below is the standard reading of that identity and is ours, not the authors'.

# Back-door adjustment of the ATE  [§2.2] [eq. 1] [eq. 7]
# Input:  features X (one row per text), binary treatment T, outcome Y
# Output: estimate of E[Y(1) - Y(0)]

def ate_by_adjustment(X, T, Y):
    # Fit an outcome model for each treatment level a in {0, 1}  [eq. 7]
    for a in (0, 1):
        mu[a] = fit_regression(X[T == a], Y[T == a])
    # Average the difference over the observed confounders  [eq. 7]
    return mean(mu[1](X) - mu[0](X))

Assumptions behind this estimator: ignorability given X eq. 4, positivity so that both treatment groups exist at every X eq. 5, and consistency eq. 6. With text, the features X must be extracted from the text, which adds a measurement step the paper flags as unverified §3.1.

Hyperparameters

Not applicable: the paper is a survey and reports no experiments or training settings §1 §5.

Results

Not applicable: no experiments and no benchmark numbers are reported. The paper cites other work for semi-synthetic evaluations, where real covariates generate treatments and outcomes, and notes that no real-world text causal benchmark exists because counterfactuals are never observed §3.4 (Benchmarks paragraph).

Paper vs code

No official code.

The paper's footnote points to a curated list of causal-inference and language papers on GitHub (causaltext/causal-text-papers) (footnote on the abstract). That is a reading list, not an implementation of the paper's content, so it is not used as code and no code is cited. Because the paper has no method to reproduce, there is no paper-versus-code comparison to make. The official repository search was skipped as there is no method name to search for; this follows the wiki rule that surveys have no code.

Pitfalls

• Survey, not evidence. Claims such as "text as confounder breaks positivity" are argued in prose §3.1. Nothing in the paper measures how often this happens.
• The identification argument needs all confounders. Conditional ignorability requires that every confounder is measured §2.2 eq. 4. The paper states that this cannot be tested from data and must rest on domain knowledge §3.1.
• High-dimensional text threatens positivity. A text can almost determine its treatment, so the overlap needed for adjustment fails §3.1.
• Consistency fails under hidden versions of a treatment or under interference. Examples given are icon colour not recorded and a classifier trained on all users' treatment assignments §2.2 §3.2.
• Measurement is part of the estimate. When the outcome is produced by a trained model, the model must be fit on a sample separate from the one used to estimate the effect, as the paper recommends §3.2.
• Counterfactual data augmentation can add new shortcuts. Manual edits are accurate but expensive; keyword swaps can miss labels and covariates; and automatic rewriting may introduce new spurious correlations §4.1.1.
• Distributional criteria need richer data. They require labels for the nuisance factor or several labelled environments, and the paper notes that many environments may be needed §4.1.2. Those criteria have been applied only to classification §4.1.2.
• Attention is not a causal explanation by itself. The paper says attention weights can mislead and suggests intervening on internal nodes instead §4.3.
• Only top-down partial models are common. Most methods model text against document-level metadata, not local word-level invariances §4.4.

Lineage

• Surveys it builds on or extends: the paper says it follows earlier reviews of text in causal inference and widens their scope to robust prediction §1 (citations to Egami et al. 2018 and Keith et al. 2020 in the source).
• Background it defers to for estimation: standard causal inference texts, cited in §2.2.
• Related concept pages: causal inference, invariant learning, counterfactual explanation, algorithmic fairness.