Counterfactual explanation

Overview

Given a model f, an input x and its prediction, a counterfactual explanation is a nearby input x' whose prediction differs, or equals a chosen target y'. The user learns what would have to change for a different outcome. The closest counterfactual solves a minimisation of a distance d(x, x') [arxiv:2103.04244 §4.1] 2103.04244 eq. 4. The review collects L0, L1, L2 and L∞ as candidate choices for d 2103.04244 eq. 5 and argues that L1 favours sparse changes 2103.04244 §4.1.

The variants share an informal list of desirable properties: proximity, plausibility (immutable features are not changed), sparsity, diversity and feasibility. The review defines each in words and gives no formula for any of them 2103.04244 §4.2.

The review's scope is post-hoc, model-agnostic methods that work on a trained black box 2103.04244 §1.2. The review's central concern is that these methods do not rest on causal theory, so an explanation can capture correlations instead of causes [arxiv:2103.04244 §1.2] 2103.04244 §9.1.

The framing is contrastive. A survey chapter on human-centred XAI draws on Miller's social-science review, which lists four properties of human explanations, the first being that they are contrastive 2110.10790 §5. Counterfactuals fit that framing directly. The same chapter lists "Why Not" and "How to Be That" as user questions that counterfactual methods can answer 2110.10790 §3 [arxiv:2110.10790 tab. 1].

Variants

Seven tabular families, grouped by the machinery each one uses [arxiv:2103.04244 §5] 2103.04244 fig. 10, followed by NLP families. Each tabular family links to the review card Counterfactuals and Causability in Explainable Artificial Intelligence.

Penalised-loss search

Minimises a loss over candidate x' that combines a squared gap between f(x') and the target and a distance term 2103.04244 §6.1 [arxiv:2103.04244 eq. 10]. The distance is L1 with each feature divided by its median absolute deviation 2103.04244 eq. 11. The review names three members:

• WatcherCF: optimised with Adam gradient descent 2103.04244 §6.1.
• Prototype counterfactuals: adds an autoencoder reconstruction term and a pull toward a class prototype in encoding space 2103.04244 eq. 12 2103.04244 eq. 13, optimised with FISTA 2103.04244 §6.1.
• Weighted counterfactuals: adds a per-feature weight to the distance 2103.04244 eq. 14, optimised with Nelder-Mead 2103.04244 §6.1.

Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Graph path search (FACE)

Measures the path cost as a density-weighted length and finds the shortest path on a graph built over the data 2103.04244 eq. 15 [arxiv:2103.04244 eq. 16]. The density can be estimated with kernel density estimators, a k-NN graph or an ε-graph 2103.04244 eq. 17. Dijkstra's algorithm gives the path 2103.04244 §6.1. The review reports that the ε-graph variant gave the most feasible counterfactuals 2103.04244 §6.1. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Diverse sets (DiCE)

Selects k counterfactuals that trade off a prediction hinge loss and proximity against a determinantal-point-process diversity term 2103.04244 eq. 18. Optimised by gradient descent, with L1 distance and MAD scaling [arxiv:2103.04244 §6.1]. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Nearest enemy (TRUCE and Growing Spheres)

Finds the nearest point with a different prediction, using an L2 distance plus a weighted count of changed features 2103.04244 eq. 19. The search grows along directions until it reaches the decision boundary [arxiv:2103.04244 §6.1]. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Constraint solving (MACE, GIC, Coherent counterfactuals)

Encodes the model, the distance and the constraints as one satisfiability formula and asks a solver for a counterfactual within a distance threshold ε 2103.04244 eq. 20 2103.04244 §6.2. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Genetic search (CERTIFAI, MOCE, LORE)

• LORE generates a neighbourhood with a genetic algorithm and fits a decision tree to it, with a mixed distance over categorical and continuous features 2103.04244 eq. 21 2103.04244 §6.3.
• CERTIFAI runs a custom genetic search with L1/MAD distance and defines a robustness score as the expected distance to the closest counterfactual 2103.04244 eq. 22 2103.04244 §6.3.
• MOCE optimises four objectives (prediction gap, Gower distance, number of changed features and likelihood under the training data) with NSGA-II 2103.04244 eq. 23 2103.04244 §6.3.

Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Surrogate regression (LIME-C, SED-C, CLEAR)

Fits a local linear model to perturbed inputs, similar to LIME [arxiv:2103.04244 §6.4]. SED-C removes the smallest set of words, ranked by coefficient times feature value, that flips a linear text classifier 2103.04244 §6.4. CLEAR combines a boundary counterfactual with a local regression, scored by a counterfactual fidelity error 2103.04244 eq. 24 [arxiv:2103.04244 §6.4]. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Shapley-value search (SHAP-C, SHAP-CC)

SHAP-C ranks features by Kernel SHAP values and then runs the SED-C loop. SHAP-CC turns a contrastive "why P rather than Q" question into two Shapley-value questions 2103.04244 §6.5. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Case-based reasoning

Stores explanation cases, each a query with its good counterfactual, retrieves the nearest case, maps its features to the new query and revises until the class changes 2103.04244 §6.6. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Probabilistic and causal (PRINCE, C-CHVAE, MC-BRP, ABELE, CRUDS, RECOURSE)

These sample or model the data distribution with random walks, Markov chains, VAEs or graphical models 2103.04244 §6.7. RECOURSE is the only method the review marks as causal. It defines recourse actions as interventions on a structural causal model that may be only partly known 2103.04244 §6.7 2103.04244 tab. 2. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Causal algorithmic recourse (survey view)

Frames the question as the minimal-cost set of interventions that flips a prediction, and uses a structural causal model to predict the effect of each intervention 2206.15475 §5.2.2. The survey treats this as a contrastive explanation, next to the feature-change counterfactuals of the review above 2206.15475 §5.2.1. The survey gives the framing only and reports no experiments for it. Card: Causal Machine Learning: A Survey and Open Problems. The same survey applies the counterfactual idea to fairness: a predictor is counterfactually fair if, for an individual, its distribution is the same when only a protected attribute is changed, with the latent exogenous variables held fixed 2206.15475 §6.1. That use is covered in algorithmic fairness.

Minimal text edits (Polyjuice, MiCE)

For text classifiers, the edit is a few words or a rephrased sentence rather than a feature vector. Polyjuice generates edits with a fine-tuned GPT-2 and keeps those that change the model's prediction 2108.04840 §9.1. MiCE uses gradients to pick the tokens to mask, then fills them with a T5 model until the prediction flips 2108.04840 §9.2. The survey notes that counterfactual datasets built without the model are not explanations of it 2108.04840 §9 intro. Card: Post-hoc Interpretability for Neural NLP: A Survey, which describes both methods but reports no experiments of its own.

Counterfactual comparisons and augmentation (causal NLP survey)

The causal-inference survey for NLP proposes no counterfactual explainer. It casts some interpretation as a comparison between an input and a counterfactual input, including counterfactual activations inside the model [arxiv:2109.00725 §4.3]. It notes that attention weights can mislead as explanations and suggests intervening on internal nodes instead 2109.00725 §4.3. Its related use is counterfactual data augmentation, which edits training text to break spurious features; the survey lists the trade-offs of that route in 2109.00725 §4.1.1 (see the table below). Card: Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond. The survey reports no experiments, so these trade-offs are argued, not measured.

When to use which

The review gives no head-to-head benchmark of these methods, so the choices below follow from the mechanisms the review describes. Where the review states a result, the citation is given.

Implementation effort is not compared in the review. The "Code?" column of its summary table marks 19 of 25 rows as having public code, but these are third party repositories that the review did not verify 2103.04244 tab. 2. Check licence and currency before reuse.

Implementation sketch

The loop below is the common core of penalised-loss search. Variants change the distance, the optimiser, or replace the loop with a graph, solver, genetic or case-based search. Parameters the review leaves open are marked.

# Penalised counterfactual search  [arxiv:2103.04244 §6.1] [eq. 10]
# Input:  instance x (p,), model f returning a score, target y'
# Output: counterfactual x' (p,)

def counterfactual(x, target, model, lam, dataset):
    # Per-feature scale from the data  [arxiv:2103.04244 eq. 11]
    scale = median_absolute_deviation(dataset)          # (p,)
    # Start point is not stated; the instance is one choice  [§6.1]
    candidate = x.copy()
    for step in range(num_steps):                       # not stated
        # Squared gap to the target, plus scaled L1 distance  [eq. 10]
        gap = (model(candidate) - target) ** 2
        distance = sum(abs(candidate - x) / scale)
        # Fixed lam; the review gives no schedule  [eq. 10]
        loss = lam * gap + distance
        # Optimiser step; learning rate not stated  [§6.1]
        candidate = optimiser_step(candidate, loss)
    return candidate

Open problems

The food-quality review lists actionable counterfactual explanations as an open gap. Explaining why a sample was judged is not enough there; users also need to know what should change to reach a target, such as a better quality grade or a longer shelf life 2504.10527 §10. The review proposes counterfactual generation as future work and suggests it could support in silico simulation of alternative formulations 2504.10527 §10. Counterfactuals appear only in that section; the review reports no counterfactual method or experiment of its own 2504.10527 §10. Card: Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review.

Contradictions

• Causal status. The authors argue that most model-agnostic counterfactual methods are not grounded in causal theory and so may not give causability 2103.04244 §9.1 2103.04244 §10. The review also marks RECOURSE as causal 2103.04244 tab. 2 and reports that its causal model may be partly unknown, which can make the result wrong 2103.04244 §6.7. The review does not resolve whether causal methods are reliable in practice. The causal ML survey adds that causal assumptions are often untestable from observational data, and that wrong assumptions can amplify bias 2206.15475 §10 2206.15475 §10.3.2.
• Definition of "counterfactual". The review defines a counterfactual as a nearby input x' whose prediction differs from the original, or equals a chosen target, with no reference to the true label 2103.04244 §4.1 2103.04244 eq. 4. The NLP survey defines a counterfactual example as a minimal edit that changes the prediction and carries a different gold label, and it reserves "adversarial" for edits that keep the gold label 2108.04840 §9 intro. Both usages appear in the literature. Check which one a paper means before comparing their results.
• Counterfactual as a query on an SCM. The causal ML survey uses "counterfactual" for a query about one individual: observed facts are kept by inferring the noise terms from them (abduction), a structural equation is replaced (action), and the model is rerun (prediction) [arxiv:2206.15475 §2.2.2] 2206.15475 §2.3.2. The survey warns that using the prior over noise instead of the posterior gives the wrong answer for this query 2206.15475 §2.3. The review defines its counterfactual as a nearby input under the model and does not model noise terms 2103.04244 §4.1, so the two outputs answer different questions even though both are called counterfactuals.
• Value of causal information to users. The authors assume a human understandable explanation must also be causal 2103.04244 §1.2. The review cites Zheng et al. as finding mixed results, with causal information helping only users without prior domain knowledge 2103.04244 §8.1 2103.04244 §9.2.
• Reported comparisons. The review reports that ABELE beats LIME on coherence, stability and fidelity, and that CLEAR has higher fidelity than LIME in five case studies. These come from other works and were not checked by the review 2103.04244 §6.4 2103.04244 §6.7.
• Algorithm counts inside the review. The text gives 26 analysed algorithms in one section and 18 in another, and the summary table has 25 rows 2103.04244 §5 2103.04244 §6 2103.04244 tab. 2. Do not cite a single count without checking which one is meant.

Papers

• Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications: a systematic review of model-agnostic counterfactual methods with a seven-family taxonomy, arguing that such explanations are not grounded in causal theory.
• Post-hoc Interpretability for Neural NLP: A Survey: a survey of post-hoc NLP explanation families, including counterfactual text edits (Polyjuice, MiCE) and how each is validated.
• Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond: a survey of causal inference for text, covering counterfactual comparisons for interpretation and counterfactual data augmentation, with no experiments.
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences: a survey chapter that maps user questions, including "Why Not" and "How to Be That", to explanation techniques, and reviews HCI evidence on whether explanations help users.
• Causal Machine Learning: A Survey and Open Problems: a survey that defines counterfactual queries on a structural causal model and reviews causal recourse and counterfactual fairness, with no experiments of its own.
• Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review: a review of XAI in food quality that names actionable counterfactual guidance as an open gap.