--- kind: concept slug: counterfactual-explanation title: Counterfactual explanation summary: >- A counterfactual explanation names the smallest change to an input that flips a model's prediction, such as "the loan would have been approved with a higher income". Tabular methods differ in search mechanism, constraints and causal grounding; NLP work edits text minimally. Six reuse-licensed cards cover it so far; the food-quality review names it only as an open problem. tags: [explainable-ai, counterfactuals, causality] papers: ["2103.04244", "2108.04840", "2110.10790", "2206.15475", "2504.10527", "2109.00725"] related: [causal-inference, explanation-evaluation, feature-attribution, adversarial-text-generation, algorithmic-fairness, invariant-learning] status: draft updated_at: 2026-11-01 --- ## Overview Given a model f, an input x and its prediction, a counterfactual explanation is a nearby input x' whose prediction differs, or equals a chosen target y'. The user learns what would have to change for a different outcome. The closest counterfactual solves a minimisation of a distance d(x, x') [arxiv:2103.04244 §4.1] [arxiv:2103.04244 eq. 4]. The review collects L0, L1, L2 and L∞ as candidate choices for d [arxiv:2103.04244 eq. 5] and argues that L1 favours sparse changes [arxiv:2103.04244 §4.1]. The variants share an informal list of desirable properties: proximity, plausibility (immutable features are not changed), sparsity, diversity and feasibility. The review defines each in words and gives no formula for any of them [arxiv:2103.04244 §4.2]. The review's scope is post-hoc, model-agnostic methods that work on a trained black box [arxiv:2103.04244 §1.2]. The review's central concern is that these methods do not rest on causal theory, so an explanation can capture correlations instead of causes [arxiv:2103.04244 §1.2] [arxiv:2103.04244 §9.1]. The framing is contrastive. A survey chapter on human-centred XAI draws on Miller's social-science review, which lists four properties of human explanations, the first being that they are contrastive [arxiv:2110.10790 §5]. Counterfactuals fit that framing directly. The same chapter lists "Why Not" and "How to Be That" as user questions that counterfactual methods can answer [arxiv:2110.10790 §3] [arxiv:2110.10790 tab. 1]. ## Variants Seven tabular families, grouped by the machinery each one uses [arxiv:2103.04244 §5] [arxiv:2103.04244 fig. 10], followed by NLP families. Each tabular family links to the review card [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Penalised-loss search Minimises a loss over candidate x' that combines a squared gap between f(x') and the target and a distance term [arxiv:2103.04244 §6.1] [arxiv:2103.04244 eq. 10]. The distance is L1 with each feature divided by its median absolute deviation [arxiv:2103.04244 eq. 11]. The review names three members: - **WatcherCF**: optimised with Adam gradient descent [arxiv:2103.04244 §6.1]. - **Prototype counterfactuals**: adds an autoencoder reconstruction term and a pull toward a class prototype in encoding space [arxiv:2103.04244 eq. 12] [arxiv:2103.04244 eq. 13], optimised with FISTA [arxiv:2103.04244 §6.1]. - **Weighted counterfactuals**: adds a per-feature weight to the distance [arxiv:2103.04244 eq. 14], optimised with Nelder-Mead [arxiv:2103.04244 §6.1]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Graph path search (FACE) Measures the path cost as a density-weighted length and finds the shortest path on a graph built over the data [arxiv:2103.04244 eq. 15] [arxiv:2103.04244 eq. 16]. The density can be estimated with kernel density estimators, a k-NN graph or an ε-graph [arxiv:2103.04244 eq. 17]. Dijkstra's algorithm gives the path [arxiv:2103.04244 §6.1]. The review reports that the ε-graph variant gave the most feasible counterfactuals [arxiv:2103.04244 §6.1]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Diverse sets (DiCE) Selects k counterfactuals that trade off a prediction hinge loss and proximity against a determinantal-point-process diversity term [arxiv:2103.04244 eq. 18]. Optimised by gradient descent, with L1 distance and MAD scaling [arxiv:2103.04244 §6.1]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Nearest enemy (TRUCE and Growing Spheres) Finds the nearest point with a different prediction, using an L2 distance plus a weighted count of changed features [arxiv:2103.04244 eq. 19]. The search grows along directions until it reaches the decision boundary [arxiv:2103.04244 §6.1]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Constraint solving (MACE, GIC, Coherent counterfactuals) Encodes the model, the distance and the constraints as one satisfiability formula and asks a solver for a counterfactual within a distance threshold ε [arxiv:2103.04244 eq. 20] [arxiv:2103.04244 §6.2]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Genetic search (CERTIFAI, MOCE, LORE) - **LORE** generates a neighbourhood with a genetic algorithm and fits a decision tree to it, with a mixed distance over categorical and continuous features [arxiv:2103.04244 eq. 21] [arxiv:2103.04244 §6.3]. - **CERTIFAI** runs a custom genetic search with L1/MAD distance and defines a robustness score as the expected distance to the closest counterfactual [arxiv:2103.04244 eq. 22] [arxiv:2103.04244 §6.3]. - **MOCE** optimises four objectives (prediction gap, Gower distance, number of changed features and likelihood under the training data) with NSGA-II [arxiv:2103.04244 eq. 23] [arxiv:2103.04244 §6.3]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Surrogate regression (LIME-C, SED-C, CLEAR) Fits a local linear model to perturbed inputs, similar to LIME [arxiv:2103.04244 §6.4]. SED-C removes the smallest set of words, ranked by coefficient times feature value, that flips a linear text classifier [arxiv:2103.04244 §6.4]. CLEAR combines a boundary counterfactual with a local regression, scored by a counterfactual fidelity error [arxiv:2103.04244 eq. 24] [arxiv:2103.04244 §6.4]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Shapley-value search (SHAP-C, SHAP-CC) SHAP-C ranks features by Kernel SHAP values and then runs the SED-C loop. SHAP-CC turns a contrastive "why P rather than Q" question into two Shapley-value questions [arxiv:2103.04244 §6.5]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Case-based reasoning Stores explanation cases, each a query with its good counterfactual, retrieves the nearest case, maps its features to the new query and revises until the class changes [arxiv:2103.04244 §6.6]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Probabilistic and causal (PRINCE, C-CHVAE, MC-BRP, ABELE, CRUDS, RECOURSE) These sample or model the data distribution with random walks, Markov chains, VAEs or graphical models [arxiv:2103.04244 §6.7]. RECOURSE is the only method the review marks as causal. It defines recourse actions as interventions on a structural causal model that may be only partly known [arxiv:2103.04244 §6.7] [arxiv:2103.04244 tab. 2]. Card: [Counterfactuals and Causability in Explainable Artificial Intelligence](/papers/2103.04244). ### Causal algorithmic recourse (survey view) Frames the question as the minimal-cost set of interventions that flips a prediction, and uses a structural causal model to predict the effect of each intervention [arxiv:2206.15475 §5.2.2]. The survey treats this as a contrastive explanation, next to the feature-change counterfactuals of the review above [arxiv:2206.15475 §5.2.1]. The survey gives the framing only and reports no experiments for it. Card: [Causal Machine Learning: A Survey and Open Problems](/papers/2206.15475). The same survey applies the counterfactual idea to fairness: a predictor is counterfactually fair if, for an individual, its distribution is the same when only a protected attribute is changed, with the latent exogenous variables held fixed [arxiv:2206.15475 §6.1]. That use is covered in [algorithmic fairness](/concepts/algorithmic-fairness). ### Minimal text edits (Polyjuice, MiCE) For text classifiers, the edit is a few words or a rephrased sentence rather than a feature vector. Polyjuice generates edits with a fine-tuned GPT-2 and keeps those that change the model's prediction [arxiv:2108.04840 §9.1]. MiCE uses gradients to pick the tokens to mask, then fills them with a T5 model until the prediction flips [arxiv:2108.04840 §9.2]. The survey notes that counterfactual datasets built without the model are not explanations of it [arxiv:2108.04840 §9 intro]. Card: [Post-hoc Interpretability for Neural NLP: A Survey](/papers/2108.04840), which describes both methods but reports no experiments of its own. ### Counterfactual comparisons and augmentation (causal NLP survey) The causal-inference survey for NLP proposes no counterfactual explainer. It casts some interpretation as a comparison between an input and a counterfactual input, including counterfactual activations inside the model [arxiv:2109.00725 §4.3]. It notes that attention weights can mislead as explanations and suggests intervening on internal nodes instead [arxiv:2109.00725 §4.3]. Its related use is counterfactual data augmentation, which edits training text to break spurious features; the survey lists the trade-offs of that route in [arxiv:2109.00725 §4.1.1] (see the table below). Card: [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](/papers/2109.00725). The survey reports no experiments, so these trade-offs are argued, not measured. ## When to use which The review gives no head-to-head benchmark of these methods, so the choices below follow from the mechanisms the review describes. Where the review states a result, the citation is given. | Situation | Use | Why | |---|---|---| | You need a causal reading and can write down a causal graph, even a partial one | RECOURSE [arxiv:2103.04244 §6.7]; the cost-minimising framing in [arxiv:2206.15475 §5.2.2] | Built on interventions. The review warns that a wrong partial model can produce incorrect counterfactuals [arxiv:2103.04244 §6.7]. The survey gives the framing only, with no algorithm to compare. | | You can take gradients of the model | Penalised-loss search, e.g. WatcherCF [arxiv:2103.04244 §6.1] | Gradient-based optimisers are the stated method. The review says the choice of optimiser is relatively unimportant, which it does not test [arxiv:2103.04244 §6.1]. | | You only have prediction queries and the model is a black box | Genetic search, e.g. LORE or CERTIFAI [arxiv:2103.04244 §6.3] | The search evaluates the model on candidates and needs no internal access. This follows from the method type; the review reports no comparison. | | You need several alternative explanations | DiCE [arxiv:2103.04244 eq. 18] | Its objective includes a diversity term over k counterfactuals. | | You need a hard distance bound and a model that can be written as a logical formula | Constraint solving, e.g. MACE [arxiv:2103.04244 eq. 20] | The search returns a counterfactual within a threshold ε, or reports none. | | The input is text and the classifier is linear | SED-C [arxiv:2103.04244 §6.4] | Removes words to flip the prediction. The review describes it for linear text classifiers only. | | Feasibility matters more than speed | FACE with the ε-graph density [arxiv:2103.04244 §6.1] | The review reports this variant gave the most feasible counterfactuals. The ε value is not stated. | | Earlier examples of good explanations exist | Case-based reasoning [arxiv:2103.04244 §6.6] | Reuses stored counterfactuals as starting points. | | The input is text and you want edits a reader can see | Polyjuice or MiCE [arxiv:2108.04840 §9.1] [arxiv:2108.04840 §9.2] | Both edit words directly. Both depend on a generation model to write the replacements, and the survey gives no head-to-head comparison. | | The explanation is for end users asking "why not" or "how could I get a different result" | Counterfactual instances, as in DiCE or CEM [arxiv:2110.10790 tab. 1] | The chapter maps these question types to counterfactual methods. It calls the mapping a suggestion for discussion, not a validated result [arxiv:2110.10790 tab. 1]. | | You need counterfactual text for training or robustness testing, not an explanation | Manual edits, or keyword swaps checked for changed labels and covariates [arxiv:2109.00725 §4.1.1] | Manual edits are accurate but expensive. Keyword swaps can miss labels and covariates, and automatic rewriting can add new spurious correlations. These are the survey's stated risks, with no measurements. | Implementation effort is not compared in the review. The "Code?" column of its summary table marks 19 of 25 rows as having public code, but these are third party repositories that the review did not verify [arxiv:2103.04244 tab. 2]. Check licence and currency before reuse. ## Implementation sketch The loop below is the common core of penalised-loss search. Variants change the distance, the optimiser, or replace the loop with a graph, solver, genetic or case-based search. Parameters the review leaves open are marked. ```pseudocode # Penalised counterfactual search [arxiv:2103.04244 §6.1] [eq. 10] # Input: instance x (p,), model f returning a score, target y' # Output: counterfactual x' (p,) def counterfactual(x, target, model, lam, dataset): # Per-feature scale from the data [arxiv:2103.04244 eq. 11] scale = median_absolute_deviation(dataset) # (p,) # Start point is not stated; the instance is one choice [§6.1] candidate = x.copy() for step in range(num_steps): # not stated # Squared gap to the target, plus scaled L1 distance [eq. 10] gap = (model(candidate) - target) ** 2 distance = sum(abs(candidate - x) / scale) # Fixed lam; the review gives no schedule [eq. 10] loss = lam * gap + distance # Optimiser step; learning rate not stated [§6.1] candidate = optimiser_step(candidate, loss) return candidate ``` ## Open problems The food-quality review lists actionable counterfactual explanations as an open gap. Explaining why a sample was judged is not enough there; users also need to know what should change to reach a target, such as a better quality grade or a longer shelf life [arxiv:2504.10527 §10]. The review proposes counterfactual generation as future work and suggests it could support in silico simulation of alternative formulations [arxiv:2504.10527 §10]. Counterfactuals appear only in that section; the review reports no counterfactual method or experiment of its own [arxiv:2504.10527 §10]. Card: [Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review](/papers/2504.10527). ## Contradictions - **Causal status.** The authors argue that most model-agnostic counterfactual methods are not grounded in causal theory and so may not give causability [arxiv:2103.04244 §9.1] [arxiv:2103.04244 §10]. The review also marks RECOURSE as causal [arxiv:2103.04244 tab. 2] and reports that its causal model may be partly unknown, which can make the result wrong [arxiv:2103.04244 §6.7]. The review does not resolve whether causal methods are reliable in practice. The causal ML survey adds that causal assumptions are often untestable from observational data, and that wrong assumptions can amplify bias [arxiv:2206.15475 §10] [arxiv:2206.15475 §10.3.2]. - **Definition of "counterfactual".** The review defines a counterfactual as a nearby input x' whose prediction differs from the original, or equals a chosen target, with no reference to the true label [arxiv:2103.04244 §4.1] [arxiv:2103.04244 eq. 4]. The NLP survey defines a counterfactual example as a minimal edit that changes the prediction and carries a different gold label, and it reserves "adversarial" for edits that keep the gold label [arxiv:2108.04840 §9 intro]. Both usages appear in the literature. Check which one a paper means before comparing their results. - **Counterfactual as a query on an SCM.** The causal ML survey uses "counterfactual" for a query about one individual: observed facts are kept by inferring the noise terms from them (abduction), a structural equation is replaced (action), and the model is rerun (prediction) [arxiv:2206.15475 §2.2.2] [arxiv:2206.15475 §2.3.2]. The survey warns that using the prior over noise instead of the posterior gives the wrong answer for this query [arxiv:2206.15475 §2.3]. The review defines its counterfactual as a nearby input under the model and does not model noise terms [arxiv:2103.04244 §4.1], so the two outputs answer different questions even though both are called counterfactuals. - **Value of causal information to users.** The authors assume a human understandable explanation must also be causal [arxiv:2103.04244 §1.2]. The review cites Zheng et al. as finding mixed results, with causal information helping only users without prior domain knowledge [arxiv:2103.04244 §8.1] [arxiv:2103.04244 §9.2]. - **Reported comparisons.** The review reports that ABELE beats LIME on coherence, stability and fidelity, and that CLEAR has higher fidelity than LIME in five case studies. These come from other works and were not checked by the review [arxiv:2103.04244 §6.4] [arxiv:2103.04244 §6.7]. - **Algorithm counts inside the review.** The text gives 26 analysed algorithms in one section and 18 in another, and the summary table has 25 rows [arxiv:2103.04244 §5] [arxiv:2103.04244 §6] [arxiv:2103.04244 tab. 2]. Do not cite a single count without checking which one is meant. ## Papers - [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](/papers/2103.04244): a systematic review of model-agnostic counterfactual methods with a seven-family taxonomy, arguing that such explanations are not grounded in causal theory. - [Post-hoc Interpretability for Neural NLP: A Survey](/papers/2108.04840): a survey of post-hoc NLP explanation families, including counterfactual text edits (Polyjuice, MiCE) and how each is validated. - [Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond](/papers/2109.00725): a survey of causal inference for text, covering counterfactual comparisons for interpretation and counterfactual data augmentation, with no experiments. - [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](/papers/2110.10790): a survey chapter that maps user questions, including "Why Not" and "How to Be That", to explanation techniques, and reviews HCI evidence on whether explanations help users. - [Causal Machine Learning: A Survey and Open Problems](/papers/2206.15475): a survey that defines counterfactual queries on a structural causal model and reviews causal recourse and counterfactual fairness, with no experiments of its own. - [Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review](/papers/2504.10527): a review of XAI in food quality that names actionable counterfactual guidance as an open gap.