Feature attribution

Overview

The problem is to say which parts of an input drove a model's output, or which parts of the image support a class. The shared idea is a map over the input that assigns each element a score. Many methods also make the scores add up to the change in the output between a reference and the actual input. This is the completeness property that a path-integrated map is checked against 2011.11036 §3.3 and that propagated Shapley values satisfy for each baseline 2105.00108 eq. 13. Shapley values are one way to get such scores from a game, by averaging each input's marginal contribution over orderings 2202.05594 §2.1.

Two families differ in what they score. Input-space methods assign credit to input elements, by perturbing the input along a path or by sampling coalitions 1803.07517 §5.2.2 2105.00108 §2. Feature-map methods such as class activation maps use the last feature map, not the input pixels, so they localise evidence rather than attribute to inputs 2203.00962 eq. 2. Counterfactual methods ask a different question: what smallest change to the input would flip the output 2103.04244 §4.1.

The field has no single agreed definition of a good explanation. The survey cards use qualitative criteria such as fidelity and clarity [arxiv:1803.07517 §5.1], and the quantitative numbers below come from the authors' own setups. On tabular data, the ranking itself can also change with the classifier fitted to the same data 2305.02012 §2.1.

Variants

Path-integrated gradient maps for dense outputs

For a super-resolution network, Local Attribution Maps integrate the gradient of a patch score along a path from a blurred input to the real input, using progressively less blur 2011.11036 §3.3 eq. 2. The authors use 100 path steps 2011.11036 §3.3 and report that the diffusion index of these maps correlates with PSNR across 48 networks, with Pearson and Spearman values above 0.8 2011.11036 §4.4 fig. 7. Card: Local Attribution Maps.

Shapley propagation through a model chain

DeepSHAP explains a series of models, such as a deep network feeding a tree model. It passes attributions back through the chain and rescales each link so the credit still sums to the output change 2105.00108 eq. 3 eq. 13. Averaging over a baseline set gives the interventional Shapley value 2105.00108 §6.2 eq. 11. The paper proves exactness only when every model is linear 2105.00108 §6.4 and says the efficiency property is approximate for mixed series 2105.00108 §6.5. Card: Explaining a Series of Models by Propagating Shapley Values.

Sampled Shapley values for tabular models

Kernel SHAP estimates Shapley values by fitting a weighted regression to a sample of feature subsets. Its sampling rule is a heuristic, and when the paper was submitted no non-asymptotic accuracy guarantee was known for it below 2^n − 2 samples 2410.01917 §1.2. Leverage SHAP keeps the regression and changes only the sampling. Its leverage scores have a closed form, so all subsets of one size share a score, and the sample can be drawn without listing every subset 2410.01917 §3.1. In the authors' experiments on XGBoost models over eight SHAP-library datasets, at 10n evaluations the mean relative squared error on Adult is 0.00416, against 0.0104 for the SHAP library's Optimized Kernel SHAP 2410.01917 tab. 1. Averaged over the 38 entries of tab. 2 that are not 1, the authors report about 50% of that baseline's error. A recount of tab. 2 that includes the ten entries equal to 1 gives about 0.60. Both methods are exact once the budget reaches 2^n, so the gap is limited to smaller budgets 2410.01917 tab. 2. Card: Provably Accurate Shapley Value Estimation via Leverage Score Sampling.

Ranking stability across models and correlated features

SHAP is usually fitted to one model at a time, but the same tabular data can give a different ranking for each classifier. On a UK Biobank cohort with four classifiers, the top three SHAP features agree across models and the other seven vary 2305.02012 §2.1. The paper names correlated features as a second source of change, because SHAP assumes the features are independent 2305.02012 §2.1. Its two proposed measures both come from a loop that drops the top-ranked feature and refits. NMR scores how far the ranking moves, and a lower value means a more stable ranking. MIP re-orders a list by each feature's relative rank across rounds, and the paper presents it as a way to allow for multicollinearity 2305.02012 §2.1 §3. On an airline satisfaction dataset with 22 features, NMR is 0.231 for LightGBM and 0.445 for a decision tree 2305.02012 §3. LightGBM also has the highest accuracy in the authors' table, 0.91 2305.02012 tab. 2. The paper says that this accuracy does not show its explanation is better, because the models used default parameters 2305.02012 §3. Neither measure has a formula in the paper 2305.02012 §2.1. Each case study uses one dataset and one split, with no seeds or confidence intervals 2305.02012 §2.1 §3. For LIME, the same paper applies it to one representative subject, and its weights describe that subject only 2305.02012 §2.2 §4. Card: A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME.

SHAP-ranked feature pruning for a forecaster

A forecasting paper ranks 47 lagged and calendar predictors by the mean absolute SHAP value from two models, a random forest and XGBoost. It removes the least important predictor one at a time, refits both models after each removal, and averages their MAE curves. The average is lowest after 36 removals, which leaves 11 predictors 2605.25305 §3.2.1 fig. 7 tab. 5. The kept set includes lags 1, 2, 3, 6 and 12, the maximum and minimum temperature, and the maximum wind speed tab. 5. A SHAP analysis of one model then shows that the same lags push forecasts in opposite directions at the two campuses 2605.25305 §4.2 fig. 12 fig. 13. Each campus has one test year, and the results are author-reported 2605.25305 §4.1. The paper does not say which rows scored the pruning curve 2605.25305 §3.2.1, and the card records that the released code fits the SHAP models on the full frame, including the test year. Card: Electricity Consumption Forecasting: An Approach Using Cooperative Ensemble Learning with SHapley Additive exPlanations.

Gradient sensitivity of fixed attributes

In the EA-LSTM rainfall-runoff model, the 27 static catchment attributes enter only through one input gate 1907.08456 eq. 7. To measure how strongly the output responds to each attribute, the authors take the mean absolute derivative with respect to the standardised attribute, averaged over days. The derivative is computed analytically 1907.08456 §2.6.2 eq. 14–15. The paper calls this a Morris-style global measure 1907.08456 §2.6.2, but the released code computes a local gradient, not a sampled average over the input space, as the card's pitfalls section records. The same paper adds Gaussian noise to the standardised attributes at σ = 0.1 to 1.0, and reports that the validation median NSE falls from 0.71 to 0.48 at σ = 1 1907.08456 §3.3 fig. 6. These are author-reported results from one model family. Card: Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets.

Full-gradient decomposition for ReLU networks

For a ReLU network with biases, the output splits exactly into an input-gradient term and a bias-gradient term 1905.00780 eq. 1. FullGrad turns these pieces into one saliency map for a CNN. It passes each piece through a post-processing step ψ (absolute value, rescale to [0, 1], upsample) and adds them up 1905.00780 eq. 2 §4.2. Because ψ is applied, the map is an approximation and no longer sums to the output 1905.00780 §4.2. On MNIST digit flipping, the authors' table gives 12.93 for the signed variant (no absolute value) and 8.26 for FullGrad, against 11.92 for plain gradients 1905.00780 tab. 1. On MNIST pixel perturbation, IntegratedGradients is lowest at all three RF values, and the paper's own summary is that no method wins on both tests 1905.00780 tab. 2 [supp. Digit Flipping]. The ImageNet pixel-perturbation and remove-and-retrain results appear only as plots 1905.00780 fig. 2. All numbers are the authors' own. Card: Full-Gradient Representation for Neural Network Visualization.

Agreement between attribution methods

Attribution methods applied to the same prediction can disagree. Krishna et al. define six agreement scores 2202.01602 §3.2 app. A. Four compare the top-k features by absolute attribution: feature, rank, sign and signed-rank agreement. Two compare a chosen feature set: rank correlation and pairwise rank agreement. Lower values mean more disagreement 2202.01602 §3.2. Their experiments show that disagreement is strongest on text. For an LSTM on AG News at k = 11, most method pairs have rank and signed-rank agreement below 0.1 2202.01602 §4.3.2 fig. 4. On ImageNet images with superpixels, LIME and KernelSHAP agree closely, with rank correlation 0.8977 [arxiv:2202.01602 tab. 2]. At pixel level, rank correlation between Integrated Gradients and SmoothGrad is 0.001 2202.01602 §4.3.3. In a user study, practitioners mostly resolved such conflicts with ad hoc heuristics 2202.01602 §5.2.4. The metrics come from practitioner interviews rather than a theory of which explanation is correct, and the results are author-reported [arxiv:2202.01602 §3.1]. Card: The Disagreement Problem in Explainable Machine Learning.

Counterfactual explanations

Counterfactual methods search for a nearby input that changes the prediction. The review groups the algorithms it covers into seven families, such as instance-centric search, constraint solvers, genetic search and Shapley-based methods 2103.04244 §5 tab. 2. It marks only one algorithm, RECOURSE, as causal, and argues that the rest rely on statistical associations 2103.04244 §6.7 tab. 2. These are the authors' claims and they run no experiments of their own. Card: Counterfactuals and Causability in Explainable Artificial Intelligence.

Class activation maps as training targets

Class activation maps (CAM) weight a feature map by the classifier's weights to show where a class is. ReCAM trains a second classifier on features masked by each class map, and then reads new maps out of the model 2203.00962 §4.1 eq. 5-9. On PASCAL VOC 2012 seeds, the authors report a rise from 48.8 to 54.8 mIoU for plain CAM versus ReCAM, and from 66.3 to 70.9 with IRN refinement 2203.00962 tab. 2. The seed step costs 1.0 to 1.9 time units on VOC in their table 2203.00962 tab. 2. Card: Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation.

Grad-CAM for a visual check

Grad-CAM is a gradient-weighted CAM. The CIFAKE paper uses it on a few random test images to see whether a synthetic-image detector looks at the main object or at the background 2303.14126 §3.4. The authors judge the maps by eye and report no quantitative explanation metric 2303.14126 §4.2. Card: CIFAKE. A dataset paper applies Grad-CAM and LIME in the same qualitative way, to its best and a weaker ImageNet-pretrained classifier on underwater images 2506.17455 §4.5 fig. 7. It reports no faithfulness test, and the Grad-CAM layer and LIME settings are not stated [arxiv:2506.17455 §4.5]. Card: AQUA20.

Relevance propagation for Transformers

Attention weights are often read as an explanation, but they leave out the value path, the feed-forward layers and the skip connections. Chefer et al. instead propagate the score of one class back through each block, using LRP-style rules for the linear layers, the additions and the matrix products 2012.09838 §3.1 §3.2. Each block contributes its gradient-weighted, head-averaged positive relevance plus an identity term, and the block matrices are multiplied in order 2012.09838 eq. 13 eq. 14. The [CLS] row, read over the patch tokens, gives the map 2012.09838 §3.4. The method is class-specific by construction and needs no contrasting pass 2012.09838 §2. On ImageNet with ViT-base, the perturbation AUC for the predicted class is 54.16 for negative and 17.03 for positive perturbation, against 53.1 and 20.05 for rollout 2012.09838 tab. 1. The margin over rollout on the negative test is small. On the segmentation test the mIoU is 61.95, against 55.42 for rollout and 32.89 for LRP 2012.09838 tab. 2. The evaluation covers ViT-base and BERT-base only, and the numbers are the authors' own 2012.09838 §4. Card: Transformer Interpretability Beyond Attention Visualization.

Sanity tests with planted defects

Adebayo et al. plant known defects and check whether the maps respond. In a birds-versus-dogs CNN where the background predicts the class, the maps of 15 attribution methods are compared with a relaxed ground-truth mask. SSIM against that mask ranges from 0.78 (KernelSHAP) to 0.98 (VGrad), which the authors read as the maps picking up the spurious background 2011.05429 §3.1 tab. 2. When 10% of training labels are flipped, the maps for a flipped and a correct training input still have SSIM between 0.73 and 0.99, and the authors conclude these methods are unlikely to find mislabelled examples 2011.05429 §3.2 fig. 6. When the top layers of a pretrained network are re-initialised, the modified back-propagation methods (Guided Backprop, DConvNet, PatternNet, PatternAttribution, LRP-SPAF) stay visually and quantitatively unchanged, so they fail that test 2011.05429 §4 fig. 7. The relaxed mask and the SSIM values are the authors' own constructions. Card: Debugging Tests for Model Explanations.

Taxonomy and criteria

The survey chapter splits explanation methods into rule extraction, attribution and intrinsic methods, and rates them against fidelity, interpretability, generalisability and explanatory power [arxiv:1803.07517 §5.1] §5.2. It reports no experiments. It is useful for framing, not for choosing an implementation. Card: Explanation Methods in Deep Learning.

A survey of the Shapley value in machine learning gives the definition that the sampled and regression methods above approximate. It states the four properties (null player, efficiency, symmetry, linearity) and the sampling and regression approximations, and it reports no experiments 2202.05594 §2.2 §3. It names the loss of these properties under approximation as the largest open issue 2202.05594 §5.1.3. Card: The Shapley Value in Machine Learning.

Later surveys frame the same space from other sides. The LLM survey files feature attribution under local explanations of fine-tuned models, in four families: perturbation, gradient-based, surrogate models such as LIME and SHAP, and layer-wise decomposition such as LRP 2309.01029 §3.1.1. It states the raw gradient in full but does not fix the norm used to turn a gradient into a score 2309.01029 §3.1.1. It reports no experiments of its own. Card: Explainability for Large Language Models: A Survey.

A health-care survey sorts post-hoc methods into model-based, attribution-based and example-based explanations. It keeps only perturbation-based attribution methods and places SHAP and QII among the Shapley-value methods 2007.15911 §4.2 tab. 1. Its evaluation table gives attribution methods a generic soundness metric, but no metric for completeness 2007.15911 tab. 2. The paper trains no models and has no experiments. Card: The role of explainability in creating trustworthy artificial intelligence for health care.

The NLP survey sorts post-hoc methods by the kind of explanation they give. For input features it covers gradients, Integrated Gradients, LIME, Kernel SHAP and Anchors, and it notes that faithfulness depends on the task 2108.04840 §6 §6.2. Card: Post-hoc Interpretability for Neural NLP. The human-centred chapter pairs each user question with example techniques, such as LIME or SHAP for "why" and counterfactual methods for "why not" 2110.10790 tab. 1. The authors present this mapping as a suggestion, not a validated result. It has no experiments. Card: Human-Centered Explainable AI.

A review of model-agnostic methods, such as permutation importance, partial dependence and Shapley values, lists ten pitfalls, each with a fix and an open question 2007.04131 §2–§9. Three bear directly on attribution. When features are dependent, perturbation-based scores can evaluate the model where the data has no points 2007.04131 §5.1. Testing 50 unimportant features at α = 0.05 gives a probability of about 0.923 of at least one false positive, a figure that assumes the tests are independent 2007.04131 §8.3. A non-zero importance does not show a cause without causal assumptions 2007.04131 §9. The paper has no benchmark, and its numbers come from simulations. Card: General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models.

A semi-systematic review of explainable AI in food quality, built from database searches and snowballing, sorts its studies by data type and explanation type 2504.10527 §1.1 §9. Pictorial data and visual explanations, mostly CAM-type heatmaps, dominate. Spectral and time-series data are rarely explained, and few studies combine several methods 2504.10527 §9 fig. 8 fig. 9. These are the authors' own tallies from a corpus they describe as non-exhaustive. Card: Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review.

When to use which

The ReCAM cost in tab. 2 is the authors' own timing table, in their time unit, and is not a benchmark on other hardware 2203.00962 tab. 2.

The Leverage SHAP results are the authors' own and were not re-run for this page 2410.01917 §5.

Implementation sketch

Common core of path-integrated attribution, with the completeness check. The path, the score and the number of steps are the only choices that change between variants.

# Path-integrated attribution for one scalar score  [arxiv:2011.11036 §3.3]
# Input:  input x (H, W, C); score(x) -> float; path(x, t) for t in [0, 1],
#         with path(x, 0) the baseline and path(x, 1) = x; m steps
# Output: map of shape (H, W, C); if the path is exact, the entries sum to
#         score(x) - score(baseline)

def path_attribution(x, score, path, m):
    # Points on the path, from the baseline to the input  [§3.3]
    points = [path(x, k / m) for k in range(m + 1)]
    attribution = zeros_like(x)
    for k in range(1, m + 1):
        # Gradient at the k-th point, times the step from the previous point
        # [§3.3] [eq. 4]
        grad = gradient(score, points[k])
        attribution += grad * (points[k] - points[k - 1])
    return attribution

Notes that the cards support:

• Use the step from the previous point, and check completeness by summing the map. The printed index in eq. 4 of the LAM paper does not satisfy this check, so do not copy it literally 2011.11036 §3.3 eq. 4.
• Do not add an extra 1/m factor. An earlier version of the LAM preprint had one, and the later version removed it 2011.11036 app. D.
• Blur width and kernel size are not given in the LAM paper, so calibrate them rather than assuming values 2011.11036 §3.3.
• For Shapley propagation, each baseline costs one pass through the whole chain, and the result is the mean over baselines 2105.00108 eq. 11. Efficiency is checked per link, not guaranteed for mixed series 2105.00108 §6.5.
• For CAM, the map is at feature-map resolution, not the input resolution 2203.00962 eq. 2.
• For sampled Shapley values, the Kernel SHAP weight of a subset of size s is 1 / (C(n, s) · s · (n − s)) 2410.01917 eq. 4. Algorithm 1 needs a budget of at least n evaluations 2410.01917 alg. 1. Report the number of evaluations actually run, because released implementations can differ from the paper's algorithm 2410.01917 §3.2.
• For agreement checks, feature agreement is trivially 1 when k equals the number of features, so it says nothing at that k 2202.01602 app. C.1. The paper's rank is a position and does not say how ties are ranked 2202.01602 app. A.1.
• For perturbed grids in permutation importance and partial dependence, use quantiles or subsamples rather than equidistant values [arxiv:2007.04131 §5.1]. The paper does not say how to draw subsampled values jointly, and drawing each feature separately breaks their dependence [arxiv:2007.04131 §5.1].
• For a re-initialisation test, the authors do not state the initialiser they used, so fix one and record it 2011.05429 §4.
• For Transformer relevance maps, the block matrix is the identity plus the positive part of gradient times relevance, averaged over heads, and the blocks are multiplied in block order 2012.09838 eq. 13 eq. 14. The head average and the positive part can be read in two orders. The released code uses one of them, and the card lists the difference, so check the Transformer card before reimplementing.
• The addition rule of the Transformer method divides by x + y and by sums of relevance, and the paper gives no guard against zero 2012.09838 eq. 6 eq. 10. The card's pitfalls section suggests a guard that returns zero where the denominator is zero.
• For ranking-stability checks, refit each model and recompute the full ranking after each removal, then report the split and the formula you used. The paper gives neither measure as a formula, and its numbers come from one split 2305.02012 §2.1 §3.
• For SHAP-ranked pruning, fit the ranking and the refits on training rows only. The paper does not say which rows scored its pruning curve, and the forecasting card records a released pipeline that used the test year 2605.25305 §3.2.1.
• For the static-attribute sensitivity, the released code takes the derivatives with respect to standardised values, so the numbers are not in physical units 1907.08456 §2.6.2. Report that scale, or convert before comparing with other attribution values. The EA-LSTM card lists the code check.
• For FullGrad, the paper applies ψ to each channel inside the bias-gradient sum 1905.00780 eq. 2. The released code applies it to the whole layer tensor, and that gives a different map. Pick one, and say which in your report. The FullGrad card lists the other differences, including batch-norm layers and the channel sum of the input term.
• For FullGrad, check the sum before ψ, not after it. The paper states that only the un-post-processed sum equals the output 1905.00780 §4.2. The paper does not mention a small constant in the rescale; the released code adds 1e-6 to the divisor, as the FullGrad card records.
• For Shapley values over models, data points or agents, the survey defines each game through a trained model or a fit, but does not say what v(∅) is. Set it by convention and record it 2202.05594 §2.1 §4.1.
• For permutation sampling, Algorithm 1 as written calls v twice per player and ordering 2202.05594 alg. 1. Caching the predecessor set saves about half the calls, because the set for one player is the input for the next. This is a card observation, not the survey's own analysis; see the Shapley survey card.

Contradictions

• What an attribution means causally. The counterfactual review argues that post-hoc, model-agnostic explanations capture correlations between features rather than the mechanism behind a prediction [arxiv:2103.04244 §1.2]. DeepSHAP takes the interventional Shapley value, defined with a do-operator on a flat causal graph, and presents that as its target 2105.00108 §6.1. The papers disagree on whether the second counts as causal. This page does not resolve it. The pitfalls review adds that a predictive model's importance does not identify a cause without causal assumptions 2007.04131 §9.

• Attribution as an explanation. The taxonomy notes that attribution does not say why a factor matters, and that low fidelity makes maps noisy 1803.07517 §5.2.2. The LAM paper uses the diffusion index of its maps as a quantity to correlate with network quality 2011.11036 §4.4, and does not test whether the maps explain the outputs to a user.

• Yardsticks for quality. The agreement paper compares explanations with one another and does not test which is correct 2202.01602 §3.2. The Leverage SHAP paper scores each estimate against exact Tree SHAP values on a model it trained 2410.01917 §5. The two papers measure different things, so their numbers cannot be compared directly. The planted-defect study uses ground truth built into the model and data, and it finds that maps do not reliably flag mislabelled examples 2011.05429 §3.2.

• Attribution as a diagnostic. The food review proposes XAI as a diagnostic layer for checking and auditing models on food data 2504.10527 §2.1. The planted-defect study finds that the maps it tested reveal a spurious background but not every defect, and that users in its study relied mainly on predicted labels 2011.05429 §3.1 app. H. The food review reports no test of this kind, so the two are not measured the same way.

• Where LIME belongs. The health-care survey files LIME and Anchors under model-based local explanations, not attribution 2007.15911 §4.1. The NLP survey and the LLM survey list LIME among input-feature methods 2108.04840 §6 2309.01029 §3.1.1. Surveys disagree on the category, so check a method's category in the paper you cite.

• What "completeness" means. The health-care survey uses the word for two things. Its definition is that an explanation carries enough information to recompute the output 2007.15911 §2.2. Its attribution metrics use the word for sensitivity-N, which the table marks as having no metric 2007.15911 §5.2 tab. 2. The sum-to-output check in the Overview is a different test, so read each paper's definition.

• Which faithfulness metric to trust. The LLM survey notes that the best-ranked explanation under one faithfulness metric can be the worst under another 2309.01029 §5.1. The agreement study finds that methods disagree on which features matter 2202.01602 §3.2. Neither paper settles which metric or method is right, so this page does not either.

• Relevance propagation under two tests. On ImageNet with ViT-base, the Transformer paper reports LRP and partial LRP below its own relevance maps on the perturbation and segmentation tests 2012.09838 tab. 1 tab. 2. The planted-defect study reports that LRP-SPAF, one LRP variant, stays visually and quantitatively unchanged after top-layer re-initialisation 2011.05429 §4 fig. 7. The two studies use different models and different tests, so neither result settles which method to use. This page does not resolve it.

• Completeness and local importance. The Overview's completeness checks 2011.11036 §3.3 2105.00108 eq. 13 are about the sum of the scores. The FullGrad paper argues that, for piecewise-linear functions, a saliency map cannot in general be both complete and weakly dependent on the input 1905.00780 prop. 1. The two statements are about different properties, and this page does not choose between them.

• Shapley axioms and their guarantees. The survey says the axioms often fail under approximation 2202.05594 §5.1.3. Leverage SHAP is presented with a guarantee on its regression objective, not directly on the Shapley values 2410.01917 §3.2, which is a different claim from the axioms. Neither paper tests the other's property, so read each claim for what it covers.

Papers

• Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges: a survey and taxonomy of explanation needs and methods, with no experiments.
• Full-Gradient Representation for Neural Network Visualization: exact full-gradient decomposition for ReLU networks, and the FullGrad saliency map, with author-reported MNIST and ImageNet tests.
• Interpreting Super-Resolution Networks with Local Attribution Maps: path-integrated gradient maps for super-resolution networks, with a diffusion index.
• Counterfactuals and Causability in Explainable Artificial Intelligence: a systematic review of counterfactual explanation methods and their causal claims.
• Explaining a Series of Models by Propagating Shapley Values: DeepSHAP, which propagates Shapley values through a chain of models.
• Class Re-Activation Maps for Weakly-Supervised Semantic Segmentation: ReCAM, a second training stage that sharpens CAM seeds for segmentation.
• CIFAKE: Image Classification and Explainable Identification of AI-Generated Synthetic Images: a synthetic-image detector with Grad-CAM heatmaps used for qualitative inspection.
• Transformer Interpretability Beyond Attention Visualization: class-specific relevance maps for Transformers, propagated through attention and skip connections, with ViT and BERT results.
• Explainability for Large Language Models: A Survey: a survey of LLM explainability by training paradigm, with feature attribution as one local family and no experiments.
• Post-hoc Interpretability for Neural NLP: A Survey: a survey of post-hoc NLP explanation families and how each is validated, with no experiments.
• Human-Centered Explainable AI (XAI): From Algorithms to User Experiences: a survey chapter that maps user questions to XAI techniques, with no experiments.
• The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective: six agreement scores for feature attributions, and a study of how practitioners resolve conflicts.
• Provably Accurate Shapley Value Estimation via Leverage Score Sampling: Leverage SHAP, a sampling change to Kernel SHAP with a guarantee on the regression objective.
• Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets: an LSTM variant whose static-attribute gate supports a gradient sensitivity read-out, with author-reported hydrology results.
• General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models: a review of ten pitfalls for permutation importance, partial dependence and Shapley values, with simulations and no benchmark.
• Debugging Tests for Model Explanations: planted-defect tests for attribution maps, and a user study with defective image classifiers.
• The Shapley Value in Machine Learning: a survey of the Shapley value's definition, approximations and ML applications, with no experiments.
• Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review: a semi-systematic review of XAI in food quality, with a five-topic taxonomy and no experiments.
• A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME: SHAP and LIME rankings across classifiers and correlated features, with the NMR and MIP stability measures; author-reported, with no new benchmark.
• The role of explainability in creating trustworthy artificial intelligence for health care: a survey of explanation types and their evaluation metrics for health-care AI, with no experiments.
• Electricity Consumption Forecasting: An Approach Using Cooperative Ensemble Learning with SHapley Additive exPlanations: SHAP-ranked backward elimination of forecasting inputs, with author-reported results on two campuses.
• AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging Conditions: a 20-class underwater image benchmark, with Grad-CAM and LIME maps read qualitatively for two models; author-reported.