A Review on Explainability in Multimodal Deep Neural Nets

In plain words

Modern AI systems often learn from several kinds of input at once, such as a picture together with a question written in text. These systems are very accurate, but their inner workings are hard to see, so people struggle to tell whether a correct answer was reached for the right reason. This review asks how to make such combined systems explain themselves.

It walks through the main ways to combine different inputs, then the main ways to explain a model, and then the explanation methods built for systems that take pictures and text. It also covers how explanations are judged, which public datasets exist for testing them, and the risks of bias and of attacks that fool the model. The authors end with a list of open problems rather than a single recommended method.

TL;DR

This is a survey and commentary, not a method. It sorts multimodal fusion designs and explainability techniques into families and points out gaps in evaluation, datasets, bias handling and robustness. It runs no experiments and reports no scores of its own. Use it to find the families and their citations, not to implement anything.

Problem

Deep networks that combine several input types are accurate but opaque. Their predictions are hard to justify, so trust, debugging, fairness checks and adoption in areas such as healthcare or driving suffer § I § III § IV. The paper asks what explainability means when the inputs are heterogeneous (different scales, representations and sampling rates), and it argues that unimodal explanation methods do not cover this case § I § II (A. Typical challenges).

Method

The paper is a literature review. Its sections, as numbered in the compiled paper, are:

• §II: multimodal learning, fusion and recent fusion techniques § II (B, C), with tables and figures tab. 3 tab. 4 tab. 5 fig. 1 fig. 2 fig. 3.
• §III: explainable AI. It defines explanation levels (pre-modelling, modelling, post-hoc) § III (A), local versus global scope § III (B), and families of post-hoc methods: model-agnostic, feature attribution, distillation and intrinsic § III (C) to (E). The taxonomy is in fig. 4 and fig. 5, and a table of techniques is in tab. 6 (first table labelled VI).
• §IV and §V: multimodal explanations. Five method families are proposed in §V: attention-based (A), counterfactual (B), interactive (C), graph-based (D) and attribute-based (E). A second table labelled VI summarises them.
• §VI: evaluation of explanations (functionally grounded, human grounded and application grounded) and metrics § VI.
• §VII: requirements that depend on the stakeholder § VII.
• §VIII: datasets for explainability in multimodal settings § VIII.
• §IX: bias and fairness in multimodal data § IX.
• §X: adversarial attacks as a tool for understanding models § X.
• §XI: fourteen numbered observations and recommendations, the last on knowledge-graph infusion § XI.
• §XII: discussion, including a short description of CLIP and DALL·E § XII (OpenAI DALL-E and CLIP).

The fusion designs are described in words in §II, and the three main ones can be sketched as follows. The paper gives no code.

# Early fusion: merge raw or low-level features, then decide once  [§ II] (B.1) [fig. 1]
# Inputs: one feature vector per modality
def early_fusion(modality_features):
    # Join the vectors (concatenation; pooling and gating are also named)
    fused = concatenate(modality_features)
    return classifier(fused)

# Late fusion: decide per modality, then combine the decisions  [§ II] (B.2) [fig. 2]
# Inputs: one input per modality
def late_fusion(modality_inputs):
    # One classifier per modality
    decisions = [classifier_m(x_m) for m, x_m in modality_inputs]
    # Combine by Bayes rule, max, average or majority vote  [§ II] (B.2)
    return combine(decisions)

# Hybrid (intermediate) fusion: a shared representation layer  [§ II] (B.3) [fig. 3]
# Inputs: one input per modality
def hybrid_fusion(modality_inputs):
    # Per-modality feature extractors
    features = [extract_m(x_m) for m, x_m in modality_inputs]
    # Joint representation in a shared layer; the training loss flows back
    joint = shared_layer(concatenate(features))
    return predict(joint)

Pre-modelling, modelling and post-hoc explanation levels are contrasted in §III (A). Post-hoc families listed in §III (C) include backpropagation methods (activation maximisation, deconvolution, Grad-CAM, LRP, DeepLIFT, Integrated Gradients, SHAP, Gradient Input) and perturbation methods (occlusion, RISE, LIME), as well as partial dependence, ICE, LIME, SHAP and influence functions in §III (B.2). Distillation (§III, D) builds a local surrogate or a global translation model. Intrinsic methods (§III, E) explain through attention or through joint training of a prediction and an explanation head.

The multimodal families in §V are: attention maps that show regions or words used for an answer § V (A); counterfactual pairs that show the smallest change that flips a decision § V (B); interactive loops with a human or a virtual agent § V (C); scene graphs and knowledge graphs § V (D); and attributes that act as a human-readable bridge § V (E).

Hyperparameters

Not applicable. The paper trains no model and states no training settings; its organisation is described in § I (A. Organization). Metric and dataset settings are described in § VI and § VIII, without values for any run.

Results

Not applicable. The paper runs no experiments and reports no results of its own. The evaluation metrics it names are BLEU-4 [225], METEOR [226], ROUGE, CIDEr [227], SPICE [228] and leakage-adjusted simulatability (LAS) § VI. Datasets it names for explainability work include VQA 2.0, Visual Genome, VQA-X, ACT-X, VQA-E, CUB, VQA-CP and RAVEN, with references in § VIII and § V. Its comparison of fusion strategies in tab. 4 is a set of qualitative judgements, not measurements.

Paper vs code

No official code. The paper links no repository or project page. A GitHub repository search for the title's keywords returned no repository attributable to the authors among the results I could read. The paper is a survey with no implementation to compare.

Pitfalls

• No search protocol. The sections read (§I, §II, §III, §XI and §XIII) state no search method, inclusion criteria or time window, so coverage cannot be checked.
• Table citations do not match the reference list. In the first table labelled VI tab. 6, Grad-CAM is cited as [130], but the text cites it as [14] and [130] is the XFlow paper in the list. Integrated Gradients and DeepLIFT both point at [135], which is a review of robust pattern recognition. Layer-wise Relevance Propagation points at [131], while the text cites [161]. Check any number from this table against the list.
• Duplicate table labels. Two tables are labelled VI, one in §III and one in §V. Cite them by section.
• Incomplete references. Entry [28] lists only "Explainable deep learning" with no venue or year. Entry [84] has no venue, and entry [58] has a garbled title.
• Opinion stated as fact. Several claims, such as that attention maps may not carry real explanatory power and that the field is still well short of human-level reasoning, are the authors' judgement. Some are cited, many are not. Descriptions of DALL·E and CLIP in §XII were not checked against their sources.
• Qualitative fusion table. The yes/no ticks in tab. 4 (handling missing modalities, flexibility and so on) are the authors' judgement and come with no experiments.
• Scope is narrower than the title. The explainability sections are almost all visual question answering, captioning and a few self-driving or medical cases. Audio, sensor and tabular modalities are mentioned mainly for fusion, not explanation.
• Attention is contested. §V notes that attention maps may highlight areas that are not the ones a human would use, citing [188]. Treat attention-based explanations as disputed, not as ground truth.
• Not an implementation guide. The fusion pseudocode above is a summary of §II, not an algorithm from the paper. The paper gives no pseudocode or equations for any explanation method.

Lineage

• Builds on: § I cites earlier general and domain surveys of explainable AI (for example [24], [25], [26], [29]) and multimodal learning surveys ([46], [57]). It does not state which of their findings it extends.
• Compared against: not stated as a benchmark comparison. The paper positions itself against prior surveys by focusing on vision and language § I (Organization).
• Superseded by: not stated in the sections read.