Feature-level Interaction Explanations in Multimodal Transformers

Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim, Randy Goebel · 2026

Read Feature-level Interaction Explanations in Multimodal Transformers by Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim, Randy Goebel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13326.txt.

Introduction

Yeji Kim and co-authors present FL-I2MoE, a way to explain predictions by multimodal transformers. Most methods flag key tokens in each modality but miss how feature pairs work as a team. Their Mixture-of-Experts layer sorts evidence as unique, synergistic, or redundant at the feature level. It runs on token and patch sequences from frozen pretrained encoders. Monte Carlo probes score synergistic pairs and redundant pairs with set measures. Masking top pairs hurt scores more than masking random pairs, which backs the method.

Catalog record

CREATORYeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim, Randy Goebel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35430
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13326

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature-level Interaction Explanations in Multimodal Transformers Yeji Kim, Housam Khalifa Bashier Babiker, Mi-Young Kim, Randy Goebel Abstract Multimodal Transformers often produce predictions without clarifying how different modalities jointly support a decision. Most existing multimodal explainable AI (MXAI) methods extend unimodal saliency to multimodal backbones, highlighting important tokens or patches within each modality, but they rarely pinpoint which cross-modal feature pairs provide complementary evidence (synergy) or serve as reliable backups (redundancy). We present Feature-level I 2 MoE (FL-I 2 MoE), a structured Mixture-of-Experts layer that operates directly on token/patch sequences from frozen pretrained encoders and explicitly separates unique, synergistic, and redundant evidence at the feature level. We further develop an expert-wise explanation pipeline that combines attribution with top- K % K\% masking to assess faithfulness, and we introduce Monte Carlo interaction probes to quantify pairwise behavior: the Shapley Interaction Index (SII) to score synergistic pairs and a redundancy-gap score to capture substitutable (redundant) pairs. Across three benchmarks (MM-IMDb, ENRICO, and MMHS150K), FL-I 2 MoE yields more interaction-specific and concentrated importance patterns than a dense Transformer with the same encoders. Finally, pair-level masking shows that removing pairs ranked by SII or redundancy-gap degrades performance more than masking randomly chosen pairs under the same budget, supporting that the identified interactions are causally relevant. 1 Introduction Multimodal models that combine diverse modalities (e.g., text and images) are increasingly deployed in high-stakes domains such as clinical decision support, legal analytics, and safety-critical content moderation [ 14 , 13 ] . In these settings, a prediction is rarely trusted based on plausible pixels or words in isolation. What matters is how modalities jointly support the decision, either by providing complementary evidence or by offering redundant, mutually confirming signals that make predictions robust to noise or partial modality failure. However, conventional saliency maps can still look convincing even when the model effectively relies on a single modality, which obscures whether the decision is truly cross-validated or fragile [ 5 , 4 , 20 ] . Figure 1 illustrates this with a simple text-image example. Even for the same movie poster and plot, a model that predicts the genre by jointly using piano-related regions in the image and words such as “piano” or “concert” is behaving very differently from a model that mostly trusts the text, or one that guesses “drama” from a face alone. For practitioners, it is crucial to know whether each patch or token acts as unique evidence, whether a given patch-token pair serves as redundant backup, or whether the two together provide genuinely synergistic information. This concrete, feature interaction determines whether a prediction is robust because modalities cross-check each other, or fragile because it hinges on a single unreliable view. Figure 1 : Input modalities for the qualitative MM-IMDb example: (a) movie poster image and (b) plot summary ( Four Minutes , true labels: drama , music ). Current multimodal explainable AI (MXAI) methods only partially address this need [ 21 ] . Most work extends unimodal attribution techniques to multimodal Transformers by computing importance scores over joint feature sequences or cross-attention layers and visualizing them as saliency maps. These explanations are useful for showing “where the model looks,” but they usually treat the backbone as a single network. They do not reveal what kind of interaction the highlighted features have or how they jointly support the prediction [ 21 , 13 ] . As a result, users cannot easily tell whether a prediction is well supported by consistent multimodal evidence or dominated by one potentially noisy source. Game-theoretic and information-theoretic approaches aim to quantify multimodal contributions more explicitly. Shapley-value methods, including the Shapley Interaction Index (SII), can in principle model pairwise interactions, and PID-style decompositions separate information into unique, redundant, and synergistic components [ 2 , 3 , 4 , 10 ] . Recent work like MultiSHAP applies Shapley-style interaction attribution post-hoc to a fixed multimodal backbone, but it typically operates at the modality level or over coarse feature groups and therefore does not expose fine-grained feature-level pairs [ 3 ] . Moreover, estimating Shapley-style pair interactions over dense token–patch spaces becomes computationally prohibitive, which often necessitates aggregation or restricted candidate definitions. Interaction-aware architectures partially address this gap by introducing explicit components for different interaction roles. In particular, I 2 MoE [ 1 ] incorporates PID-inspired experts (uniqueness, synergy, and redundancy) and an interaction loss to encourage specialization, but it operates on pooled modality vectors and therefore cannot reveal which feature-level cross-modal pairs each expert actually relies on. In this work, we revisit interaction-aware multimodal modeling from a feature-level explainability perspective: we extend I 2 MoE to operate directly on token and patch sequences, and we use Monte Carlo interaction probes together with pair-level masking to quantitatively identify the cross-modal feature pairs that serve as synergistic or redundant evidence. Our contributions are summarized as follows: • We introduce a Feature-level I 2 MoE (FL-I 2 MoE) that routes features through uniqueness, synergy, and redundancy experts, providing structured interaction-aware explanations while improving performance. • We develop an expert-wise pipeline combining attribution with top- K\% masking, and show improved faithfulness and interaction specificity over a dense Transformer on three benchmarks. • We define cross-modal interaction scores (SII-based synergy and a redundancy-gap metric) and use pair-level masking to show that top-ranked feature pairs are causally important and aligned with expert roles, revealing concrete synergistic and redundant pair interactions. 2 Related Work 2.1 Multimodal XAI and Feature Attribution Multimodal explainable AI (MXAI) seeks to make model predictions over text, images, speech, and tables more transparent [ 13 ] . In practice, most methods extend unimodal feature attribution to multimodal Transformers by computing attention- or gradient-based scores over joint feature sequences or cross-attention layers and visualizing them as saliency maps [ 7 , 8 , 9 ] . While these approaches are widely used and often yield plausible explanations, they typically treat the backbone as a single monolithic network and do not explicitly distinguish unique, redundant, or synergistic cross-modal information [ 21 ] . As a result, they mainly reveal where the model attends, but not how concrete cross-modal feature pairs support or substitute for each other in the prediction. 2.2 Quantifying Multimodal Contributions and Interactions Beyond saliency maps, game-theoretic and information-theoretic approaches aim to quantify multimodal contributions more explicitly. Shapley-value methods, including the Shapley Interaction Index (SII), can in principle model feature contributions and pairwise interaction effects [ 2 , 3 , 4 ] , while Partial Information Decomposition (PID) separates information into unique, redundant, and synergistic components [ 10 ] . These tools provide notions of interaction and are often applied post-hoc to an already trained backbone, including sample-level analyses such as MultiSHAP that attribute cross-modal effects for individual inputs [ 3 ] . However, existing work typically defines features at the modality level or over coarse feature groups, and the resulting interaction scores are frequently aggregated into global statistics. Moreover, estimating Shapley-style pair interactions over dense token-patch spaces quickly becomes computationally prohibitive, which often necessitates aggregation or restricted candidate definitions rather than exposing fine-grained feature-level pairs. Related perturbation-based analyses include EMAP [ 5 ] and DIME [ 6 ] , which remain post-hoc to a fixed backbone. 2.3 Interaction-aware Multimodal Architectures A separate line of

Read the complete text