Dynamic Interaction-Aware and Causality-Disentangled Framework for Multimodal Sentiment Analysis

Guangyuan Dong NUS Ziwei Hong UPenn Shenghao Liu CUC Chenyu Wu Duke Yuanyuan Fang BU Zihao Li Liverpool Xudong Zhang ∗ PKU Bingchen Liu ∗ SDU Yuchen Zhang SeeWay.ai Haitao Ding JLU Zhenzhou Zhou ∗ NEU Ziyu Song ∗ JLU · 2026

Read Dynamic Interaction-Aware and Causality-Disentangled Framework for Multimodal Sentiment Analysis by Guangyuan Dong NUS Ziwei Hong UPenn Shenghao Liu CUC Chenyu Wu Duke Yuanyuan Fang BU Zihao Li Liverpool Xudong Zhang ∗ PKU Bingchen Liu ∗ SDU Yuchen Zhang SeeWay.ai Haitao Ding JLU Zhenzhou Zhou ∗ NEU Ziyu Song ∗ JLU (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-30994.txt.

Introduction

Dong and colleagues improve multimodal sentiment analysis, combining language, visual, and acoustic signals. Old methods fail in two ways: fixed conflict controls cannot adapt across samples, and language bias misguides learning from other signals. Their MCAF first builds a causal model to strip language bias, yielding a clean guide. A dynamic router then judges how the three signals interact at three levels, weights them, and a denoising step filters leftover noise. Tests on two benchmarks set new best scores on key metrics.

Catalog record

CREATORGuangyuan Dong NUS Ziwei Hong UPenn Shenghao Liu CUC Chenyu Wu Duke Yuanyuan Fang BU Zihao Li Liverpool Xudong Zhang ∗ PKU Bingchen Liu ∗ SDU Yuchen Zhang SeeWay.ai Haitao Ding JLU Zhenzhou Zhou ∗ NEU Ziyu Song ∗ JLU
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37490
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.30994

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Dynamic Interaction-Aware and Causality-Disentangled Framework for Multimodal Sentiment Analysis Guangyuan Dong NUS Ziwei Hong UPenn Shenghao Liu CUC Chenyu Wu Duke Yuanyuan Fang BU Zihao Li Liverpool Xudong Zhang ∗ PKU Bingchen Liu ∗ SDU Yuchen Zhang SeeWay.ai Haitao Ding JLU Zhenzhou Zhou ∗ NEU Ziyu Song ∗ JLU Abstract Although Multimodal Sentiment Analysis (MSA) has proven effective by leveraging rich information from language, visual, and acoustic modalities, existing methods still grapple with two core challenges: 1) Static conflict suppression mechanisms fail to adapt to dynamic variations across samples, and 2) The inherent sentimental bias within the language modality, which can misguide the learning from other modalities, remains entangled. To this end, we propose a Dynamic Multimodal Causal Disentanglement and Adaptive Fusion Framework (MCAF). The cornerstone of our framework is the Multi-Granularity Causal Dynamic Router and a Conditional Diffusion Denoising Module. First, we introduce a causal intervention module based on the information bottleneck principle, which constructs a Structural Causal Model to disentangle the sentimental bias from language features, yielding a “de-confounded” language representation as a pure guiding signal. Second, we devise a Dynamic Multimodal Router that can evaluate the interaction states (complementary, conflicting, or redundant) among visual, acoustic, and de-confounded language signals in real-time across three levels: feature, temporal, and modality. It then adaptively allocates weights and routes information flow for fine-grained regulation. Finally, we incorporate a lightweight Conditional Diffusion Denoising Module, which performs iterative denoising on the fused joint representation to explicitly filter out residual irrelevant information, thereby generating a robust hyper-modality representation. Extensive experiments on the CMU-MOSI and CMU-MOSEI benchmarks demonstrate that MCAF sets new state-of-the-art on key classification metrics, achieving an Acc-2/F1 of 86.52%/86.51% on MOSI and 86.72%/86.65% on MOSEI, while remaining highly competitive on other metrics. Comprehensive analyses and visualizations further validate the model’s efficacy in dynamically perceiving interactions, disentangling bias, and enhancing interpretability. I Introduction Multimodal Sentiment Analysis (MSA) aims to integrate information from various modalities such as language, vision, and audio to comprehensively understand human emotional expressions, holding significant application value in fields like human-computer interaction and mental health monitoring [ 1 , 2 , 3 ] . Although existing research has made remarkable progress through multimodal fusion, sentiment-irrelevant and conflicting information across modalities remains a critical bottleneck limiting further performance improvement. Language-guided adaptive hyper-modality learning can effectively suppress redundant and conflicting information in visual and acoustic modalities [ 4 , 5 , 6 , 7 ] , thereby significantly enhancing the robustness and accuracy of sentiment analysis. However, we identify two fundamental challenges that current methods fail to address adequately: First, existing methods rely on static conflict suppression mechanisms. While the model learns to suppress irrelevant information under language guidance, its suppression strategy is essentially a static weight distribution learned from global training data. In real-world scenarios, the complementary, conflicting, or redundant relationships among modalities vary dynamically across different samples and even across different moments within the same sample. For instance, in sarcastic expressions, a positive tone may strongly conflict with negative text, and this conflict itself is a crucial clue for sentiment judgment and should not be simply suppressed [ 8 ] . Therefore, we need a mechanism capable of dynamically perceiving and adapting to real-time inter-modal interactions, rather than relying on static suppression. Second, the dominance of the language modality may introduce implicit bias. Existing methods predominantly use language features as guidance due to their rich and clean emotional signals. However, language itself may carry strong prior emotional tendencies, which could lead the model to inappropriately suppress genuinely valuable, yet superficially contradictory, fine-grained emotional cues from visual and acoustic modalities (e.g., micro-expressions, tonal shifts). This modal bias induced by language dominance hinders the model from capturing genuinely complex, multimodal collaborative emotional expressions, particularly under advanced contexts such as sarcasm or irony. To address these issues, this paper proposes a Dynamic Multimodal Causal Disentanglement and Adaptive Fusion Framework. Our core idea is to model MSA as a process of dynamic interaction perception and causal disentanglement. Specifically, we first introduce a Causality-Guided Modal Disentanglement Module, which explicitly separates semantic content from potential emotional bias in language features by constructing a structural causal model, thereby obtaining a ”purified,” debiased language representation as the guiding signal. Building upon this, we design a Dynamic Multimodal Interaction Router that can evaluate the interaction states (complementary, conflicting, or redundant) among visual, acoustic, and purified language signals in real-time across three levels: feature, temporal, and modality. Based on this evaluation, it adaptively routes information flow and adjusts fusion weights, achieving fine-grained, context-aware multimodal integration. Finally, we incorporate a lightweight Generative Denoising Fusion Refiner that iteratively refines the fused joint representation to filter residual noise further and enhance the expressiveness of discriminative features. Experiments on multiple benchmark datasets show that our method not only significantly outperforms existing state-of-the-art models, but also demonstrates stronger robustness on challenging samples containing abundant sarcastic or contradictory expressions. Extensive analytical experiments further validate the model’s effectiveness in dynamic relation perception, modal bias disentanglement, and fine-grained emotional cue capture. The main contributions of this paper are summarized as follows: 1. We are the first to identify the issues of static suppression limitations and language modal bias in current language-guided multimodal fusion frameworks, offering a novel solution from the perspective of causal dynamic interaction. 2. We propose a Dynamic Multimodal Causal Disentanglement and Adaptive Fusion Framework, comprising: 1) a causal disentanglement module for removing language bias; 2) a dynamic router for real-time perception and routing of multimodal interactions; 3) a generative denoising fusion refiner for joint representation refinement. 3. We achieve breakthrough performance on multiple public datasets and provide systematic interpretability analyses, offering deep insights into how dynamic interaction and bias disentanglement are crucial for advancing multimodal sentiment understanding. The outcomes of this research provide new theoretical foundations and practical tools for building more robust and intelligent multimodal sentiment analysis systems. II Related Work Multimodal Sentiment Analysis (MSA) has evolved significantly, with research efforts primarily advancing along three interconnected fronts: the design of fusion mechanisms, the learning of modality representations, and the handling of cross-modal dynamics. This section situates our work within this broader landscape. II-A Evolution of Multimodal Fusion Paradigms Early MSA research focused on designing fusion architectures to combine unimodal features. Early fusion methods, such as Tensor Fusion Networks (TFN) (Zadeh et al., 2017) [ 9 ] and its low-rank approximation LMF [ 10 ] (Liu et al., 2018), combined features at the input or model level but often struggled with modality asynchrony. Late fusion aggregated decisions from separate unimodal classifiers but ignored low-level interactions. The advent of the Transformer propelled a shift towards hybrid and hierarchical fusion, enabling more flexible interaction modeling. MulT [ 11 ] (Tsai et al., 2019a) employed directional cross-modal attention to handle unaligned sequences, while MISA [ 12 ] (Hazarika et al., 2020) pioneered a paradigm of learning both modality-invariant and modality-specific representations via adversarial and reconstruction losses, highlighting the importance of representation learning prior to fusion. Subsequent works like Self-MM [ 13 ] (Yu et al., 2021) and MMIM [ 14 ] (Han et al., 2021) further integrated self-supervised objectives and mutual information maximization to learn richer and more coordinated multimodal representations. However, a common thread among these methods

Read the complete text