Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits

Pranuthi Tenali 2 1 , Sahil Sidheekh 2 1 , Saurabh Mathur 4 1 , Erik Blasch 3 , Kristian Kersting 4 , Sriraam Natarajan 2 · 2026

Read Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits by Pranuthi Tenali 2 1 , Sahil Sidheekh 2 1 , Saurabh Mathur 4 1 , Erik Blasch 3 , Kristian Kersting 4 , Sriraam Natarajan 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26629.txt.

Introduction

Tenali and co-authors present C2MF, a fusion framework that judges each source's credibility in context before merging multimodal data. Sources can clash when a sensor degrades or one class gets corrupted, and fixed trust rules cope poorly. Their method scores per-instance reliability with a conditional probabilistic circuit. A KL-based measure, computed exactly from the circuit, formalises that trust score. On a new Conflict benchmark with planted cross-modal clashes, accuracy rises by up to 29% over static-trust rivals in heavy noise.

Catalog record

CREATORPranuthi Tenali 2 1 , Sahil Sidheekh 2 1 , Saurabh Mathur 4 1 , Erik Blasch 3 , Kristian Kersting 4 , Sriraam Natarajan 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39526
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26629

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Context-specific Credibility-aware Multimodal Fusion with Conditional Probabilistic Circuits Pranuthi Tenali 2 1 , Sahil Sidheekh 2 1 , Saurabh Mathur 4 1 , Erik Blasch 3 , Kristian Kersting 4 , Sriraam Natarajan 2 Abstract Multimodal fusion requires integrating information from multiple sources that may conflict depending on context. Existing fusion approaches typically rely on static assumptions about source reliability, limiting their ability to resolve conflicts when a modality becomes unreliable due to situational factors such as sensor degradation or class-specific corruption. We introduce C 2 MF , a context-specfic credibility-aware multimodal fusion framework that models per-instance source reliability using a Conditional Probabilistic Circuit (CPC). We formalize instance-level reliability through Context-Specific Information Credibility (CSIC), a KL-divergence–based measure computed exactly from the CPC. CSIC generalizes conventional static credibility estimates as a special case, enabling principled and adaptive reliability assessment. To evaluate robustness under cross-modal conflicts, we propose the Conflict benchmark, in which class-specific corruptions deliberately induce discrepancies between different modalities. Experimental results show that C 2 MF improves predictive accuracy by up to 29% over static-reliability baselines in high-noise settings, while preserving the interpretability advantages of probabilistic circuit-based fusion. I Introduction Decision making in real-world domains requires fusing information from diverse modalities of available data for robust and reliable reasoning [ 1 , 2 ] . In safety-critical domains such as autonomous navigation, industrial robotics, and medical decision support, the redundancy provided by multiple sensors is expected to mitigate the failure of any single modality. However, the presence of multiple sources can also introduce a fundamental challenge in determining which modality to trust, when the sources provide conflicting information . Fig. 1 : Credibility-Aware Fusion using Latent Context . Unlike static fusion methods, the C 2 MF framework extracts a joint neural embedding that serves as a context for a Conditional Probabilistic Circuit. The circuit dynamically evaluates the credibility of each source based on its position in the latent space (i.e., identifying when one modality is unreliable), ensuring that the final prediction is dominated by the most reliable information source for that specific instance. A common strategy for resolving such conflicts is to model meta-information about each source’s reliability [ 25 , 4 ] . In such fusion methods, each modality is assigned a weight reflecting the expected credibility or trustworthiness of its information, and the final predictions are aggregated accordingly. These reliability estimates are typically learned from data and treated as global properties of the modality. While effective under stationary conditions, this global assumption breaks down in real-world settings where the reliability of a modality is inherently context-dependent [ 3 , 32 ] . For example, a high-resolution camera may be highly reliable in daylight but quite untrustworthy in a low-light tunnel. Similarly, a microphone may provide clear signals in a quiet room but fail in a crowded street. When two highly confident sources disagree, a static reliability model has no principled way to resolve the conflict, increasing the risk of errors. This observation motivates the central question of our work: How can a fusion system dynamically infer the context-specific credibility of a modality at an instance level, while retaining principled probabilistic reasoning and semantics? Addressing this question effectively requires building a multimodal fusion system that satisfies the following desiderata: (D1) It must infer the context directly from the data. (D2) It must dynamically adapt source reliability on an instance-by-instance basis, based on the inferred context. (D3) It must remain tractable, providing exact and explainable reasons for its decisions. Recent deep learning based fusion architectures often rely on attention or gating mechanisms to reweight modalities. While powerful, these approaches typically produce opaque intermediate representations and do not provide an explicit probabilistic interpretation of modality credibility. In contrast, probabilistic circuit (PC) based fusion [ 29 ] has emerged as a promising way to combine information from multiple modalities effectively, while also enabling principled and probabilistic credibility measures via exact and tractable inference over predictive distributions. Such systems have also been shown to be more robust and reliable compared to neural counterparts, yielding well-calibrated classifiers, especially in noisy settings [ 30 ] . However, existing PC-based approaches model reliability as a static property, independently of any contextual variables, and are therefore unable to resolve context-specific conflicts. In this work, we extend credibility-aware fusion to the context-conditional setting and propose C 2 MF (Context-Specific Credibility Aware Multimodal Fusion) , a hybrid framework that combines deep neural unimodal encoders with Conditional Probabilistic Circuits (CPCs) [ 26 ] as the fusion function. Our approach extracts a joint neural embedding from the input modalities, which serves as a latent context. This context is then used to parameterize the weights of the PC using a hyper-network, allowing the fusion logic to adapt dynamically for making reliable predictions, while preserving tractability for exact probabilistic inference and instance-specific credibility assessment. Figure 1 illustrates the overall architecture. We also generalize the PC-based credibility measure from [ 29 ] to the context-conditional setting. We define Context-Specific Information Credibility (CSIC) as the KL divergence between the full multimodal posterior and the posterior with a modality marginalized out, both conditioned on the joint context z . This recovers the prior static credibility measure [ 29 ] as a special case when the context carries no additional information about modality reliability. To evaluate our C 2 MF framework, we identify a lack of existing benchmarks for context-dependent conflict resolution and introduce the Conflict benchmark. In this benchmark, we intentionally corrupt specific modalities based on class labels to induce structured cross-modal disagreement. We also propose the Reliable Modality Identification Score (RMIS) , a metric that directly measures whether a model correctly identifies the more reliable modality under conflict. Thus, overall we make the following key contributions: (1) We introduce C 2 MF, a CPC-based fusion framework that dynamically conditions credibility inference on a learned latent context, enabling dynamic instance-level reliability modeling while preserving exact probabilistic semantics. (2) We generalize the probabilistic divergence-based credibility measure definitions to this conditional setting. (3) We propose a structured corruption framework and introduce a new benchmark for evaluating fusion approaches on their ability to handle context-specific corruptions and cross-modal conflicts. (4) We propose RMIS, a new evaluation metric that directly quantifies a model’s ability to identify the reliable modality under conflict. (5) We demonstrate through extensive experiments that C 2 MF substantially improves predictive performance and reliable modality identification compared to static baselines. II Background and Related Work II-A Multimodal fusion and reliability Multimodal fusion [ 10 , 1 , 16 ] aims to integrate information from M distinct sources, \mathbf{X}=\{X_{1},\dots,X_{M}\} to improve the accuracy and robustness of a target prediction Y\in\mathcal{Y} . Formally, given a labelled dataset \mathcal{D}=\{(x^{(i)}_{1},\dots,x^{(i)}_{M},y^{(i)})\}_{i=1}^{N} these methods aim to learn a function f to model the conditional distribution over the target given multimodal data, i.e., P(Y\mid x_{1},\dots,x_{M})=f(x_{1},\dots,x_{M}). Fusion architectures are commonly classified as early (data-level), intermediate (feature-level), and late (decision-level). Early/Signal Fusion [ 33 , 9 ] integrates raw features or input data at the earliest stage, typically by concatenating (x_{1},\dots,x_{M}) into a single input vector and learning P(Y\mid\mathbf{x})=f(x_{1}\oplus\dots\oplus x_{M}) . While this allows the model to capture low-level correlations between sensors, it is highly sensitive to noise and makes reasoning about the contribution of individual

Read the complete text