Interpretable graph-based models on multimodal biomedical data integration: a technical review and benchmarking Authors: Alireza Sadeghi, Farshid Hajati, Ahmadreza Argha, Nigel H Lovell, Min Yang, Hamid Alinejad-Rokny Published: 2026 ABSTRACT AbstractIntegrating diverse biomedical modalities is essential for robust healthcare insights, and graph-based models are increasingly used to capture complex relational structures. Yet, their clinical translation hinges on interpretability. This review surveys interpretable graph-based models applied to multimodal biomedical data, highlighting dominant trends in disease classification, static graph construction, and post-hoc explainability. We categorize explainable artificial intelligence (XAI) techniques, benchmark SHAP, saliency, sensitivity, and graph masking on Alzheimer’s disease data, and reveal complementary strengths. A development flowchart and future directions, such as dynamic graphs, knowledge integration, and LLM-based explainability, position this work as a key reference for trustworthy biomedical AI.Subject terms: Data integration, Machine learning, Alzheimer's disease Introduction Deep learning (DL) has revolutionized biomedical research through its ability to process complex data and extract meaningful patterns1. In healthcare, DL models have improved diagnostics2, treatment planning3, and drug discovery4, while also enabling personalized medicine through predictive risk modeling5. However, conventional DL approaches, such as convolutional neural networks (CNNs) and fully connected neural networks, are often limited to homogeneous data and struggle to integrate diverse modalities, like imaging, clinical text, and genomics, critical for comprehensive clinical insights6–8. Graph-based models offer a promising alternative by naturally encoding complex, non-Euclidean relationships via nodes and edges. Beyond applications to single-modality data, such as medical imaging9–12 or omics13–15, they are particularly well-suited for integrating multimodal biomedical data, capturing both intra- and inter-modality interactions. Despite these advantages, the adoption of such models in clinical settings is hindered by their limited interpretability16. Their “black-box” nature raises concerns around transparency, accountability, and regulatory acceptance17,18, highlighting the urgent need for explainable approaches that support clinical trust and informed decision-making. This review aims to address that need by presenting a comprehensive technical survey of interpretable graph-based models in multimodal biomedical data analysis. Our contributions include: (i) a systematic review of studies incorporating graph-based models and interpretability techniques; (ii) a categorization and benchmarking of explainable AI (XAI) methods; (iii) a step-by-step development guide for building interpretable graph-based models; and (iv) a detailed outlook on future research directions in this rapidly evolving field. Survey and benchmarking of XAI strategies Graph-based models have become integral in analyzing multimodal biomedical data for various regression and classification tasks related to cancer analysis, disease diagnosis, and biomolecular interactions. Table 1 provides an overview of these models, including deep neural networks (DNNs), graph neural networks (GNNs), graph isomorphism networks (GINs)19, graph convolutional networks (GCNs)20, graph attention networks (GATs)21, and graph transformer networks (GTNs)22, along with their key characteristics. Table 1The summary of various graph-based models employed in the surveyed studies, alongside their respective formulasModelExplanationFormulaDenotationDNNIn graph-related studies, a feedforward framework is built using the nodes and edges of the graph as input.zn=ϕn(WnXn+bn)zn: the output of the neuron n. ϕn: the activation function applied to the neuron n. Xn: the input to the neuron n. Wn and bn: the weights and bias inputted to the neuron n.GNNA framework for updating each node’s features by aggregating the features of its neighboring nodeshil+1=COMBl(hil,AGGlhjl:j∈Ni)hil+1 and hil: node i’s feature vector in layer l+1 and l, respectively. COMBl: the combination function in layer l. AGGl: the aggregation function in layer l. Ni: node i’th neighborhood.GIN19An architecture with adjustments to the GNN, to make it more powerfulhil+1 = MLPl(1+ϵl.hil+∑j∈Nihjl)hil+1 and hil: node i’s feature vector in layer l+1 and l, respectively. MLPl: a multi-layer perceptron in layer l. ϵ: a learnable parameter or a fixed scalar. Ni: node i’th neighborhood.GCN20A framework for updating the features of graph nodes through convolutional filtering applied to the input graph signalsHl+1=ϕl(AHlθl)Hl+1 and Hl: nodes’ feature vector in layer l+1 and l, respectively. ϕl: non-linear activation function of layer l. A: the adjacency matrix. θl: layer l’s convolutional filter’s weight matrix.GAT21An attention-based architecture for computing the hidden representation of each node by considering the importance of features from other nodesαij=softmaxj(aWhi,Whj)αij: attention coefficient of node j’s features to node i. a: the shared attention mechanism (single-layer feedforward NN). hi: node i’s feature vector. W: weight matrix of the shared linear transformationGTN22A framework similar to natural language processing transformers but made for graph inputshil+1=Ohl(Hk=1∑j∈Ni(wi,jk,lVk,lhjl))hil+1 and hil: node i’s feature vector in layer l+1 and l, respectively. Ohl: an additional linear transformation in layer l. ||: concatenation operation. k: the attention head (from 1 to H heads). wi,jk,l: the weight of the k’th attention head calculated by features from node i and j in layer l. Vk,l: the value vector of node i in the k’th attention head in layer l. Ni: node i’th neighborhoodWe also provided a detailed summary of all reviewed papers in Table S1, including the modalities used, graph construction strategies, interpretability techniques applied, and the primary objectives of each study. Additionally, we categories graph-based modeling approaches and the studies in which they were implemented in Table S2. The summary of various graph-based models employed in the surveyed studies, alongside their respective formulas We also provided a detailed summary of all reviewed papers in Table S1, including the modalities used, graph construction strategies, interpretability techniques applied, and the primary objectives of each study. Additionally, we categories graph-based modeling approaches and the studies in which they were implemented in Table S2. Fig. 1a illustrates the architectural schematic of these models, while Fig. 1b displays their distribution across surveyed studies. These models are designed primarily to accomplish two main objectives: performing graph-level tasks or node-level tasks (refer to Fig. 1c, d). In graph-level tasks, each sample in the raw dataset is represented as a graph. Labeled graphs, derived from labeled samples, are used to train the graph-based model. Subsequently, this trained model is employed to make predictions for unlabeled graphs (samples). On the other hand, in node-level tasks, each sample is represented as a node within the constructed graph. The objective is to generate predictions for unlabeled nodes by leveraging information from the labeled nodes in the graph. This approach focuses on predicting individual nodes rather than the entire graph structure. An overview of graph-based models. a Architecture showcasing various graph-based models utilized in studies on multimodal data. b Distribution of different graph-based models in the reviewed studies (n = 31). c The process of implementing graph-based models for graph-level analysis. d The process of implementing graph-based models for node-level analysis. DNN deep neural network, GNN graph neural network, GIN graph isomorphism network, GCN graph convolutional network, GAT graph attention network, GTN graph transformer network. Graph construction in the reviewed studies Graph-based models rely on structured input where a graph G=(V,E), consists of nodes V and edges E, with node features FV, derived from input modalities. During training, nodes update by aggregating information from neighbors, enabling predictions based on local interactions. Node definitions vary by study objective and modality. Brain imaging studies define brain regions as nodes23–33, while genomic and cancer studies use gene or proteins level29,34–40, and patient-level prediction tasks (e.g., Alzheimer’s or COVID-19) represent each patient as a node 41–49. Most studies adopt static graph construction, often based on prior biological knowledge (e.g., protein–protein, gene–gene interactions)34–37,40,50 or similarity functions like Pearson correlation23,24,26,29,32,43,51, Euclidean distance27,31,44,47,52, and cosine similarity32,42,48. Edges are either thresholded or selected via K-nearest neighbors. Some task-specific strategies also exist, such as Kan et al.’s30 dMRI-based connectivity. A few recent studies explore dynamic graph construction, adapting edge definitions during training to better capture evolving relationships. These dynamic approaches aim to improve performance by overcoming limitations of fixed graph structures.46,49. Figure 2 presents an overview of the methods employed by different studies to construct graphs in their research, along with distributions of the techniques utilized. Overview of various techniques employed in different studies to construct graphs for their research. a Distribution of various connection methodologies in graph construction utilized by surveyed studies. b Distribution of various similarity functions used to draw edges between nodes in the constructed graphs. c Distribution of different entities implemented as nodes in the constructed graphs. Interpretation in multimodal biomedical data integration DL models, including graph-based ones, are often viewed as black-box systems, which poses a significant challenge when deploying them in trust-sensitive fields like healthcare. To tackle this challenge, researchers in the surveyed studies have explored various approaches to make these models explainable. We examined all the methods implemented in existing studies and categorized them into the following four categories: category I: modality or feature elimination; category II: model-agnostic XAI; category III: graph-based XAI; category IV: inherent interpretability. Table S3 provides an overview of studies that employed one of the aforementioned categories to enhance the model interpretability. In the following subsections, we explore each category in detail, discussing their specific use cases. For the fourth category, which focuses on developing inherently interpretable graph-based models, we provide a step-by-step guide to assist researchers in creating transparent graph-based models in their studies. Finally, we conclude this section with a benchmark analysis, where we use methods from each category on a specific graph-based medical study to practically evaluate the strengths and weaknesses of each interpretability category. It is worth noting that beyond the techniques employed in the reviewed studies, there are additional techniques that may fall under the categories of model-agnostic XAI53,54, graph-based XAI55–57, and inherent interpretability58–60 for enhancing the explainability of graph-based models. However, since these techniques were not utilized in multimodal biomedical data integration, they are not included in the following subsections. Category I: modality or feature elimination In the context of graph-based models applied to multimodal medical data, a primary focus for researchers is determining the significance of each modality. They seek to understand whether the utilization of multiple modalities enhances their model’s performance compared to scenarios involving a single modality, and if so, to what extent. A common technique employed by most of the reviewed studies involves removing a specific modality and evaluating the model’s performance without the features from that specific modality (Fig. 3). The degree of performance decline indicates the importance of that modality to the model’s overall performance. For instance, Yang et al.23 substituted the features from one modality with dummy variables and evaluated the model’s performance, while other reviewed studies opted to eliminate a certain modality, retrain the model, and recalculate its performance to demonstrate the significance of that particular modality. Implementation of the modality elimination technique to assess the importance of each modality in a graph-based model. In this technique, one modality is removed at each step, and the model is retrained using the remaining modalities. This process allows for the calculation of the model’s performance without the influence of the eliminated modalities. By comparing the decline in performance for each modality elimination, insights into the importance of each modality are obtained. Note that eliminating a modality may alter the structure of the graph-based model. Although modality or feature elimination is not a classical XAI method, it is sometimes considered under the umbrella of interpretability techniques because it provides empirical evidence of input importance by assessing performance changes upon input removal. Nevertheless, it differs from post-hoc methods that analyze internal model behavior without modifying the input space. However, modality elimination provides only a basic insight into the importance of each modality. For a comprehensive analysis, it is crucial to evaluate the model across all possible combinations of modalities to demonstrate the significance of that particular modality. Moreover, this method serves as an initial step in several model-agnostic XAI techniques like SHAP61. Despite some similarities, there are key differences that set them apart. In modality elimination, one modality is removed, and the model is retrained from scratch on the remaining modalities. This retraining process, which focuses exclusively on the remaining features, is not employed in SHAP and similar methods. Therefore, modality elimination can be considered as a distinct category. Modality elimination can introduce a limitation by potentially altering the graph structure. This alteration is particularly significant when the graph structure relies on the similarity between node feature vectors. Such changes can skew the analysis of modality importance, leading to unfair conclusions. To delve into it further, let us consider a feature vector FVn=f1n,f2n,…,fmn, where FVn represents the feature vector of the n-th node and fin represents features derived from the i-th modality. To gauge the effect of eliminating the i-th modality, we create a new feature vector FV−in=f1n,f2n,…,fi−1n,fi+1n,…,fmn which includes information from all modalities except the i-th one. After creating this new feature vector for all nodes, the similarity between nodes will change. This means that nodes n and k that were initially similar may no longer be similar, and vice versa: Here, Sim⋅ represents the similarity function between two vectors, and τ represents the threshold below which nodes with similarity values are considered neighbors. Researchers face two choices in this context: (i) they can reconstruct the graph structure based on the new similarities, or (ii) they can retain the original graph structure and continue with the analysis. In the first scenario, where the graph structure is reconstructed, comparing the performance of the graph-based model on this new graph with the original one would not provide a fair assessment of the eliminated modality’s impact. However, in the second scenario, maintaining the structure of the initial graph poses a risk of connecting dissimilar nodes. This could confuse the model by incorporating information from irrelevant neighbors, potentially leading the model away from its optimal performance. Therefore, removing modalities in graph-based studies, particularly when edges are determined by node features, is not a fair method for assessing the impact of each modality. Category II: model-agnostic XAI In addition to evaluating the importance of each modality, several studies have focused on determining the significance of various components of their constructed graph. This includes assessing the importance of an individual node, the edges between nodes, and features derived from different modalities for each node. To achieve this, researchers have employed several post-hoc XAI techniques originally developed for general machine learning, rather than specifically for graph-based models. For example, the saliency map technique62,63, which relies on gradient calculation, was used by several studies26,28,29,51 to assess the importance of each node’s feature. Bi et al.29 also used this technique to determine the importance of graph nodes, where each node represents a specific brain ROI. Another gradient-based XAI technique, which integrated gradient analysis64, was used by Huo et al.39 to unveil the importance of each genomic feature employed in their study. Additionally, Qu et al. used Grad-RAM24, another gradient-based technique, to identify the importance of each node in the graph, where each node also represents a specific brain ROI. Qu et al.24,33 further provided edge-level interpretability using an edge-masking technique, which involved systematically removing one edge at a time from all patient graphs and retraining the model to observe the resulting performance changes, thereby determining the significance of each edge. Zhang et al.35 and Pfiefer et al.37 used SHAP61, a technique based on Shapley values, to illustrate the importance of each feature and node, respectively. Sensitivity analysis65, a common technique to determine feature importance in neural networks, by multiplying network weights and feature standard deviation, was used by Li et al.44 to identify feature importance in their study. Figure 4 provides detailed visualizations of each of these techniques. Table S4 summarizes the various model-agnostic XAI methods along with their corresponding formulae employed in different studies. An overview of various model-agnostic XAI techniques used in graph-based studies to gain insights from the model. a Gradient-based method: this technique is used to assess the importance of different components in a graph-based model. It involves calculating the gradient of the model’s predicted value with respect to various elements, such as graph nodes, edges, input features, or modalities. By analyzing these gradients, researchers can identify which components most significantly influence the model’s predictions. b Sensitivity analysis: this is a common method for determining the importance of features in a neural network. It involves calculating the standard deviation of a specific feature across all samples and multiplying it by the sum of the weights associated with that feature in the trained network. This product reflects the feature’s overall importance to the model’s performance. c SHAP: SHAP is a widely used method for enhancing model interpretability. It calculates the model’s performance with and without specific modalities (e.g., brain ROIs on the left side). The difference in performance when a modality is included versus when it is excluded is computed across all possible modality combinations. These differences are then combined linearly to quantify the contribution of each modality to the model’s overall performance. Category III: graph-based XAI While post-hoc model-agnostic XAI techniques can offer valuable insights into the importance of various components in graph-based studies, it is important to recognize their limitations. Gradient-based XAI techniques, for example, have been shown to be misleading in certain scenarios66 and can suffer from issues like gradient saturation64,67. Additionally, these techniques are not specifically designed for graphs where inputs are often discrete, which further limits their effectiveness when applied to graph-based models68. To overcome these challenges, researchers have employed XAI techniques specifically tailored for graph-based models. Zhou et al.27 employed a general graph masking technique to identify the most significant subgraph and subset of features within their model. They specifically developed optimal masks for both the graph structure and the feature vector, allowing them to distinguish the most important subgraph and feature subset. Additionally, GNNExplainer68, another masking-based technique, was employed by Pfeifer et al.36, Tang et al.52, and Patel et al.32 to pinpoint the most crucial nodes and features within a graph during the prediction process for a specific sample. The primary difference between the general graph masking technique and GNNExplainer lies in their scripts. While GNNExplainer focuses on the construction graph of a particular node, essentially a subset of the original graph that directly influences the output for that node, the general graph masking technique considers the entire original graph. GNNExplainer offers computational advantages, especially in practical applications involving large graphs with potentially millions of nodes. Another technique used to determine the importance of each node in an input graph is the top-k pooling technique69. In this method, the importance score of graph nodes is computed using multi-layer perceptron (MLP) models. This approach, which offers node-level interpretability, was applied by Sebenius et al.25 to brain ROIs to identify the most important ROIs for diagnosing patients with schizophrenia. The details of the graph-based XAI techniques are also demonstrated in Fig. 5, while Table S5 summarizes different graph-based XAI techniques along with their corresponding formulae employed in the surveyed studies. An overview of graph-based XAI techniques used to gain insights from graph-based models. a Graph masking: this technique involves applying a mask (either an edge mask or a feature mask) to the original graph structure. The mutual information between the masked graph and the true label distribution is then calculated. The masked graph with the highest mutual information value is identified as the most important or informative graph. b GNNExplainer: similar to the graph masking method, GNNExplainer applies masks to the graph to uncover important components, such as nodes, edges, or features, that significantly influence the model’s predictions. c Top-k pooling: in this method, each node’s features are passed through a single-layer MLP to calculate the importance of each node. The nodes are then ranked based on their importance scores, and the top-k nodes are selected to construct the final graph. XAI eXplainable artificial intelligence, GNN graph neural network, MLP multi-layer perceptron. Category IV: inherent interpretability While post-hoc XAI techniques are valuable for gaining insights into the components of complex models, interpretable models offer inherent transparency by allowing tracking of the data-processing steps. This transparency enables understanding of how different components collaborate to generate predictions, clearly showing how features, nodes, and edges contribute to the model’s outputs. In the realm of multimodal medical data integration using graph-based models, several studies have developed inherent interpretable models. The elements used in these studies can be classified into the following three categories. Attention mechanism: several studies integrated attention mechanisms, which assign higher attention weights to more important components. In the study conducted by Yang et al.23, graph inputs are first processed through a multi-head attention layer to compute attention weights for each node. By employing gradient sensitivity analysis to these weights, the study reveals specific features and their interactions pertinent to the classification process. In another study, Ouyang et al.49 introduced an attention layer within their model to elucidate the significance of each modality. They also incorporated inner product regularization on the feature indicator matrix to emphasize the more critical features within each modality. Similarly, Huo et al.39 also employed attention mechanisms to identify significant clinical and pathological features within the corresponding graphs in their study. In another work, Ma et al.40 utilized the attention mechanism in their model to determine the importance of genes within specific cells. Kazi et al.46 developed IAM, an attention-based component in their model designed to reveal the importance of each input feature. Additionally, the authors introduced a novel loss function tailored specifically for this component, enhancing the IAM’s ability to detect more important features. Lastly, in the model proposed by Bintsi et al.45, features are initially processed through an attention layer. Subsequently, the weighted features, constructed by multiplying feature values, and the weighted features—created by multiplying feature values by their corresponding attention weights—serve as the feature vector for each node (patient). Once the model is fully trained, the attention weights reveal the importance of each feature, identifying the most crucial ones. Graph attention: beyond the general attention layers used in the above studies, some researchers incorporated graph-specific attention mechanisms to enhance the interpretability of their model. In the study by Keicher et al.41, they used a GAT21, where the attention weights shed light on how neighboring nodes (patients) influence the classification of a specific node (patient). This approach is particularly valuable in pandemic scenarios, such as the COVID-19 study they conducted, offering valuable insights into population interactions and potential outcomes for individual patients. Similarly, Safai et al.28 also employed GAT to understand the significance of nodes (brain ROIs) in the model’s prediction, aiming to elucidate the interactions between various brain regions. Additionally, Li et al.50 used GAT to identify the importance of different nodes (drug-protein pairs) in their study. Xiao et al.48 leveraged GAT in their proposed model to determine each node’s (patient) contribution to the prediction of outcomes for other patients. Additionally, Kaczmarek et al.22 used the graph transformer, another graph-specific attention mechanism, to analyze the significance of interactions between nodes (specifically mRNA–miRNA interactions) in detecting various types of cancers. They further calculated the average edge weights associated with each node to identify key nodes (mRNA or miRNA) relevant to specific cancer classification tasks. Other methods: some researchers have developed interpretable graph models without relying on attention. For example, Kan et al.30 used network weights to evaluate node and feature importance, while Bi et al.47 assessed filter weights in graph convolution layers. Pfeifer et al.37 integrated decision trees, leveraging their inherent interpretability to highlight modality and feature contributions. Model interpretability strategies often depend on the study’s objective. Benchmarking existing strategies We conducted a benchmarking analysis to evaluate the performance and practicality of the previously described XAI techniques when applied to a graph-based model. Specifically, we focused on MOGONet, a graph-based model developed by Wang et al.42 for biomedical classification tasks across various diseases. In our study, we replicated the MOGONet architecture using the ROSMAP dataset70, which is curated for Alzheimer’s disease (AD) classification. The dataset comprises features from three distinct modalities, including mRNA expression data, DNA methylation data, and microRNA expression data, each processed through its respective graph-based model within the MOGONet framework. Final classification outcomes were obtained by integrating the predictions generated by these individual models. To assess feature importance, the original study in MOGONet employed a modality elimination technique (category I), wherein the performance impact of excluding specific features was measured. Building on this approach, we extended the analysis by implementing additional XAI techniques from categories II and III to gain deeper insights into the relative significance of the dataset’s features. Category IV methods were excluded from this analysis, as these inherently interpretable models are not model-agnostic and thus cannot be retrofitted onto pre-existing architectures like MOGONet. For this benchmarking analysis, we selected gradient saliency Map, SHAP, and sensitivity analysis from category II. From category III, we incorporated graph masking, a generalization of GNNExplainer, into the analysis due to its broader applicability in identifying important features and subgraphs. However, the top-k pooling method, which is specifically designed to assess node importance, was excluded as it is not directly applicable to feature-focused analyses. After training, we used the trained model’s parameters and the feature distributions from the test set to calculate the importance of each feature using sensitivity analysis. This technique was relatively fast because it didn’t require backpropagation or additional model training. To compute feature importance using gradient saliency maps, we calculated the gradient of the model’s output for each test sample with respect to each feature. The feature importance was revealed by averaging these gradients across all test samples. For the graph masking technique, we aimed to find a mask that, when applied to the graph, maximized mutual information with the non-masked graph. We started with a randomly initialized mask and optimized it over 100 epochs using the binary cross-entropy cost function. This method was more time-consuming than the previous two, as we needed to find the optimal mask for each test sample individually. Finally, we used Kernel SHAP to compute feature importance. We generated 100 weighted neighbors for each test sample, with the weights calculated using the equations proposed by Lundberg et al.61. By training a logistic regression model on these generated samples, we determined the feature importance for each sample. For global interpretability, we averaged these importance values across all patients. We first compared the top genes identified by each XAI method. As Fig. 6a showed, all four methods—SHAP, sensitivity analysis, gradient saliency, and graph masking—identify partially overlapping yet distinct sets of top genes, reflecting the inherent variability in how each explainability algorithm quantifies feature importance. SHAP and sensitivity analysis share more genes in common (e.g., CTB-171A8.1, RP11-552D4.1, and OR7A5), consistent with both methods having a relatively broad coverage of biologically relevant signals. Gradient saliency highlights genes, such as NPNT, SYTL1, and ANKRD30B, which do not appear prominently in the other lists, suggesting their sensitivity to different aspects of model gradients. Graph masking surfaces genes associated with membrane transport and cell signaling (SLC25A18, GPIHBP1, CLCA4, and SCD), reflecting its unique strategy of learning a mask that pinpoints the most relevant subsets of the graph structure. Benchmarking analysis. a Top 30 features captured by different XAI techniques from categories II and III are considered the most important features for Alzheimer disease classification. As shown in the figure, SHAP and gradient saliency were roughly able to detect a combination of the most important genes found by other techniques, as the most important genes for Alzheimer disease classification. b A permutation analysis was performed on the important features identified by each technique. Specifically, 30 features were randomly chosen from the entire feature set, and their average importance scores were calculated. This random selection process was repeated 1000 times, creating a distribution of average importance scores, which was visualized in a histogram. The average importance score of the top 30 ranked features (shown by a red dotted line) was then compared to the mean of the permutation-based score distribution to assess the significance of these selected features. The results show that the gradient saliency technique outperformed the other techniques. c In the GO analysis, we identified several GO terms, retaining only those with a p ≤ 0.01. These terms, along with their p-values and the XAI techniques that identified them, are shown. d Pathway analysis was also conducted, keeping only pathway terms with a p ≤ 0.01 and an overlap >1; notably, none of these terms were identified by gradient saliency. The retained pathway terms, the XAI techniques that found them, and their p-values are displayed. XAI eXplainable artificial intelligence, SHAP SHapley Additive exPlanations GO gene ontology. To assess how each interpretability method’s top 30 genes compared against random chance, we calculated the average importance score of those top 30 genes and then generated a null distribution by repeatedly (1000 times) sampling 30 genes at random (Fig. 6b). In the figure, the ratio indicates how far this observed average lies from the mean of the random distribution. Notably, gradient saliency shows the largest offset, implying that its top-ranked genes are especially unlikely to have been identified by chance, whereas sensitivity analysis yields a smaller offset but still remains distinctly above the random null distribution. Graph masking and SHAP occupy intermediate positions, further underscoring that each algorithm captures important gene sets in different ways. We then conducted gene ontology (GO) analysis on the top genes identified by each XAI technique and extracted the corresponding GO terms, keeping only those with p ≤ 0.01. Figure 6c presents the GO terms alongside the XAI techniques that identified them. As the figure shows, SHAP identified ten out of eighteen Alzheimer-related GO terms (unique GO terms identified by all methods), while sensitivity analysis identified six. In comparison, graph masking and gradient saliency each detected only two Alzheimer-related GO terms. Interestingly, each XAI method highlighted distinct yet sometimes overlapping GO terms relevant to AD. SHAP detected multiple AD-associated processes, including positive regulation of cytosolic calcium ion concentration, endocrine system development, and temperature homeostasis, each tied to neuronal dysfunction and metabolic dysregulation in AD. Sensitivity analysis captured partially overlapping GO terms (e.g., endocrine system development) while adding ensheathment of neurons, gliogenesis, and forebrain development, reflecting broader coverage of neurodevelopmental processes. Gradient saliency identified fewer terms, but still pinpointed positive regulation of cytosolic calcium ion concentration and carbohydrate transport, both linked to neuronal health and metabolic deficits in AD. Graph masking introduced unique findings like 14-3-3 protein binding, relevant to tau pathology, and actin filament, associated with cytoskeletal disruptions. We then repeated the same analysis on the 15 unique pathway terms identified by all the methods (we only considered those pathways with p ≤ 0.01). Notably, none of these retained pathways were identified by gradient saliency; seven were identified by SHAP, while both sensitivity analysis and graph masking methods identified six pathways (Fig. 6d). SHAP and sensitivity analysis both prominently identified pathways involving G protein-coupled receptor (GPCR) signaling (e.g., GPCR ligand binding, GPCR downstream signalling, class A/1 rhodopsin-like receptors, signaling by GPCR), processes closely linked to synaptic transmission and neurodegeneration in AD. Sensitivity analysis, in particular, highlighted dopaminergic neurogenesis and hematopoietic stem cell differentiation, reflecting broader aspects of neuronal and immune system development relevant to AD. SHAP, meanwhile, singled out the cAMP signaling pathway, another key mediator of neuronal plasticity and survival that has been implicated in AD pathophysiology. In contrast, graph masking emphasized more general transport pathways (e.g., SLC-mediated transmembrane transport, bile secretion, gastric acid secretion) and metabolic processes, which may indirectly relate to AD through systemic metabolic dysregulation but are less directly established in AD literature compared to GPCR signaling or dopaminergic function. Discussion, perspective, and future directions In this paper, we examined how graph-based models support multimodal medical data integration and enhance interpretability in tasks, such as disease classification. Through benchmarking various explainability techniques, ranging from post-hoc XAI to inherently interpretable models, we highlighted trade-offs between computational efficiency and explanatory depth. While gradient- and sensitivity-based methods offer quick insights into feature importance, techniques like SHAP and graph masking yield richer interpretations at a higher computational cost. Figure 7 presents a practical framework that we propose to guide researchers in integrating explainability into graph-based biomedical models. For node- and edge-level interpretability, attention mechanisms should be positioned closer to the output layers to highlight the components most influential in the final prediction. Conversely, for feature-level interpretability, attention layers are best applied earlier in the architecture to capture the relevance of input features. A general guideline for developing an interpretable graph-based platform on multimodal medical data. It is important to note that the purpose of interpretability and the study’s objective significantly influence the type of elements to use and their placement within the model. Building on this framework, models, such as GAT and GTN, offer practical implementations that help visualize node interactions and their predictive contributions. However, while inherently interpretable models can enhance transparency, they often require architectural simplifications that may reduce predictive performance. This highlights the inherent trade-off between model accuracy and interpretability. Currently, most studies still depend on general-purpose XAI tools like SHAP and gradient-based saliency methods. Only a limited number of works employ graph-specific interpretability approaches, such as GNNExplainer or construct models with built-in interpretability mechanisms. Researchers are encouraged to adopt a balanced approach, selecting methods aligned with the interpretability needs of their application, the complexity of the data, and the clinical or biological questions at hand. In addition to architectural challenges, practical deployment must also consider the efficiency and feasibility of interpretability methods. In real-world settings, limited time and computational resources necessitate careful selection of post-hoc XAI techniques. For instance, SHAP, while offering deep interpretability, can be prohibitively slow, taking weeks for high-dimensional data. As a result, inherently interpretable models are gaining traction. However, these often trade accuracy for transparency, requiring researchers to balance performance needs, computational costs, and interpretability demands71. Hybrid models, blending attention mechanisms or lightweight post-hoc methods, have shown promise in mitigating this trade-off72–74. In clinical applications, where actionable insights are critical, opting for higher interpretability at the expense of slight accuracy loss may be justified75. Crucially, interpretability is not just a technical preference; it underpins trust, accountability, and safety in healthcare by clarifying model rationale and supporting clinical decision-making. Beyond the design of individual models, recent innovations in how biomedical data is structurally represented also shape interpretability and model utility. To extend the utility of these graph models beyond traditional tasks, recent advances have proposed “graph-in-graph” strategies. In this setup, each data sample (e.g., a patient) is both a node in a global graph and a graph itself, capturing detailed molecular or imaging data. This hierarchical representation enables simultaneous patient-level and system-level reasoning, which can uncover patterns otherwise lost in flat graph structures76. For example, in cancer studies, each patient node could contain a subgraph of gene interactions or imaging-derived features. This dual-layer architecture allows researchers to identify both local disease markers and broader trends across the population, providing a richer and more nuanced understanding of disease progression and treatment response. While these architectural innovations expand modeling capability, fundamental technical challenges remain in deep graph learning. Deep graph models often suffer from over-smoothing, where node features become indistinguishable with increased message-passing layers. Rusch et al.77 showed that addressing this issue is vital for model performance. Another limitation, over-squashing, arises when information from distant nodes is compressed, impairing long-range dependency learning. Topping et al.78 linked this to graph topology and proposed an edge-based combinatorial curvature method as a mitigation strategy. Despite technical progress, the application of these models remains uneven across disease types and tasks. While interpretable graph-based models have shown promise in cancer analysis34,36,38,39, current studies largely focus on a few cancers like breast35,44 and kidney37, leaving others, such as prostate, skin, and thyroid, underexplored. Similarly, their application to disease diagnosis has been limited to COVID-19 [1941, Alzheimer’s27,29,46,47, and Parkinson’s28,43, highlighting opportunities to extend these methods to diseases like malaria, AIDS, and hepatitis. Most existing models address classification or regression tasks; expanding to image segmentation or object detection could uncover new challenges and validate generalizability. Such diversification requires flexible graph architectures capable of integrating heterogeneous data, e.g., imaging, sequences, and clinical variables, especially in settings where multimodal data availability and quality vary significantly. Table S6 summarizes disease areas explored to date. One factor that influences both performance and interpretability is how graphs are constructed. Existing studies have constructed graphs using either domain knowledge34–38 or similarity functions, such as cosine48,49, Euclidean44,46,47,52, and Pearson correlation24,26,29,43. Future work could leverage multimodal knowledge graphs or hybrid approaches combining knowledge and similarity-based edges. Exploring dynamic graph construction, where edges adapt based on disease context or patient subgroups, could uncover hidden patterns. While the effects of static vs. dynamic construction on explainability79 and performance80 have been studied individually, their combined impact remains underexplored. Additionally, tracking the evolution of the graph structure during the training process can provide insights into how the model updates over time, potentially guiding improvements in performance. Visualization tools, such as NetworkX with Matplotlib animation (Python), GraphStream (Java), or dg2pix81 (a pixel-based method), can support this type of analysis. While some studies have adopted post-hoc explainability tools specifically designed for graph networks, such as GNNExplainer68, the field still underutilizes other advanced graph-specific XAI techniques. Recent developments, including PGExplainer82 and CF-GNNExplainer83 offer promising alternatives that go beyond simple node- or edge-level attribution by identifying minimal subgraphs or generating counterfactual explanations. These methods can be more adaptable to diverse graph structures and potentially yield deeper, more localized biological insights. Future research should explore these alternatives more systematically, particularly in multimodal biomedical applications, where interpretability at the subgraph or pathway level is critical. To fully realize the potential of these advanced techniques, rigorous evaluation frameworks must be adopted. Most reviewed studies used explainability methods and qualitatively aligned their results with prior knowledge (e.g., ground truth). However, quantitative evaluation offers a more objective assessment of how well these techniques highlight meaningful model components. When ground truth is available, metrics like accuracy, F1, precision, specificity, and sensitivity can be used53,68. In its absence, metrics, such as fidelity, inverse fidelity, and sparsity, are preferred57. Fidelity measures the performance drop after removing important components; inverse fidelity evaluates performance using only those components; and sparsity penalizes overly large explanations. Ideally, effective methods achieve high fidelity, high sparsity, and low inverse fidelity (see more in the “Methods section”). While our reviewed studies employed spatial GNNs, where node representations are iteratively updated via message passing, it is equally important to consider spectral GNNs, which operate in the frequency domain using graph Fourier transforms. These models transform input signals into the spectral domain via the graph Laplacian, apply a learnable filter, and then invert the result back to the spatial domain. The effectiveness of spectral GNNs largely depends on the design of the filter, which modulates how frequency components are emphasized20,84. Bo et al.85 benchmarked different spectral filters across datasets, showing that while more advanced filters can enhance performance, they also introduce significant computational complexity, often limiting their practical adoption. Nonetheless, as spectral models mature, they may offer complementary strengths to spatial methods, especially in capturing global graph structures. Complementing spatial GNNs, spectral approaches offer alternative mechanisms for learning and interpreting graph structure. The explainability techniques reviewed earlier are largely tailored to spatial GNNs and are not directly applicable to spectral-based models. However, spectral GNNs offer intrinsic interpretability via their frequency-domain representation: the eigenvectors of the graph Laplacian define frequency components that the model selectively amplifies through learnable filters. Low, mid, and high frequencies, respectively, capture global patterns, balanced structures, and local variations. While this spectral insight offers a unique lens into model behavior, it lacks direct spatial interpretability, crucial for clinical applications. Bridging this gap remains an open research challenge in multimodal biomedical explainability. Our benchmarking revealed that each XAI method offers distinct advantages in uncovering Alzheimer’s disease (AD)-related biology. SHAP and sensitivity analysis consistently retrieved well-established GO terms and pathways, such as GPCR signaling, neurogenesis, and protein binding, making them particularly valuable for studies prioritizing biological interpretability. In contrast, gradient saliency and graph masking surfaced alternative insights, including 14-3-3 protein binding and metabolic transport, highlighting underexplored aspects of AD. Permutation testing confirmed that all methods outperformed random baselines, with gradient saliency producing uniquely distributed importance scores. Computationally, SHAP and graph masking were more resource-intensive, while gradient saliency and sensitivity analysis ran faster but offered more limited biological overlap. These findings suggest that combining multiple XAI approaches may yield a more comprehensive and nuanced understanding of AD pathophysiology by balancing interpretive depth with efficiency. However, challenges remain. Many reviewed studies focus narrowly on specific diseases or modalities and rely heavily on qualitative interpretations, limiting generalizability and rigor. In our benchmarking, the absence of human-annotated ground truth for important graph components constrained direct validation. Furthermore, while this review covered a broad range of multimodal studies, excluding single-modality work and computational bioinformatics may have overlooked relevant progress. Future work should strive to bridge these areas for a more unified framework of explainable graph learning. Emerging XAI methods, such as counterfactual reasoning86 and contrastive explanations87, offer promising directions by identifying minimal, actionable changes to node features or edges that shift model outputs. These techniques have the potential to enhance model transparency and support precision interventions, an essential step for translating graph-based models into high-stakes clinical settings. Future directions: LLM-integrated graph models Integrating large language models (LLMs) into graph-based biomedical analysis offers a promising step toward more dynamic and interpretable multimodal AI. LLMs excel at extracting structured knowledge from unstructured text, such as clinical notes88, scientific literature89, and patient records90, and can enhance graph construction by identifying relevant entities and relationships for node and edge definition91. This enables the development of context-aware, dynamically updated graphs that move beyond static similarity-based connections. LLMs also show potential in improving interpretability by generating natural language explanations. While current graph-specific XAI methods like GNNExplainer68 or graph masking provide numerical or visual outputs, they often lack semantic context. Prompting LLMs to explain model-predicted subgraphs or node clusters could bridge this gap, linking model insights to known pathways, disease mechanisms, or drug–gene interactions. This approach can make graph explanations more intuitive and clinically meaningful, fostering trust and utility among healthcare stakeholders. Beyond summarizing background knowledge, LLMs can enhance XAI by converting graph-based explanations into human-readable narratives that clarify not just what influenced a prediction, but why. This “explanation chaining” approach, illustrated in other domains by Huang et al.’s chain of explanation (CoE) prompting92, could be adapted for biomedical graphs to narrate how a sequence of molecular or clinical interactions leads to a specific diagnosis. LLMs can also provide meta-interpretations by explaining outputs from other XAI methods. For instance, they can articulate why SHAP scores highlight certain features or why a graph masking method identifies a particular subgraph. Gao et al.‘s DR.KNOWS93. exemplifies this by translating medical knowledge graphs into diagnostic narratives. Similarly, He et al.94 used LLMs to generate and process textual explanations in graph-based learning, enhancing node representations and transparency. Zhang et al.95 further showed how combining Grad-CAM with multimodal LLMs improves both interpretability and performance. Collectively, these approaches signal a future where LLMs serve as bridges between complex graph analytics and clinician-friendly, multimodal interpretability frameworks. LLMs can generate semantic embeddings from biomedical literature that, when aligned with numeric features like gene expression or imaging metrics, enhance graph-based models by incorporating both statistical and contextual relevance96. For example, two genes may be co-expressed and also frequently co-mentioned in Alzheimer’s studies, reinforcing their role in GNN message-passing. LLM–GNN integration also enables interactive querying, where clinicians can ask questions like “what markers predict advanced Parkinson’s disease?” and receive subgraph-based, literature-backed answers, bridging the gap between expert insight and model output. However, several challenges remain. LLMs are computationally intensive to train or fine-tune97, and may generate false or unsupported information (“hallucinations”)98, highlighting the need for validation pipelines. Privacy concerns in handling sensitive clinical data necessitate strong anonymization, while biases in training corpora raise fairness issues in clinical applications 99. In conclusion, advancing explainable, multimodal GNN frameworks, particularly through LLM integration, offers transformative potential, but it still requires careful validation, ethical safeguards, and methodological innovation to ensure reliable clinical translation. Methods and materials Methodology To gather relevant research papers, we used three primary sources: Google Scholar, Scopus, and PubMed. We conducted research using various combinations of the following keywords related to our study: Interpretable, explainable, or transparent graph networks Interpretable, explainable, or transparent graph networks Multimodal, integrated, multimodality, cross-modal, multi-source, integration, fusion, or heterogenous Multimodal, integrated, multimodality, cross-modal, multi-source, integration, fusion, or heterogenous Medical, biomedical, biological, healthcare, genetic, genomic data, clinical notes, electronic health records, neuroimaging, MRI, fMRI, PET, CT, proteomic, transcriptomic, metabolomic, pathology, radiology, or omics. Summary of the diverse biomedical data types used across the reviewed studies has been provided in Table S7. Medical, biomedical, biological, healthcare, genetic, genomic data, clinical notes, electronic health records, neuroimaging, MRI, fMRI, PET, CT, proteomic, transcriptomic, metabolomic, pathology, radiology, or omics. Summary of the diverse biomedical data types used across the reviewed studies has been provided in Table S7. An example of the queries used to identify the papers is provided in Table S8. This comprehensive search yielded a total of 3129 papers published between January 1, 2019, and June 30, 2025. Subsequently, we screened these papers by their titles, leading to the inclusion of 401 papers for further examination. The abstracts of these selected papers were reviewed in the next step, resulting in the identification of 138 papers for a more in-depth analysis. Upon analyzing the full-length papers, 107 of them were excluded for various reasons. Some papers were duplicates, others used graph-based models in different contexts unrelated to multimodal data analysis, and some failed to provide a clear explanation of the interpretability techniques employed in their models. This filtering process left us with 31 papers eligible for final review and inclusion in our study. To illustrate our selection process, we have depicted a PRISMA diagram in Fig. S1, outlining the stages of paper selection and exclusion. It is important to note that each step of the methodology described was conducted independently by all authors involved in this study. Gene ontology and pathway analyses We employed over-representation analysis (ORA) via WebGestalt (https://2019.webgestalt.org/)100 to investigate gene ontologies and biological pathways associated with protein-coding genes. This approach aimed to identify fundamental biological processes, molecular functions, and cellular components. For GO terms, we consulted non-redundant databases encompassing the three major domains, biological processes, cellular components, and molecular functions. We additionally included pathway annotations from KEGG, Reactome, Panther, and WikiPathway to capture a broad spectrum of potential molecular mechanisms. All GO terms and pathways were prioritized based on their false discovery rate (FDR) values (p < 0.01), thereby minimizing the likelihood of false positives and highlighting the most statistically significant findings. Benchmarking XAI approaches on MOGONet To replicate MOGONet on the ROSMAP dataset, we strictly followed the methodology implemented by its authors. The ROSMAP dataset comprises 351 samples, with 169 (48%) classified as normal cases and 182 (52%) as AD cases. The dataset is divided into training and testing sets, with 70% (245 samples) used for training and 30% (106 samples) used for testing. The division is stratified to ensure that Alzheimer’s cases consistently represent 52% of both subsets. Each sample includes three modalities, DNA methylation, mRNA expression, and microRNA expression, each represented by 200 features, which serve as input to the model. Each modality is processed by its respective graph-based model. The initial predictions from these modality-specific graph-based models are then passed to a view correlation discovery network (VCDN), which integrates the information to make a final prediction. MOGONet is trained in two phases: in the first phase, only the graph-based models are trained for 500 epochs using the Adam optimizer with a learning rate of 10−3, while the VCDN is not trained. In the second phase, both the graph-based models and the VCDN are trained for 2500 epochs, with the learning rate for the graph-based models halved (5 × 10−4) and the VCDN trained with a learning rate of 10−3, using the Adam optimizer. All coding was done on a CPU-enabled Google Colab notebook using the PyTorch framework. Evaluation of explainability without ground truth In scenarios where ground truth annotations are unavailable, fidelity, inverse fidelity, and sparsity can be used for evaluation of the explainability. These parameters measure how well the selected components influence the model’s prediction and how concise the explanations are. Fidelity evaluates how much a selected component (e.g., node, edge, modality, or feature) contributes to the model’s prediction by measuring the performance drop after its removal. Inverse fidelity, on the other hand, measures the performance when only the selected component is retained. Sparsity controls the size of the modified input by penalizing explanations that include too many components, helping to ensure fair comparisons between different scenarios. Ideally, a good explanation yields high fidelity and sparsity, and low inverse fidelity57. In the equations above, fG represents the model’s performance when all components are present, fG\g denotes the performance when specific components g are removed from the full model G, while fg refers to the performance when only those components g are retained. |g| and |G| indicate the number of elements in the selected subset and the full model, respectively. Supplementary information Supplementary information Acknowledgements Analysis was made possible with computational resources provided by the UNSW Biomedical Engineering School and BioMedical Machine Learning Laboratory. Author contributions H.A.R. designed and conceptualized the study. A.S. and H.A.R. wrote the paper. H.A.R., F.H., A.A., M.Y., N.H.L., and A.S. edited the manuscript. A.S. carried out all the analyses, including statistical analyses and benchmarking. A.S. generated all figures and all tables. All authors have read and approved the final version of the paper. Peer review Peer review information Nature Communications thanks Bastian Pfeifer and the other anonymous reviewer(s) for their contribution to the peer review of this work. Funding This study was supported by the UNSW Scientia Program Fellowship and the Australian Research Council Discovery Early Career Researcher Award (DECRA), under grant DE220101210 to H.A.R. Data availability The source code and the dataset can be found at https://github.com/alrzsdgh/Graph-Review/. Competing interests The authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary information The online version contains Supplementary material available at 10.1038/s41467-026-74126-5. References Associated Data Supplementary Materials Supplementary information Data Availability Statement The source code and the dataset can be found at https://github.com/alrzsdgh/Graph-Review/.