Do Segmentation Models Understand Vascular Structure? A Blob-Based XAI Framework

Guillaume Garret, Antoine Vacavant, Carole Frindel · 2025

Read Do Segmentation Models Understand Vascular Structure? A Blob-Based XAI Framework by Guillaume Garret, Antoine Vacavant, Carole Frindel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-11469.txt.

Introduction

AI models can outline blood vessels in medical scans with high scores, yet doctors cannot see why the model draws each line. Guillaume Garret, Antoine Vacavant, and Carole Frindel build a tool to check that reasoning in a 2025 paper. They ask whether models use the global shape of the vessel tree or only tiny local cues.

Their method picks meaningful points from the true vessel graph, then maps which scan voxels drove the choices of the model near those points. A custom blob finder studies those maps at large and small scales. On two data sets, the credit clustered in small blobs near the chosen points. The maps barely tracked vessel traits like thickness or branching, which suggests the models reason locally, not globally.

Catalog record

CREATORGuillaume Garret, Antoine Vacavant, Carole Frindel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-11469

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do Segmentation Models Understand Vascular Structure? A Blob-Based XAI Framework Guillaume Garret, Antoine Vacavant, Carole Frindel Abstract 1 Abstract Deep learning models have achieved impressive performance in medical image segmentation, yet their black-box nature limits clinical adoption. In vascular applications, trustworthy segmentation should rely on both local image cues and global anatomical structures, such as vessel connectivity or branching. However, the extent to which models leverage such global context remains unclear. We present a novel explainability pipeline for 3D vessel segmentation, combining gradient-based attribution with graph-guided point selection and a blob-based analysis of Saliency maps. Using vascular graphs extracted from ground truth, we define anatomically meaningful points of interest (POIs) and assess the contribution of input voxels via Saliency maps. These are analyzed at both global and local scales using a custom blob detector. Applied to IRCAD and Bullitt datasets, our analysis shows that model decisions are dominated by highly localized attribution blobs centered near POIs. Attribution features show little correlation with vessel-level properties such as thickness, tubularity, or connectivity — suggesting limited use of global anatomical reasoning. Our results underline the importance of structured explainability tools and highlight the current limitations of segmentation models in capturing global vascular context. Keywords — explainable artificial intelligence attribution vessel segmentation 2 Introduction Reconstruction of vascular anatomical structures and detection of adjacent lesions are essential steps in clinical protocols, but their automation remains a constant challenge [ 8 , 15 ] . Nowadays, the unmissable methods for meeting this need are based on deep learning. To slightly improve the quality of vascular segmentation, the community has to design increasingly complex models, using ensembling [ 10 , 26 ] , attention modules [ 24 , 18 , 29 ] or joint-models [ 31 , 21 ] . However, deep learning models might be seen as black-boxes. While the number of parameters and non-linearities increase, the models become progressively more difficult to explain and interpret. This lack of transparency poses a significant barrier for their adoption in high-stakes fields like healthcare. In a recent publication, Chouvarda et al. [ 4 ] reported AI explainability and interpretability as the main concerns for clinical researchers. Ensuring that the basis of models’ decisions aligns with domain expert knowledge is then crucial. In particular, a reliable vessel segmentation should rely on both local features, like shape and intensity, as well as global features, such as vessel inter-connections, which are meaningful in a broader context. To address the opacity of deep models, the AI community has turned its attention to eXplainable Artificial Intelligence (XAI). In this relatively new area of research, attribution-based explanations are among the most popular approaches. Ranging from simple methods using gradients [ 2 ] to more robust methods inspired by cooperative game theory [ 1 , 14 ] , attribution aims to identify the most influential input features, e.g. voxels, in the model’s decision-making process. In this context, Rajpurkar et al. [ 19 ] use CAM [ 32 ] to ensure the classifier makes its decision focusing on the right pathological areas. Using Integrated Gradients [ 25 ] , the authors of [ 27 ] leverage attribution to compare two training strategies for multiple sclerosis detection. Finally, in [ 28 ] , a pathology classifier is constrained with a new loss function that expects attribution to be negative on healthy areas. Although attribution methods are commonly used to explain classification models, their application to segmentation tasks is less frequent. Indeed, as segmentation inherently involves localization, attribution maps are often considered redundant in this context. Additionally, many attribution methods are logit-specific, rather than class-specific, making the interpretation of attribution more challenging. A few attempts have been made to explain medical segmentation models. For instance Saleem et al. [ 20 ] identify the impact of multiple MRI sequences on model decisions using per-channel explanations derived from a CAM-based approach. In their study, [ 9 ] introduce Seg-XRes-CAM, a method that generates layer-level and class-discriminative attribution for liver and stomach segmentation. In our work, we argue that while segmentation is inherently a spatial computer task, attribution can still provide valuable insights into how and why a model is making its decisions, potentially leading to improved model performance and interpretability. Considering segmentation task as pixel-wise classification, our main contribution consists in the design of a novel XAI pipeline dedicated to the interpretation of vessel segmentation models. On two vascular datasets, we extract ground-truth graphs and define anatomically meaningful points of interest (POIs), such as bifurcations, endpoints, and mid-branches. We compute gradient-based attribution maps at these POIs to highlight the input regions most influential to the model’s predictions. To analyze the structure of these influential regions, we introduce a blob-based approach that detects compact attribution zones. By comparing blob characteristics to vessel features like connectivity, thickness, and tubularity, we aim to quantify the impact of these high-level features on the decision-making process. 3 Methodology This section details the components of our explainability framework, from the extraction of vascular points of interest to the computation and analysis of Saliency maps. The diagram presented in Figure 1 depicts the key stages of the pipeline. Figure 1 : The proposed XAI pipeline. The red curves represent the vascular graph, with the green point indicating the selected POI. The resulting Saliency map (orange) is processed to compute the blob mask (cyan) and analyzed using our blob-based approach. Finally, the Saliency characteristics are combined with the POI’s vessel properties to assess their contribution to the model’s decision. 3.1 Vascular Points of Interest In the following, we will use ”voxel” to refer to an input voxel and ”logit” to refer to an output voxel of the model. Gradient-based attribution methods are logit-specific, meaning they are not directly aware of the classes. In the context of classification, each class is represented by its own logit, where the value is 1 for the predicted class and 0 elsewhere. Hence, the predicted class is a natural abstraction and corresponds to the label associated to the logit with the highest value. In contrast, label-encoded segmentation does not have a single logit associated with a class, as the output and input have the same dimension. In this latter context, every output logits can belong to any class depending on its value, meaning that we can not derive the ”vessel class” all at once. Instead, we must consider segmentation as a voxel-wise classification, leading to attribution relative to a single voxel. However, computing attribution for every independant logit seems not relevant and is computationally heavy. Therefore, it is essential to identify points of interest — specific logits — that we aim to explain. Vascular networks are tree-like structures that can be depicted by a hierarchical graph. As vessel recognition should partly be based on global context, e.g. connections to other vessels, we produce explanations according to vascular structure. Given the ground-truth of the vessels, we extract the vasculature using the vessel graph extraction pipeline from the Voreen framework [ 5 ] . This pipeline implements a scalable algorithm to extract the annotated graph of a binary volume using a novel iterative refinement approach. Figure 2 depicts two graphs generated by Voreen. Figure 2 : Images produced using Voreen . Vascular structure graphs from

Read the complete text