No Masks Needed: Explainable AI for Deriving Segmentation from Classification Mosong Ma 1 and Tania Stathaki 1 and Michalis Lazarou 2 Abstract Medical image segmentation is vital for modern healthcare and is a key element of computer-aided diagnosis. While recent advancements in computer vision have explored unsupervised segmentation using pre-trained models, these methods have not been translated well to the medical imaging domain. In this work, we introduce a novel approach that fine-tunes pre-trained models specifically for medical images, achieving accurate segmentation with extensive processing. Our method integrates Explainable AI to generate relevance scores, enhancing the segmentation process. Unlike traditional methods that excel in standard benchmarks but falter in medical applications, our approach achieves improved results on datasets like CBIS-DDSM, NuInsSeg and Kvasir-SEG. 1 Introduction Medical image segmentation has become a focal point in research due to its potential to revolutionize healthcare by enabling faster and more accurate diagnoses [ 1 ] . This task, however, is challenging, particularly because deep learning methods rely heavily on large, well-annotated datasets. In the medical domain, such datasets are difficult to obtain because of privacy concerns and the need for expert-level annotations. These annotations require precise pixel-level labelling, which is time-consuming and costly, often demanding physicians to divert time from patient care. To overcome these challenges, recent approaches have explored using weakly supervised learning [ 2 ] with weak annotations or applying transfer learning [ 3 ] from pre-trained models. Our work focuses on the latter, building on the idea of fine-tuning pre-trained classification models specifically for medical tasks. While prior studies [ 4 , 5 , 6 ] have used pre-trained models for segmentation without fine-tuning, achieving notable results in standard image datasets, these methods struggle or fail in medical image segmentation. Figure 1: General Pipeline for ExplainSeg Our approach addresses these limitations by fine-tuning a linear classifier on the medical dataset and utilizing Explainable Artificial Intelligence (XAI) [ 7 ] to interpret the model’s decision-making process. This interpretation is converted into segmentation masks, providing a more human-readable and clinically useful output. By doing so, we demonstrate that fine-tuning, combined with XAI, can enhance the performance of pre-trained models in the medical imaging domain, leading to more effective and interpretable segmentation results. Our contributions are summarized below: • We propose a novel method for medical image segmentation ( \mathrm{ExplainSeg} ). • We show the efficacy of XAI in medical segmentation by simply fine-tuning a linear probing classifier on top of a pre-trained model • \mathrm{ExplainSeg} achieves state-of-the-art performance and significantly outperforms other segmentation methods using pre-trained medical image segmentation models. 2 Related work 2.1 Medical image Segmentation Medical image segmentation [ 8 ] has been widely studied, with various methods developed to address its challenges. Early approaches relied on manual or semi-automated techniques, which require significant human input, making them time-consuming and prone to variability between observers, limiting their use in large-scale studies. Segmentation methods [ 9 ] can be broadly categorized into traditional techniques, such as edge-based, region-based, and thresholding [ 10 ] , along with machine learning methods, which include supervised, semi-supervised, and unsupervised approaches. 2.2 Transfer Learning Segmentation Transfer learning [ 3 ] involves leveraging knowledge from a pre-trained model, typically trained on a large dataset, and applying it to a different but related task. This approach has become popular in image segmentation tasks, particularly when labelled data is scarce or models must generalize across different domains. Several methods have successfully applied transfer learning to segmentation tasks. TokenCut [ 4 ] is a notable example, where a Vision Transformer (ViT) [ 11 ] is pre-trained through the self-supervised learning method of DINO [ 12 ] and used to segment objects by treating image patches as tokens and optimizing a normalized cut criterion. Cut-and-LEaRn (CutLER) [ 5 ] and FOUND [ 6 ] also employ pre-trained networks to analyze background regions and generate coarse object proposals. 2.3 Explainable Artificial Intelligence (XAI) Machine learning has significantly advanced medical image segmentation, with deep learning models like U-Net becoming the standard for their ability to learn complex features. However, these models are often viewed as "black boxes" due to their opaque decision-making processes. Explainable AI (XAI) is increasingly vital in addressing this issue. Techniques like integrated gradients [ 13 ] and Grad-CAM [ 14 ] provide visual explanations, helping users understand which parts of an image influenced the model’s predictions. While these tools enhance the balance between performance and interpretability, challenges remain in ensuring that these explanations are accurate and relevant. 2.4 XAI to Segmentation Our approach leverages deep learning and Explainable AI (XAI) techniques to enhance interpretability and reduce reliance on pixel-level annotations in medical image segmentation. Using transfer learning, we adapt a pre-trained model to our dataset and employ XAI to generate segmentation masks, ensuring both accuracy and transparency. Previous works, such as [ 15 ] and [ 16 ] , leverage XAI to create segmentation masks without pixel-level labels in generic domains like infrastructure imagery. In contrast, our work focuses on the medical imaging domain, addressing its higher precision and robustness requirements. By tailoring XAI methods to weakly annotated medical data, we demonstrate their potential to deliver accurate and meaningful segmentation results, highlighting the novelty of our approach. 3 Method 3.1 Problem formulation At the fine-tuning stage, we assume access to a labelled dataset, D_{\text{train}}=\{\mathbf{x}_{i},y_{i}\}_{i=1}^{N} , of N image-label pairs, where every image \mathbf{x}_{i} has a label y_{i}\in Y , that belongs to the label set Y . We fine-tune a classification network F on D_{\text{train}} . This fine-tuned network F is then trained with an extra classification head G to generate and interpret classification results with the help of XAI. The generated relevance map is refined into segmentation masks with extensive post-processing including morphology and other segmentation methods. 3.2 Optional Fine Tuning & Training When working with a new dataset, fine-tuning [ 17 ] is necessary if a pre-trained classification network struggles to classify accurately. To minimize computational demands, gradients are typically frozen for most of the hidden layers. A task-specific classification layer can also be added on top of the deep learning model, and its parameters are trainable allowing the model to adapt to the specific medical image dataset. 3.3 Integrated Gradients (IG) Integrated gradients [ 13 ] trace how the network determines an image’s class by assigning relevance scores to each pixel that indicate its contribution to the classification result, and is chosen due to its model agnostic feature making it compatible to most models. The integrated gradient along the i -th dimension for an input \mathbf{x} and baseline \mathbf{x}^{\prime} is defined as follows: \text{IG}_{i}(\mathbf{x})=(\mathbf{x}_{i}-\mathbf{x}^{\prime}_{i})\times\int_{\alpha=0}^{1}\frac{\partial F(\mathbf{x}^{\prime}+\alpha\times(\mathbf{x}-\mathbf{x}^{\prime}))}{\partial\mathbf{x}_{i}}\,d\alpha (1) where \alpha is a scaling factor that interpolate between the baseline \mathbf{x}^{\prime} and the input \mathbf{x} , \frac{\partial F(\mathbf{x})}{\partial x_{i}} is the gradient of F(\mathbf{x}) along the i -th dimension. The Noise Tunnel function in Captum [ 18 ] is an extension of existing attribution methods used for explainable AI (XAI) by adding stochastic noise to input samples to enhance the robustness and reliability of the explanations. It applies Gaussian noise to the input data in multiple iterations, averaging the resulting attribution maps. This process helps reduce the model sensitivity to small perturbations and highlights consistent patterns in the attributions. As a result, Noise Tunnel produces more stable and interpretable explanations. We did not employ other XAI techniques such as Grad-CAM, as preliminary experiments have consistently shown that the coarse heatmaps produced by Grad-CAM offer significantly inferior spatial localization compared to IG, particularly for fine-grained structures in medical images. In our experiments, Grad-CAM failed to highlight diagnostically relevant regions with sufficient resolution or fidelity. Given these limitations and the superior performance of IG in our setting, further exploration of Grad-CAM and related methods was deemed unnecessary. 3.4 Feature Map Extraction Alongside integrated gradients for relevance scores, we also explored extracting and visualizing feature maps from the ViT network [ 19 ] . Feature map extraction is a key process in deep learning. It captures the essential patterns from an input image. As the image passes through the layers of a network, each layer applies filters that detect specific features and identify patterns relevant to tasks like object recognition or segmentation. The result of this filtering process is the feature map, which highlights the areas of the image where the features were detected. Feature maps play a crucial role in helping the network understand the content of an image and perform tasks like classification or localization. 3.5 Image Segmentation The relevance maps generated by XAI methods are insufficient for accurate segmentation mask generation. Therefore, additional post-processing is necessary to refine them into precise segmentation masks. Otsu Thresholding The simplest approach involves applying thresholding to the IG results. Otsu thresholding [ 20 ] is a method used to automatically find the optimal threshold for separating an image into two classes: foreground and background. It works by analyzing the histogram of pixel intensities and selecting the threshold that minimizes the intra-class variance, which measures the spread of pixel values within each class. This approach ensures that the selected threshold effectively distinguishes between the two groups of pixels. In our application, we use the relevance map generated from the Integrated Gradients method as input for Otsu thresholding. The relevance map highlights the importance of each pixel in the input image for the prediction of the model. By applying Otsu thresholding to this map, we can filter out less relevant pixels and focus on the most significant areas that contributed to the model’s decision. This helps reduce noise and emphasize the most pertinent regions in the relevance map. Morphological Processing We applied a series of image processing and morphological operations to refine the pixel relevance maps generated by the Integrated Gradients method. First, the relevance maps were converted to grayscale and enhanced through histogram equalization [ 21 ] . Thresholding was then used to create binary masks that highlight significant regions. We labelled connected components, removed border artefacts, and used dilation to connect fragmented areas. Erosion was applied to clean the boundaries, and small holes were filled to produce smooth, continuous masks. Normalized Cut The Normalized Cut method [ 22 ] models an image as a weighted graph G=(V,E) , where each pixel corresponds to a node v\in V and edges e\in E connect pairs of nodes. The weight w_{ij} of an edge between nodes i and j represents the similarity between the corresponding pixels. The goal is to partition the graph into disjoint sets A and B by minimizing the Ncut value and therefore separating foreground and background, which can be defined as: \text{NCut}(A,B)=\frac{\text{Cut}(A,B)}{\text{Assoc}(A,V)}+\frac{\text{Cut}(A,B)}{\text{Assoc}(B,V)} (2) where: \text{Cut}(A,B)=\sum_{i\in A,j\in B}w_{ij} (3) \text{Assoc}(A,V)=\sum_{i\in A}\sum_{j\in V}w_{ij},\text{Assoc}(B,V)=\sum_{i\in B}\sum_{j\in V}w_{ij} (4) Fusion of Feature and Relevance Map Extracting the feature map from the Vision Transformer network enhances the XAI segmentation by revealing how the network perceives medical images. By integrating pixel-level importance with the feature space representation, we can reduce the influence of outlier pixels in the integrated gradient map. We combine the relevance map and feature map by multiplying them, effectively using the feature map as a weighted map to denoise the relevance map. 3.6 DenseCRF Edge Refinement Dense Conditional Random Field (DenseCRF) [ 23 ] is a powerful post-processing technique used to refine coarse segmentation masks, particularly by leveraging the information from the original image. They work by incorporating both the initial segmentation predictions and the pixel-level details of the original image to produce more accurate and coherent results. DenseCRF models combine unary potentials, which come from the initial segmentation output and indicate the likelihood of each pixel belonging to a specific class, with pairwise potentials that consider the relationships between pixels based on their spatial location and appearance (colour, texture). The key advantage of DenseCRF is its ability to enforce pixels with similar appearances and close proximity in the original image are more likely to share the same label in the refined segmentation. The energy function in DenseCRF is defined as: E(\mathbf{x})=\sum_{i}\phi_{u}(\mathbf{x}_{i})+\sum_{i