DifCluE: Generating Counterfactual Explanations with Diffusion Autoencoders and modal clustering

Suparshva Jain, Amit Sangroya, Lovekesh Vig · 2025

Read DifCluE: Generating Counterfactual Explanations with Diffusion Autoencoders and modal clustering by Suparshva Jain, Amit Sangroya, Lovekesh Vig (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11509.txt.

Introduction

Suparshva Jain, Amit Sangroya, and Lovekesh Vig present DifCluE, a method that explains a model's call by showing varied cases that flip it. Their 2025 paper works on counterfactual explanations. A counterfactual shows the smallest change that would flip a result. One class can hold several distinct modes, and each mode may need its own flip case. Past tools tend to miss that spread.

A diffusion autoencoder learns the modes hidden in the data. Grouping points in its inner space reveals the path tied to each mode. DifCluE follows those paths to build flip cases for every mode in turn. Tests show the method finds modes more steadily and yields more trustworthy flips than leading rivals. Clearer, more varied flips help users see what a model truly keys on, a gain for trust and audits.

Catalog record

CREATORSuparshva Jain, Amit Sangroya, Lovekesh Vig
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-11509

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DifCluE: Generating Counterfactual Explanations with Diffusion Autoencoders and modal clustering Suparshva Jain, Amit Sangroya, Lovekesh Vig Abstract . Generating multiple counterfactual explanations for different modes within a class presents a significant challenge, as these modes are distinct yet converge under the same classification. Diffusion probabilistic models (DPMs) have demonstrated a strong ability to capture the underlying modes of data distributions. In this paper, we harness the power of a Diffusion Autoencoder to generate multiple distinct counterfactual explanations. By clustering in the latent space, we uncover the directions corresponding to the different modes within a class, enabling the generation of diverse and meaningful counterfactuals. We introduce a novel methodology, DifCluE , which consistently identifies these modes and produces more reliable counterfactual explanations. Our experimental results demonstrate that DifCluE outperforms the current state-of-the-art in generating multiple counterfactual explanations, offering a significant advancement in model interpretability. 1. Introduction Real-world classification problems often deal with situations in which the feature set can have multiple modes but are classified within a single class. For example, a skin lesion can be classified as malignant due to many different value combinations of clinical features like a asymmetry, size, uneven colour, irregular borders etc (Figure 1 ). In order to achieve larger trust in the outcomes of deep models, we often rely on counterfactual explanations. Generating counterfactual explanations in such a case can prove to be tricky, since the class can have multiple modes. This challenge that has received limited attention. Prior work on counterfactual generation utilizes three distinct approaches; (a) generate multiple counterfactuals based on a distinctness loss ( Ghandeharioun et al., 2021 ) , (b) use semantic information gathered from an encoder to generate counterfactuals ( Downs et al., 2020 ) , and (c) use gradient information from the target class for counterfactual explanations of input instances ( Jeanneret et al., 2022 ) . However, during the generation of counterfactuals, when there are multiple modes in a class, the generation method needs to generate counterfactual exemplars for each mode with in the class. This is important to get the coverage of all valid feature perturbations that would classify an input into a target class, and helps us to get a better understanding of the target class. Additionally, being able to generate perturbations at the conceptual feature-level, and not just at the pixel-level or at the image-attribution level, would improve counterfactual exemplars and in-turn improve the level of explainability of the counterfactual examples for the end-users. In our work, we explore the ability of a Diffusion Autoencoder to come up with multiple distinct counterfactual explanations by performing clustering in latent space, when we have a class which contains more than one mode. We show that clustering in the latent space can help us to uncover the various modes within a class helping us to generate multiple distinct counterfactual explanations. Diffusion probabilistic models (DPMs) have been shown to outperform GANs in terms of the quality of the images generated. In this paper, we make following key contributions: (1) We propose our methodology DifCluE ( Dif fuse, Clu ster and E xplain), that generates distinct counterfactual explanations. This approach outperforms the current state of the art models for various quality parameters such as modal coverage, and quality of images generated. (2) Our results demonstrate superior performance in terms of mode capture and counterfactual generation when compared to prior approaches (See Figure 2 ). (3) We demonstrate that DifCluE approach which is based on a diffusion autoencoder to capture the data distribution, and clustering the resulting class embeddings leads to a better discovery of intra-class modes. Our experimental results show that after mixing different classes into a single class and employing clustering we are able to closely recover the original classes, suggesting that some degree of disentanglement is being achieved in the intra class modes. Figure 1. Various features that melanoma skin lesions may exhibit as compared to a non melanoma skin lesions Figure 2. Comparison between the counterfactual explanations provided by DISSECT model and DifCluE model 2. Related Work Counterfactual explanation generation using diffusion models represents a cutting-edge approach in interpretable AI. Diffusion models, known for their ability to generate high-quality samples, are now being adapted to create counterfactuals—hypothetical scenarios that explain model decisions by altering input features. By leveraging the stochastic nature of diffusion processes, these models can produce diverse and plausible counterfactuals, enhancing the transparency and robustness of AI systems in fields such as finance, healthcare, and legal decision-making. Diffusion probabilistic models (DPM) have achieved superlative image generation quality ( Sohl-Dickstein et al., 2015 ; Song and Ermon, 2019 ; Ho et al., 2020 ; Song et al., 2020 ) and Diffusion-based image editing has drawn much attention. There are two main categories of image editing and generation. Firstly, image-guided generation works to edit an image by mixing the latent variables of DPM and the input image ( Choi et al., 2021 ; Lugmayr et al., 2022 ; Meng et al., 2021 ) . However, using images to specify the attributes for editing may cause ambiguity, as pointed out by ( Kwon et al., 2023 ) . Secondly, the classifier guided works ( Dhariwal and Nichol, 2021 ; Avrahami et al., 2021 ; Liu et al., 2021 ) edit images by utilizing the gradient of an extra classifier. In our work, we build upon the second category, aiming to generate multiple counterfactual explanations with the the help of clustering and gradients of the classifier. Bengio et al. introduced disentangled representation learning, where the target is to discover underlying explanatory factors of the observed data ( Bengio et al., 2012 ) . The disentangled representation is defined such that each dimension (or set of dimensions) of the representation correspond to an independent factor. Based on this definition, some VAE-based techniques achieve disentanglement by constraining the probabilistic distributions of representations ( Chen et al., 2018 ; Kim and Mnih, 2018 ; Higgins et al., 2016 ) . In ( Locatello et al., 2018 ) , authors point out the identifiable problem by proving that only these constraints are insufficient for disentanglement and that extra inductive bias is required. A number of work has been done in generation of counterfactual explanations. Nemirovsky et al. uses a Residual Generative Adversarial Network (RGAN) to generate counterfactuals ( Nemirovsky et al., 2020 ) . They enhance a regular GAN output with a residual that appear like perturbations used in counterfactual search, and then use a fixed target classifier to provide the counterfactuals. Support Vector Data description based counterfactual generation is described in ( Carlevaro et al., 2023 ) . They use data envelopes extracted via singular value decomposition to generate counterfactuals for multi-class setting. However, different modes in a class does not seem to be addressed. In ( Sun et al., 2023 ) , authors generate class-specific attribution maps based on counterfactuals to find which region of the image are important for that classification and then use a simple logistic regression classifier to make predictions. Authors claim these methods to be inherently interpretable. In ( Barberan et al., 2022 ) authors describe NeuroView-RNN as a family of new RNN architectures that explains how each hidden state per time step contributes to the decision-making

Read the complete text