DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations
Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio · 2025
Read DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations by Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09663.txt.
Introduction
Anis Bourou and four co-authors built in 2025 a way to explain image classifiers by showing what change would flip their verdict. Deep networks sort microscope images well but hide their reasons, which blocks trust in biology. Their method, DiffEx, uses diffusion models to generate visible attributes that separate two classes. It can then point to the cell features that drive a decision.
The team showed the explanations work on both natural and biological images. Applied to microscopy data, the method surfaced real differences in cell appearance between conditions. Such clues can hint at how a treatment acts or how a disease progresses. The authors say clear visual explanations could speed the hunt for new biomarkers and drugs. The approach keeps the classifier's power while opening its reasoning to inspection.
Catalog record
| CREATOR | Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09663
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations Anis Bourou, Saranga Kingkor Mahanta, Thomas Boyer, Valérie Mezger, Auguste Genovesio Abstract In recent years, deep learning models have been extensively applied to biological data across various modalities. Discriminative deep learning models have excelled at classifying images into categories (e.g., healthy versus diseased, treated versus untreated). However, these models are often perceived as black boxes due to their complexity and lack of interpretability, limiting their application in real-world biological contexts. In biological research, explainability is essential: understanding classifier decisions and identifying subtle differences between conditions are critical for elucidating the effects of treatments, disease progression, and biological processes. To address this challenge, we propose DiffEx, a method for generating visually interpretable attributes to explain classifiers and identify microscopic cellular variations between different conditions. We demonstrate the effectiveness of DiffEx in explaining classifiers trained on natural and biological images. Furthermore, we use DiffEx to uncover phenotypic differences within microscopy datasets. By offering insights into cellular variations through classifier explanations, DiffEx has the potential to advance the understanding of diseases and aid drug discovery by identifying novel biomarkers. † † affiliationnotice: Equal contribution 1 Introduction Image classification is a fundamental task in deep learning that has achieved remarkable results ( Li et al. 2020 ; He et al. 2016 ; Huang et al. 2017 ; Dosovitskiy et al. 2020 ; Liu et al. 2022 ) . The success of classifiers is primarily due to their ability to extract patterns and features from images to distinguish between classes. However, these patterns can often be difficult to discern ( Li et al. 2020 ; Zeiler & Fergus 2014 ) , particularly in microscopy images ( Xing et al. 2018 ; Meijering 2020 ) , which poses challenges for the interpretability of these models. Explaining the decision-making processes of discriminative models is an active area of research. Various strategies ( Selvaraju et al. 2017 ; Chattopadhay et al. 2018 ; Lang et al. 2021a ; Jeanneret et al. 2024 ; Jeanneret et al. 2023 ) have been proposed to clarify how deep learning models arrive at their outputs, aiming to make the decision processes more transparent and understandable. In biological imaging, interpreting classifier decisions is essential for understanding extracted features and uncovering biological insights. For instance, when classifying healthy versus diseased tissues or treated versus untreated samples, it is crucial to determine which attributes influence predictions. Identifying these cellular variations—phenotypes—not only deepens our understanding of diseases but also clarifies treatment effects. Thus, pinpointing the attributes that drive classifier outcomes is fundamental. By uncovering them, we can reveal biologically meaningful phenotypes that offer deeper insights into complex phenomena ( Bourou & Genovesio 2023 ; et al. 2022 ; Bourou et al. 2024 ) . In this work, we introduce DiffEx , a method for uncovering the attributes leveraged by a classifier to make its decisions, and demonstrate its effectiveness on both natural and microscopy images. Our method first builds a latent space that incorporates the classifier’s attributes using diffusion models. We then identify interpretable directions in this latent space using a contrastive learning approach. The discovered directions are ranked by selecting those that most significantly change the classifier’s decision. We summarize our contributions in this work as follows: • We introduce DiffEx, a novel method leveraging diffusion models to identify interpretable attributes that explain the decisions of a classifier. • We demonstrate the versatility of DiffEx by applying it to classifiers trained on both natural and biological image datasets. • In biological datasets, we employ DiffEx to uncover subtle cellular variations between different conditions. 2 Related Work 2.1 Classifiers Explainability Class Activation Maps (CAMs) ( Selvaraju et al. 2017 ; Chattopadhay et al. 2018 ) are a well-known technique for explaining classifier decisions, as they highlight the most influential regions in an image that affect the classifier’s output. However, these methods typically require access to the classifier’s architecture and all its layers, as they involve computing gradients of the outputs with respect to the inputs. Additionally, CAMs only indicate important regions in images without explicitly identifying the affected attributes, such as shape, color, or size. This can be limiting, particularly in microscopy images where subtle variations are of interest. Counterfactual visual explanations represent another family of methods aimed at explaining classifier decisions. These methods seek to identify minimal changes that would alter the classifier’s decision. Generative models have been widely used to generate such counterfactual explanations. Generative Adversarial Networks (GANs), for instance, have been employed for this purpose ( Singla et al. 2020 ; Lang et al. 2021a ; Goetschalckx et al. 2019 ) . While some approaches generate counterfactual explanations all at once ( Singla et al. 2020 ; Goetschalckx et al. 2019 ) , the work in ( Lang et al. 2021a ) identifies a set of attributes that influence the classifier’s decision. However, GANs suffer from training instability due to the simultaneous optimization of two networks: the generator and the discriminator. Recently, diffusion models have demonstrated more stable training, superior generation quality, and greater diversity ( Dhariwal & Nichol 2021 ; Nichol & Dhariwal 2021 ) . They have also been adopted for generating visual counterfactual explanations ( Augustin et al. 2022 ; Jeanneret et al. 2024 ; Sobieski & Biecek 2024 ) . 2.2 Diffusion Models Generative models have recently achieved significant success in various tasks ( Goodfellow et al. 2014 ; Song & Ermon 2020 ; Dhariwal & Nichol 2021 ; Kingma & Welling 2014 ) . Diffusion models ( Ho et al. 2020 ; Song et al. 2022 ; Dhariwal & Nichol 2021 ; Nichol & Dhariwal 2021 ) , a class of generative models, have been applied to different domain ( Dhariwal & Nichol 2021 ; Guo et al. 2023 ; Rombach et al. 2022a ) . These models consist of two processes: a known forward process that gradually adds noise to the input data, and a learned backward process that iteratively denoises the noised input. Numerous works have proposed improvements to diffusion models ( Dhariwal & Nichol 2021 ; Rombach et al. 2022a ; Nichol & Dhariwal 2021 ) , enhancing their performance and making them the new state-of-the-art in generative modeling across different tasks. Recently, it has been shown that diffusion models can be used to learn meaningful representations of images that facilitate image editing tasks ( Preechakul et al. 2022 ; Kwon et al. 2023 ) . In ( Preechakul et al. 2022 ) , the authors proposed adding an encoder network during the training of diffusion models to learn a semantic representation of the image space. This approach enables the model to capture high-level features that can be manipulated for various applications. In ( Kwon et al. 2023 ) , the authors modified the reverse process—introducing an asymmetric reverse process—to discover semantic latent directions in the space induced by the bottleneck of the U-Net ( Ronneberger et al. 2015 ) used as a denoiser in the diffusion model, which they refer to as the h-space . By exploring this space, they were able to identify directions corresponding to specific semantic attributes, allowing for targeted image modifications. These advancements demonstrate the potential of diffusion