Explainable embeddings with Distance Explainer
Christiaan Meijer [email protected] · 2025
Read Explainable embeddings with Distance Explainer by Christiaan Meijer [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15516.txt.
Introduction
Christiaan Meijer and Patrick Bos explain distances between data embeddings in a 2025 paper. Machine learning turns data into vectors in abstract spaces. Why two vectors sit near or far apart is hard to see. Explainable AI has few tools for that gap.
Their method, Distance Explainer, adapts a masking trick from image work. It hides parts of inputs and watches how the distance shifts. Masks ranked by distance impact get filtered for clean signals. Tests use image pairs and image-caption pairs from known sets. Scores for faithfulness and robustness stayed high in the trials. Tuning the mask count changed explanation quality in clear ways. The work aids trust in embedding-based systems.
Catalog record
| CREATOR | Christiaan Meijer [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15516
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Explainable embeddings with Distance Explainer Christiaan Meijer [email protected] Abstract While eXplainable AI (XAI) has advanced significantly, few methods address interpretability in embedded vector spaces where dimensions represent complex abstractions. We introduce Distance Explainer, a novel method for generating local, post-hoc explanations of embedded spaces in machine learning models. Our approach adapts saliency-based techniques from RISE to explain the distance between two embedded data points by assigning attribution values through selective masking and distance-ranked mask filtering. We evaluate Distance Explainer on cross-modal embeddings (image-image and image-caption pairs) using established XAI metrics including Faithfulness, Sensitivity/Robustness, and Randomization. Experiments with ImageNet and CLIP models demonstrate that our method effectively identifies features contributing to similarity or dissimilarity between embedded data points while maintaining high robustness and consistency. We also explore how parameter tuning, particularly mask quantity and selection strategy, affects explanation quality. This work addresses a critical gap in XAI research and enhances transparency and trustworthiness in deep learning applications utilizing embedded spaces. Meijer and Bos † † heading: 1 2025 1- 4/00 10/00 00-000 † † shortheadings: Explainable embeddings with Distance Explainer / Meijer and Bos † † firstpage: 1 † † editor: JMLR editor keywords embedded spaces, explainable AI, attribution, multi-modal, saliency maps 1 Introduction The soaring popularity in the last decade of machine learning (ML) and deep learning (DL) methods resulted in a high demand for understanding trained models. The field specifically aiming to solve this, eXplainable AI (XAI), has therefore been vibrant. XAI algorithms are continuously developed for images, text, time-series and tabular data ( Vilone and Longo 2021 ) . However, XAI methods for general “embedded spaces” are less common. By the term “embedded space” we refer to a multi-dimensional vector space which original data can be projected or encoded into. “Latent space” is a popular alternative term. Embedded spaces are used in dimensional reduction operations that aim to preserve certain properties of the original data, for instance in applications like FaceNet ( Schroff et al. 2015 ) , Word2vec ( Church 2017 ) , VAE ( Kingma and Welling 2019 ) , Spec2vec ( Huber et al. 2021 ) , and Life2Vec ( Savcisens et al. 2024 ) . Multi-modal models such as CLIP ( Radford et al. 2021 ) project multiple data modalities into shared embedded spaces. The increasingly broad scientific application of (multi-modal) embedded space models to model complex phenomena, e.g. for studying visually grounded language learning ( Chrupala 2022 ) to learn about analogies with human learning, make explainable methods especially promising and urgent for increasing the efficiency and trustworthiness of research far beyond the domain of AI research itself ( Yang et al. 2022 ; Gade et al. 2019 ; Gevaert 2022 , see also e.g.) . The structure of embedded spaces created by methods such as deep neural nets are by nature difficult to understand as its dimensions often represent multi-step abstractions of the original data space ( Shahroudnejad 2021 ; Bau et al. 2020 ) . Nevertheless, the structure of some embedded spaces can be made more interpretable using XAI methods ( Ali et al. 2023 ; Boselli et al. 2024 ) . A variety of XAI methods have been developed to interpret and explain model predictions, and these can be broadly categorized based on their approach. Much of the work done for embedded spaces specifically has been focused on the interpretability 1 1 1 We follow the XAI nomenclature proposed by ( Ali et al. 2023 ) . of the spaces or the model as a whole ( Boselli et al. 2024 ) . Methods for interpretability can be contrasted with methods for explainability of model decisions. To the best of our knowledge, no methods are currently available for explaining individual model decisions for embedded vectors. In particular, the method introduced in this paper provides a local, post-hoc explanation of aspects of distance between two data items’ projections within embedded spaces. In general, quantifying the input’s features’ effect on the model output is called attribution ( Achtibat et al. 2023 ) . Other local, post-hoc methods like RISE ( Petsiuk et al. 2018 ) , LIME ( Ribeiro et al. 2016 ) and GradCAM ( Selvaraju et al. 2020 ) take a single input data instance, and calculate the effect of its features on the output of the model that is being explained. Our method however, processes a single data instance by comparing to some other reference data instance. In this paper we introduce a method to locally explain embedded spaces using attribution. In section 2 we describe the what and why of our method. In the rest of the paper, we evaluate its performance. To do so, we ran experiments on a set of models and data items that we list in section 3 . We assess the results of the experiments on our explainer using proven quantitative measures in section 4 . We also test the explainer’s usefulness from our own human point of view in the intuitive qualitative assessment of section 5 . Open questions, future work and other concluding remarks are provided in sections 6 and section 7 . 2 Algorithm We consider a model that encodes data instances from multiple data modalities into an single vector space. For example, the CLIP model encodes both image data and text data into a common semantic vector space, making it possible to reason about semantic similarity between instances from different modalities. Proximity between instances in this space can be interpreted as semantic similarity. We would like to be able to ask the model why certain instances end up closer to each other than others. This question is what our proposed algorithm is developed to address. 2.1 Algorithm description Our starting point was RISE ( Petsiuk et al. 2018 ) , a method that assigns saliency values to each pixel or region of the input data. It does so by randomly masking pixels in a given image and examining the effect on the model’s activation value of a user-specified class of interest. Reasons for choosing RISE include that an implementation was freely available, that it is easy to reason about and hence to extend. Also, RISE is model-agnostic; it does not need any internal information about the model, making it widely applicable. Finally, by masking randomly, the method feeds into the model different parts that potentially hold the most meaning when they’re combined, as opposed to masking isolated regions and ignoring the meaning of combinations of parts. This way, it is sensitive to a wide range of semantic contents. Task The RISE implementation accompanying the original paper ( Petsiuk et al. 2018 ) was tailored to the task of classification, specifically on images. In Ranguelova et al. 2024 , RISE was extended to classification on tabular data, textual data and time series. In this work, we extended the method to a task that differs in 2 important aspects: 1. In our task, both input and output types are different to what is the case in a classification task. The classification task only has one input data-point, and the model is asked to output a certain class, or more generally a vector of activations over some class-space. In the current task,