Representational Similarity via Interpretable Visual Concepts
Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona · 2025
Read Representational Similarity via Interpretable Visual Concepts by Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15699.txt.
Introduction
Neehar Kondapaneni, Oisin Mac Aodha, and Pietro Perona present a study from twenty twenty-five on how to compare what two neural networks see in an image. Most gauges give one number for how alike two networks are at a layer. That number hides why they match or part. Their method, RSVC, names the visual ideas each network uses.
Shared ideas and lone ideas come out of the check. Some gaps trace to ideas that one model holds and the other lacks. Tests span many network builds and training styles. The authors say idea level checks can guide fixes that a lone score cannot. The work aids the craft of probing vision models.
Catalog record
| CREATOR | Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-15699
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Representational Similarity via Interpretable Visual Concepts Neehar Kondapaneni, Oisin Mac Aodha, Pietro Perona Abstract How do two deep neural networks differ in how they arrive at a decision? Measuring the similarity of deep networks has been a long-standing open question. Most existing methods provide a single number to measure the similarity of two networks at a given layer, but give no insight into what makes them similar or dissimilar. We introduce an interpretable representational similarity method (RSVC) to compare two networks. We use RSVC to discover shared and unique visual concepts between two models. We show that some aspects of model differences can be attributed to unique concepts discovered by one model that are not well represented in the other. Finally, we conduct extensive evaluation across different vision model architectures and training protocols to demonstrate its effectiveness. Project Page: RSVC Code: github.com/nkondapa/RSVC 1 Introduction The accuracy of deep neural networks has steadily increased over the last few years thanks to improvements in model architectures, dataset size, and pretraining strategies. However, much less is understood regarding how the representations of different models have changed to make the models more effective. Thus, there is growing interest in developing methods that allow practitioners to compare different networks. Comparing the activation matrices of two neural networks over the same set of inputs underpins current representational similarity methods, e.g., CCA ( Hotelling, 1936 ) , CKA ( Kornblith et al., 2019 ) , RSA ( Kriegeskorte et al., 2008 ) , and Brain-score ( Schrimpf et al., 2018 ) . While these approaches provide a score denoting the similarity between two different models, they do not identify the specifics of what makes two models’ computations similar or dissimilar, and what aspects of a representation lead to differences in model decisions. In parallel, methods for concept-based eXplainable AI (XAI) have improved our ability to understand what features individual models use to arrive at decisions. Understanding these features is critical for ensuring model fairness and identifying potential sources of bias ( Kaminski & Urban, 2021 ; Kop, 2021 ) . In general, XAI methods sacrifice model fidelity to produce explanations that are simple enough for human interpretation ( Fel et al., 2023a ; Cunningham et al., 2024 ) . Thus, there is a tension between model fidelity and human understanding. We propose that contrasting two models is an effective way to identify and highlight what makes a model unique, so that users can identify critical features that drive differences in model behavior. To investigate this idea we develop a new approach that extends concept-based XAI methods so they can quantitatively measure representational similarity and provide interpretable insights into the differences between models. We name our method Representational Similarity via Interpretable Visual Concepts (RSVC). Our method builds on interpretability approaches in which the activations of a given layer are decomposed into a coefficient matrix and a vector basis. These approaches group images that produce similar activation patterns together. Thus, we define a visual concept to be the equivalence class of images that produce similar activation patterns. After computing visual concepts from each model, our method evaluates whether the visual concepts from one model are also used by the other model. We make three contributions: • RSVC, a new approach for providing human-interpretable insights into model differences ( Fig. 1 ). • A validation strategy to link representational differences to model decisions. • Experiments to show that RSVC can measure similarity at both coarse- and fine-grained levels. 2 Related Work 2.1 Representational Similarity Similarity methods attempt to quantify the similarity/dissimilarity between pairs of different models ( Hotelling, 1936 ; Kornblith et al., 2019 ; Raghu et al., 2017 ; Li et al., 2015 ; Huh et al., 2024 ) . Models can be compared based on their functional similarity (i.e., how their outputs relate) or their representational similarity (i.e., how the activations of intermediate layers relate) ( Klabunde et al., 2023 ) . While functional similarity can tell us about how model outputs vary, two models can achieve the same performance with significantly different representations. These differences matter, e.g., while two different forms of pretraining can achieve similar performance on certain datasets, they may transfer poorly to others ( Xie et al., 2023 ) . Representational similarity metrics have successfully been used to analyze the differences between architectures ( Nguyen et al., 2021 ; Raghu et al., 2021 ) , explore the effects of different kinds of pretraining ( Xie et al., 2023 ; Neyshabur et al., 2020 ; Park et al., 2024 ) , develop novel strategies for efficient ensembling ( Zhang et al., 2020 ) , or perform ensembling that is robust to distribution shifts ( Lee et al., 2023 ) . Some methods have leveraged representational similarity to build tools for text-to-image generation ( Rombach et al., 2020 ) or model-to-model translation ( Dravid et al., 2023 ) . In neuro- and cognitive science, representational similarity is used to measure how well models are able to approximate the neural recordings of the brain and/or the behavior of humans ( Schrimpf et al., 2018 ; Muttenthaler et al., 2023 ; Fel et al., 2022b ; Ahlert et al., 2024 ; Kriegeskorte et al., 2008 ) . In the disentanglement literature, representational similarity has been used to evaluate how well the learned representation matches known ground truth latent factors. A popular approach is to use regularized linear predictors to map learned factors to ground truth latent factors ( Eastwood & Williams, 2018 ; Eastwood et al., 2023 ; Locatello et al., 2019 ; Roth et al., 2023 ; Duan et al., 2020 ) . Recently, Dravid et al. (2023) proposed a method that uses correlated activation patterns across networks to mine for “Rosetta Neurons”. These neurons provide insights about features that re-occur consistently in many models. In contrast to our proposed method, Rosetta Neurons do not quantify the overall similarity between networks and do not identify neurons that explain model differences. Most closely related to our work is that of ( Schrimpf et al., 2018 ) and ( Li et al., 2015 ) which use a linear regression model as a similarity metric between the activations of two networks. We describe these methods in more detail in Sec. 3.2 . The primary limitation of most existing representational similarity methods is that they can quantify how similar the representations of two models are, but can not tell users what makes the models similar or dissimilar. We propose a new approach that aims to address this question. Our approach leverages concept-based explainability as an intermediate step to measure representational similarity. 2.2 Explainable AI (XAI) XAI methods aim to answer the questions (1) what features did a model use to arrive at a decision and (2) where is the relevant information in the input. Local explanation methods focus on pixel-based attribution, in which a heatmap indicates the region of the image that is most relevant to the model’s decision ( Selvaraju et al., 2020 ; Ribeiro et al., 2016 ; Lundberg & Lee, 2017 ) . While local explanations are able to answer the “where” question, it can be challenging to interpret “what” is being highlighted as attributions can be noisy.