Causes and Consequences of Representational Similarity in Machine Learning Models
Zeyu Michael Li *, Hung Anh Vu *, Damilola Awofisayo, Emily Wenger · 2025
Read Causes and Consequences of Representational Similarity in Machine Learning Models by Zeyu Michael Li *, Hung Anh Vu *, Damilola Awofisayo, Emily Wenger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13899.txt.
Introduction
Zeyu Michael Li, Hung Anh Vu, Damilola Awofisayo, and Emily Wenger study why machine learning models come to represent the world in like ways in a 2025 paper. Many past works note this match, even across image and text models, but few ask what causes it.
The team tests two causes: overlap in the data a model trains on, and overlap in the task it learns. Tests span small classifiers up to large language models. Both kinds of overlap raised how alike the models' inner views were, and the two together gave the strongest effect. A last part shows what that match means in use.
Catalog record
| CREATOR | Zeyu Michael Li *, Hung Anh Vu *, Damilola Awofisayo, Emily Wenger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13899
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Causes and Consequences of Representational Similarity in Machine Learning Models Zeyu Michael Li *, Hung Anh Vu *, Damilola Awofisayo, Emily Wenger Abstract Numerous works have noted similarities in how machine learning models represent the world, even across modalities. Although much effort has been devoted to uncovering properties and metrics on which these models align, surprisingly little work has explored causes of this similarity. To advance this line of inquiry, this work explores how two factors—dataset overlap and task overlap—influence downstream model similarity. We evaluate the effects of both factors through experiments across model sizes and modalities, from small classifiers to large language models. We find that both task and dataset overlap cause higher representational similarity and that combining them provides the strongest effect. Finally, we consider downstream consequences of representational similarity, demonstrating how greater similarity increases vulnerability to transferable adversarial and jailbreak attacks. 1 1 footnotetext: These authors contributed equally and are ordered alphabetically by last name. 1 Introduction A large body of work has considered ways in which AI model representations—models’ mathematical depictions of real-world inputs—align (e.g. Sucholutsky et al. (2024) ; Kornblith et al. (2019) ; Klabunde et al. (2023) ; Raghu et al. (2017) ; Morcos et al. (2018) ; Laakso & Cottrell (2000) ; Nguyen et al. (2021) ; Dravid et al. (2023) ; Zou et al. (2023) , among many others). Although much of this work has focused on neural networks, recent work found evidence of “feature universality” (a.k.a. representational alignment) across generative models like large language models (LLMs) ( Huh et al., 2024 ) . Although representational alignment across AI models is well-documented and novel methods for measuring it are developed frequently, only limited work works has investigated its root causes . Understanding the causes and consequences of this phenomenon may enable more principled interactions with the AI models now pervasive in critical arenas like healthcare and national security ( Wang et al., 2022 ; Caballero & Jenkins, 2025 ; Ma et al., 2024 ; Steinberg et al., 2021 ) . Huh et al. (2024) postulated that task alignment among large-scale AI models will increase their similarity, since as models grow larger they will perform more tasks, and this set of tasks will eventually converge. While compelling, this argument lacks empirical evidence. Another, yet-unexplored factor could also drive model alignment: dataset overlap . Many of today’s generative AI models are trained on huge swaths of the internet, since AI “scaling laws” suggest that training on more data produces a better model ( Kaplan et al., 2020 ) . While cheap to acquire, internet-sourced model training datasets may overlap significantly. For example, many models are trained on subsets of Common Crawl, Reddit, and Wikipedia ( Naveed et al., 2025 ; Bommasani et al., 2022 ) . This overlap could drive downstream model similarity. Since limited work explores causes of representational similarity, these two competing hypotheses for model alignment remain unexplored. The task alignment hypothesis of Huh et al. (2024) is only ever argued-for theoretically, so no experiments exist. Similarly, the dataset overlap hypothesis not been explored, since the AI research community focuses more on issues of privacy and copyright in large-scale internet data use, not downstream effects on model behaviors. Our contribution. We propose empirical methods to explore of causes of representational similarity. Building on hypotheses of Huh et al. (2024) and known realities of model training, we explore the relative effects of dataset overlap and task overlap on downstream representation similarity. To do this, we propose two novel methods, dataset splitting and task splitting , that generate datasets with controlled overlap at both the data point and task level, allowing us to quantify how overlap on these properties affects representation similarity. Along the way, we create a new dataset, called ColorShapeDigit800K , to train models with nuanced task differences. We run dataset and task splitting experiments on models ranging from classifiers to small language models and find that: • Both dataset and task overlap are positively correlated with increased representational similarity in models. We observe this trend in nearly all models and datasets we evaluate, from image classifiers to diffusion to text generation models. • Increasing dataset and task overlap together induces strongest representational alignment between models, as measured by mutual information. We compare the mutual information between different overlap types and measured representational similarity in models. • Higher dataset/task overlap cause higher vulnerability to transfer adversarial and jailbreaking attacks , indicating a possible downstream consequence of unexamined model similarity. The rest of the paper proceeds as follows. § 2 situates our study in the broader landscape of representational similarity. § 3 motivates the two causal factors we explore—dataset and task overlap. § 4 provides details on our experimental setup. § 5 and § 6 present key findings. § 7 discusses limitations and broader impacts of our work. Please see code 2 2 2 https://anonymous.4open.science/r/ReprSimCauses-5D26 and dataset ColorShapeDigit800K 3 3 3 https://drive.google.com/file/d/1N1xWKNWm3rD-a7EOmMwQn86pyus8Xn0P/view?usp=share_link . 2 Related Work Observations of model similarity. Numerous prior works have demonstrated that AI models, even those trained for different tasks, can exhibit similar properties, either in their outputs ( functional similarity) or their internal behaviors ( representational similarity, the focus of this work) ( Mehrer et al., 2020 ; Nguyen et al., 2021 ; Räuker et al., 2023 ; Kornblith et al., 2019 ; Li et al., 2015 ; Hermann & Lampinen, 2020 ; Sucholutsky et al., 2024 ) . Prior work on representational similarity has considered similarities in neuron activation patterns and feature space representations of inputs ( Li et al., 2015 ; Hermann & Lampinen, 2020 ; Nguyen et al., 2021 ; Dravid et al., 2023 ; Gurnee et al., 2024 ) , and numerous metrics have been proposed to measure similarities between models ( Kornblith et al., 2019 ; Klabunde et al., 2025 ; Huh et al., 2024 ; Morcos et al., 2018 ; Geirhos et al., 2020 ; Hacohen et al., 2020 ; Bansal et al., 2021 ; Moschella et al., 2023 ; Hermann & Lampinen, 2020 ) . Extensions of this phenomenon, including alignment between AI model and human representations of the world, is an ongoing area of research (see Sucholutsky et al. (2024) for a detailed overview). Causes of model similarity. Prior work has explored factors affecting similarity of computer vision models, such as increases in model and dataset size ( Kornblith et al., 2019 ; Roeder et al., 2021 ; Huh et al., 2024 ) , overall training objectives ( Ciernik et al., 2025 ; Lindsay et al., 2022 ) , multi-modal task generalization ( Huh et al., 2024 ) , and model initialization ( Mehrer et al., 2020 ) . The main findings show that representational alignment scales with model and dataset size and that differences in training objectives and tasks cause divergent model representations ( Sucholutsky et al., 2024 ) . However, these factors have only been explored at a high level, and few efforts have been made to untangle direct causes of representational similarity in models. This motivates our work, which empirically studies how two possible causal factors—dataset overlap and task overlap—affect representational similarity. Downstream effects of model similarity. Beyond curiosity about causes of model similarity, our work is