RADAR: Relative Angular Divergence Across Representations
Xavier Cadet Dartmouth College [email protected], Mateusz Nowak Dartmouth College [email protected] Peter Chin Dartmouth College [email protected] · 2026
Read RADAR: Relative Angular Divergence Across Representations by Xavier Cadet Dartmouth College [email protected], Mateusz Nowak Dartmouth College [email protected] Peter Chin Dartmouth College [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23028.txt.
Introduction
Cadet, Nowak, and Chin propose RADAR, a simple geometric metric for gauging cross-domain transferability in foundation models. Adding data from new sources does not always help and can cause negative transfer, where performance falls. RADAR tracks how representations evolve layer by layer, measuring angular alignments and distance changes along displacement trajectories, then compares within-domain and cross-domain dynamics. It predicted transfer competitively on vision and text benchmarks, most strongly when domain shifts were smooth or cleanly separated.
Catalog record
| CREATOR | Xavier Cadet Dartmouth College [email protected], Mateusz Nowak Dartmouth College [email protected] Peter Chin Dartmouth College [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90087 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.23028
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RADAR: Relative Angular Divergence Across Representations Xavier Cadet Dartmouth College [email protected], Mateusz Nowak Dartmouth College [email protected] Peter Chin Dartmouth College [email protected] Abstract Machine learning methods rely on data. However, gathering suitable data can be challenging due to availability constraints, cost, or the need for domain expertise. Expanding datasets with additional sources is a common response to limited data, yet this practice does not always improve downstream performance and can sometimes lead to a loss of performance, known as negative transfer. We propose RADAR, a simple, geometrically grounded metric for estimating cross-domain transferability in foundation models. RADAR analyzes the layer-wise evolution of representations by measuring angular alignments and relative changes in distance along layer-to-layer displacement trajectories, and by comparing empirical distributions of within-domain and cross-domain dynamics. We hypothesize that domain transferability is related to the divergence between these trajectory distributions. We evaluate the metric across multiple modalities, including cross-lingual sentiment classification with text embedding models and cross-domain image classification with foundation vision models. Across several settings, RADAR provides competitive predictive performance relative to existing transferability metrics on several vision and text benchmarks, with particularly strong results when domain transitions are smooth or cleanly separated. Our ablations further suggest that the effectiveness of transferability estimation depends on the geometry of the model’s internal representation space, with different modalities favoring different topological formulations. 1 Introduction In modern machine learning, expanding training datasets with auxiliary sources is a standard strategy to address data constraints. However, this practice does not guarantee improved downstream performance and can even lead to negative transfer. Identifying which source domains will yield positive transfer remains a difficult task, as empirically evaluating each candidate impact requires costly full-model retraining. To address this challenge, we propose RADAR ( R elative A ngular D ivergence A cross R epresentations), a simple, geometrically grounded metric designed to efficiently estimate cross-domain transferability in foundation models using only frozen features. Unlike baseline metrics that require pre-training on the target dataset or evaluate only the static output of a single layer, RADAR analyzes the dynamic, layer-wise structural evolution of representations using frozen foundation models. We provide a high-level overview of the metric in Figure 1 . The framework measures divergence by extracting intermediate feature representations from the model to map layer-to-layer displacement trajectories. These trajectories are then characterized by computing relative distance changes and angles between samples within the same domain versus samples across different domains. Based on these angles and relative distances, we fit high-dimensional Gaussian Mixture Models (GMMs) to model the continuous probability distributions of the within-domain and cross-domain dynamics. Finally, given these two distributions, we measure the overall structural shift between domains by computing the symmetric Kullback-Leibler (KL) divergence between them. Figure 1 : Overview of the RADAR framework. RADAR identifies optimal source data for transfer learning by extracting layer-wise ( \ell ) geometric trajectories from pre-trained models. By computing the divergence in angle ( \theta ) and relative distance ( d ) densities between within- and cross-domain pairs, it efficiently ranks auxiliary candidates without requiring computationally expensive fine-tuning. We evaluate RADAR across both vision modalities (using CLIP [ 36 ] and DINOv3 [ 38 ] architectures) and text modalities (using Qwen3-Embedding [ 48 ] and EmbeddingGemma [ 43 ] ) on benchmark datasets such as DomainNet [ 35 ] , OfficeHome [ 42 ] , PACS [ 25 ] for the vision modality, AmazonReview [ 34 ] , and EuroEval [ 34 ] for the text modality, comparing it against state-of-the-art transferability metrics. We show that RADAR achieves top-three performance in seven out of ten configurations, securing the absolute best performance in six of them. Furthermore, we thoroughly evaluate the design choices of our algorithm—specifically focusing on domain subsampling, density estimation, and alternative optimal transport algorithms—demonstrating the overall robustness of our method. Our core contributions and findings are summarized as follows: • Dynamic, Layer-Wise Structural Evolution: We propose RADAR—a geometrically grounded metric that evaluates dynamic representation shifts between different domains. By measuring the divergence between these dynamic, continuous distributions, we effectively capture both smooth domain transitions and distinctly separated domains. • Highly Competitive Predictive Performance: RADAR consistently ranks as the best or second-best transferability metric against established baselines (such as LEEP [ 32 ] , LogME [ 45 ] , NCE [ 41 ] , TransRate [ 13 ] , and S-OTDD [ 33 ] ) on text and vision modalities. • Robust and Efficient Estimation: We demonstrate the robustness and high computational efficiency of our method through an extensive ablation study, focusing on hyperparameter choices, density estimation techniques, representation subsampling, and alternative optimal transport algorithms. Ultimately, RADAR offers a competitive, cross-modal transferability metric that reliably predicts transfer gain rankings across diverse vision and text benchmarks — particularly when domain transitions are smooth or cleanly separated — while remaining computationally efficient relative to empirical- and optimal transport-based metrics. 2 Problem Statement Domains and shared label space. Let \mathcal{D}=\{\mathcal{D}_{\mathrm{1}},\ldots,\mathcal{D}_{\mathrm{K}}\} be a collection of K domains, each containing labeled data \mathcal{D}_{\mathrm{i}}=\{(x_{j},y_{j})\}_{j=1}^{N_{i}} drawn from a domain-specific joint distribution P_{i}(X,Y) . We assume all domains share a common label space \mathcal{Y} , with every class represented in every domain. One domain \mathcal{D}_{\mathrm{\mathrm{tgt}}}\in\mathcal{D} is designated the target ; the remaining K-1 domains form the pool of source candidates, \mathcal{D}_{\mathrm{src}}=\mathcal{D}\setminus\{\mathcal{D}_{\mathrm{\mathrm{tgt}}}\} . For instance, assuming we have three domains {real, paintings, sketches} dataset, we would like to know whether we should use paintings, and sketches to improve our performance on real images. Blended training set. A blend is a subset \mathcal{S}\subseteq\mathcal{D}_{\mathrm{src}} ; the corresponding blended dataset is the union of its member domains, \mathcal{D}_{\mathrm{blend}}\left(\mathcal{S}\right)=\bigcup_{\mathcal{D}_{\mathrm{i}}\in\mathcal{S}}\mathcal{D}_{\mathrm{i}}. (1) There are 2^{K-1} possible blends. For instance, if our target domain is real, should we train with {{}, {paintings}, {sketches}, {sketches, paintings}}. Given a blend \mathcal{D}_{\mathrm{blend}}\left(\mathcal{S}\right) , we train a classifier on the union of the target data and the source blend, \mathcal{D}_{\mathrm{\mathrm{tgt}}}\cup\mathcal{D}_{\mathrm{blend}}(\mathcal{S}) , and evaluate it on \mathcal{D}_{\mathrm{\mathrm{tgt}}} . We can now naturally evaluate the target-only baseline as \mathcal{S}=\varnothing , yielding A(\varnothing)=\acc\bigl(f_{\mathcal{D}_{\mathrm{\mathrm{tgt}}}},\mathcal{D}_{\mathrm{\mathrm{tgt}}}\bigr) . We record two quantities as ground truth: \displaystyle A(\mathcal{S}) \displaystyle=\acc\bigl(f_{\mathcal{D}_{\mathrm{\mathrm{tgt}}}\cup\mathcal{D}_{\mathrm{blend}}\left(\mathcal{S}\right)},\mathcal{D}_{\mathrm{\mathrm{tgt}}}\bigr), (2) \displaystyle\Delta(\mathcal{S}) \displaystyle=A(\mathcal{S})-A(\varnothing). (3) A positive \Delta(\mathcal{S}) indicates that blending in the sources \mathcal{S} improves over training on target data alone; a negative value indicates negative transfer. Source selection problem. Computing A(\mathcal{S}) for every subset \mathcal{S} requires 2^{K-1} full training runs, which is intractable for large K . Our goal is to predict \Delta(\mathcal{S}) — and thereby rank blend configurations — from frozen features alone , without any model retraining. Concretely, we seek a blend similarity function \dsim\bigl(\mathcal{D}_{\mathrm{blend}}(\mathcal{S}),\mathcal{D}_{\mathrm{\mathrm{tgt}}}\bigr)\in\mathbb{R} (4) such that its ranking over all 2^{K-1}-1 non-empty blends is consistent with the ranking induced by \Delta(\mathcal{S}) : \dsim\!\bigl(\mathcal{D}_{\mathrm{blend}}(\mathcal{S}),\,\mathcal{D}_{\mathrm{\mathrm{tgt}}}\bigr)<\dsim\!\bigl(\mathcal{D}_{\mathrm{blend}}(\mathcal{S}^{\prime}),\,\mathcal{D}_{\mathrm{\mathrm{tgt}}}\bigr)\implies\Delta(\mathcal{S})>\Delta(\mathcal{S}^{\prime}). (5) We evaluate the quality of a similarity function by its Spearman rank correlation \rho with the ground-truth \Delta values across all blend configurations. 3 Relative Angular Divergence Across Representations as a Metric Feature extraction. Let \Phi denote a pre-trained foundation model, such as a text embedding or vision architecture. For an input sample x_{i} drawn from a specific domain \mathcal{D}_{\mathrm{d}}\in\mathcal{D} , we define its intermediate feature representation extracted at layer l of \Phi as \mathbf{h}^{(l)}(x_{i})\in\mathbb{R}^{H_{l}} , where H_{l} is the dimensionality of the representation at layer l . (a) (b) Figure 2 : Geometric triangle of displacement vectors spanning layers l and