Source-Free Domain Adaptation by Optimizing Batch-Wise Cosine Similarity
Harsharaj Pathak [email protected], Vineeth N Balasubramanian · 2026
Read Source-Free Domain Adaptation by Optimizing Batch-Wise Cosine Similarity by Harsharaj Pathak [email protected], Vineeth N Balasubramanian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17408.txt.
Introduction
Harsharaj Pathak and Vineeth N. Balasubramanian work on source-free domain adaptation. The task moves a trained model to a new domain with no labels. Source data cannot be used at that stage. Most strong methods lean on neighborhood consistency. They trust that near points share labels. Bad neighborhoods can mislead that trust and spread error.
The authors use batch-wise cosine similarity as their guide. Similarity across a batch gives a broader, steadier signal. That should cut the harm from one bad neighbor. Source-free methods matter where data is private or lost. Medical and retail models face that wall often. A steadier method widens where adaptation can work.
Catalog record
| CREATOR | Harsharaj Pathak [email protected], Vineeth N Balasubramanian |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17408
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Source-Free Domain Adaptation by Optimizing Batch-Wise Cosine Similarity Harsharaj Pathak [email protected], Vineeth N Balasubramanian Abstract Source-Free Domain Adaptation (SFDA) is an emerging area of research that aims to adapt a model trained on a labeled source domain to an unlabeled target domain without accessing the source data. Most of the successful methods in this area rely on the concept of neighbor- hood consistency but are prone to errors due to misleading neighborhood information. In this paper, we explore this approach from the point of view of learning more informative clusters and mitigating the effect of noisy neighbors using a concept called neighborhood signature, and demonstrate that adaptation can be achieved using just a single loss term tailored to optimize the similarity and dissimilarity of predictions of samples in the target domain. In particular, our proposed method outperforms existing methods in the challenging VisDA dataset while also yielding competitive results on other benchmark datasets. 1 Introduction Deep learning models trained on labeled data are known to perform very well on a wide range of tasks. However, one key assumption for most models is that the test data comes from the same distribution as the training data ( Valiant 1984 ; Vapnik 1995 ) . These distributions are also referred to as "domains," as data collected from a particular domain tends to follow some specific distribution. When the test domain is different from the training domain, the performance of the models deteriorate, often drastically, due to the distribution shift ( Gretton et al. 2009 ; Quionero-Candela et al. 2009 ) . As a result, there has been a growing body of research in the area of Unsupervised Domain Adaptation (UDA) which seeks to adapt a model trained on a source domain to a target domain using unlabeled data from the target domain, which are generally easier to collect than labeled data ( Ganin & Lempitsky 2015 ; Gong et al. 2012 ; Bousmalis et al. 2017 ; Ganin et al. 2016 ) . However, most UDA methods also require the labeled data from the source domain during the adaptation ( Daumé III 2007 ; Long et al. 2018 ; Shu et al. 2018 ) . This can be impractical in many scenarios where we only have access to the source model and not the source data due to privacy or copyright reasons. To tackle this, a more restrictive setting has been explored, namely, Source-Free Domain Adaptation (SFDA) where we only have access to the trained source model and the unlabeled target data. In this paper, we specifically focus on SFDA in the context of image classification. Several methods have thus far been explored for SFDA in image classification ( Kundu et al. 2020a ; Li et al. 2020a ; Liang et al. 2020 ; Sun et al. 2020 ; Chen et al. 2018 ; Yang et al. 2021a ; Yang et al. 2022 ) . A defining characteristic of many of these methods is the reliance on neighborhood consistency in the feature space ( Liang et al. 2020 ; Yang et al. 2021a ; Litrico et al. 2023 ) . The assumption here is that despite the domain shift, the source model forms clusters based on the semantic similarity of the samples from the target domain in the feature space ( Yang et al. 2021a ) and this can be used to encourage similar predictions for samples that are neighbors of one another in the feature space. Existing methods often leverage this through techniques such as estimating pseudo labels by aggregating the predictions of neighbors ( Liang et al. 2020 ; Litrico et al. 2023 ) or directly enforcing neighborhood consistency with multiple neighbors ( Yang et al. 2021a ; Yang et al. 2022 ; Hwang et al. 2024 ) along with additional objectives such as diversity ( Liang et al. 2020 ) and contrastive loss ( Litrico et al. 2023 ; Wang et al. 2024 ) . The key limitation of these methods is that they are prone to learning misleading labels due to noisy neighbors, as there is no guarantee that most neighboring samples of a sample are correctly classified at any point in the training. In this paper, we propose a new method for this task to mitigate the effect of noisy neighbors. Our key idea is to use neighborhood consistency in a more reliable manner. To this end, we propose some key insights that can be used to estimate the pairwise semantic similarity and dissimilarity of samples in a training batch in the target domain, which ultimately leads to the model estimating the classes of these samples. Our core approach is to align the predictions of the samples in a batch with respect to cosine similarity, such that the samples that are estimated to be more semantically similar have greater cosine similarity in their predictions and vice versa instead of directly mapping samples into classes. This encourages the model to consider a more holistic view of the semantic information and enables more flexibility in dealing with difficult samples that have overlapping semantic characteristics with multiple classes. In order to estimate the semantic similarity, we introduce a scheme called neighborhood signature, which in our case is simply the mean prediction of the neighborhood samples, but unlike previous methods, we don’t use the neighbors to directly estimate training pseudo labels. Another idea we propose is to encourage the model to learn more informative clusters, i.e., learn semantic differences of samples even when they are estimated to belong to the same class. The goal is to prevent the model from taking shortcuts and assigning samples to classes based on the most common semantic characteristics of the class and thus avoid confidently fitting samples to the wrong class due to noisy neighbors. For example, a model may learn to classify any yellow vehicle as a bus while ignoring the possibility of it being a car or a truck. The model has a better chance of avoiding this if it recognizes the variations within the bus, car, and truck classes. This can also implicitly prevent mode collapse without a separate diversity objective. The third consideration in our method is to consider the confidence with which samples are predicted and use this as inertia while aligning the predictions. For example, if we believe that two objects are semantically similar, and we are confident that one is a cat, and we are less confident that the other is a dog, then it is more likely that both are cats rather than both are dogs. Finally, we also consider the effect of class imbalance. Since different classes may have different frequencies of occurrence while aligning the predictions, the classes with greater frequency can have a dominating effect on the other classes; hence, we counter this by scaling the effect of samples based on their estimated class and the estimated frequency of the classes, even though these estimated classes are not directly used as training labels. Despite all of the above factors, our method still boils down to a single loss term where all the above factors are encoded in a mask that dictates the strength with which the pairwise cosine similarities of the predictions of samples