SCoDA: Self-supervised Continual Domain Adaptation DOI: XXXXXXX.XXXXXXX Conference: ICVGIP 2025; June 03–05, 2025; Mandi, India ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS:
Chirayu Agrawal, Snehasis Mukherjee · 2025
Read SCoDA: Self-supervised Continual Domain Adaptation DOI: XXXXXXX.XXXXXXX Conference: ICVGIP 2025; June 03–05, 2025; Mandi, India ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: by Chirayu Agrawal, Snehasis Mukherjee (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-09935.txt.
Introduction
Chirayu Agrawal and 1 co-author adapt vision models to new domains without ever seeing the old training data in a 2025 machine learning paper. Source-free adaptation starts from a finished source model and must retune it on unlabeled target examples alone. Standard recipes lean on fully supervised pretraining and match features by angle, a habit that throws away the shape of the model's inner space, where much useful structure lives.
Their SCoDA framework starts instead from a teacher trained purely by self-supervision. A student then learns through feature matching plus a space similarity loss that honours the latent geometry, while the teacher slowly follows the student by an exponential moving average to resist forgetting. Benchmark trials place the method well ahead of earlier approaches.
Catalog record
| CREATOR | Chirayu Agrawal, Snehasis Mukherjee |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-09935
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SCoDA: Self-supervised Continual Domain Adaptation DOI: XXXXXXX.XXXXXXX Conference: ICVGIP 2025; June 03–05, 2025; Mandi, India ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper CCS: Do Not Use This Code Generate the Correct Terms for Your Paper Chirayu Agrawal, Snehasis Mukherjee Abstract . Source-Free Domain Adaptation (SFDA) addresses the challenge of adapting a model to a target domain without access to the data of the source domain. Prevailing methods typically start with a source model pre-trained with full supervision and distill the knowledge by aligning instance-level features. However, these approaches, relying on cosine similarity over L2-normalized feature vectors, inadvertently discard crucial geometric information about the latent manifold of the source model. We introduce Self-supervised Continual Domain Adaptation (SCoDA) to address these limitations. We make two key departures from standard practice: first, we avoid the reliance on supervised pre-training by initializing the proposed framework with a teacher model pre-trained entirely via self-supervision (SSL). Second, we adapt the principle of geometric manifold alignment to the SFDA setting. The student is trained with a composite objective combining instance-level feature matching with a Space Similarity Loss. To combat catastrophic forgetting, the teacher’s parameters are updated via an Exponential Moving Average (EMA) of the student’s parameters. Extensive experiments on benchmark datasets demonstrate that SCoDA significantly outperforms state-of-the-art SFDA methods. The codes will be available after acceptance. 1. Introduction The success of Deep Learning in the computer vision area relies on vast, meticulously labeled datasets. However, a model’s performance often drops when it is applied on a new environment (task) where the data distribution is different from its source environment - a problem known as domain shift. While Unsupervised Domain Adaptation (UDA) is the primary way to address the problem of domain shift, data privacy and intellectual property rules have created a tougher challenge: how to adapt a model to the target domain, without the knowledge learned from the source domain? This problem is called Source-Free Domain Adaptation (SFDA). Figure 1 . The core contribution of SCODA. We replace biased, supervised pre-training with a general, self-supervised foundation. We don’t use brittle, instance-level feature matching because it discards crucial information. Instead, we use robust geometric manifold alignment to preserve the model’s holistic knowledge. This leads to state-of-the-art adaptation while preventing catastrophic forgetting. With SFDA, we adapt a pre-trained model using only unlabeled data from the target, which introduces a major risk of catastrophic forgetting, where the model loses its source knowledge. Most SFDA strategies use a “teacher" model to distill knowledge to a “student" ( Singh and Wang 2022 ) . However, these methods have two key limitations. First, knowledge distillation approaches embed biases from the source classes, which hinders adaptation (thus compromising plasticity). Second, these approaches discard crucial geometric information about the teacher’s learned knowledge space. In the context of Unsupervised Knowledge Distillation (UKD), the typical reliance on aligning L2-normalized feature vectors is a critical flaw in the existing SFDA models, as it discards the holistic geometric structure of the teacher’s learned manifold. The knowledge being transferred to the student is about what the teacher represents for each sample, but not how the teacher organizes its entire latent space. Moreover, a majority of SFDA frameworks commence with a source model pre-trained with full supervision on a labeled source dataset ( Yang et al. 2021a ) . This approach often embeds biases from the source classes into the model. That bias hinders adaptation when we move to target domains with different visual styles or class distributions. We posit that a model pre-trained via self-supervised learning (SSL) approach, which learns more general, label-agnostic visual representations, provides a more robust and adaptable foundation for SFDA. To fix these problems, we propose Self-supervised Continual Domain Adaptation (SCODA). Our work makes a key contribution by adapting the principle of geometric manifold alignment from the Unsupervised Knowledge Distillation (UKD) literature to the SFDA setting. We integrate the Space Similarity Loss proposed by Singh et al. ( Singh and Wang 2022 ) , into a stable teacher-student framework inspired by recent work in continual adaptation such as CoSDA ( Feng et al. 2023 ) . This dual-speed architecture provides a consistent learning target and effectively mitigates catastrophic forgetting. Figure 1 summarizes the proposed idea of SFDA in contrast with the traditional SFDA approaches. The main contributions of this paper are summarized as follows: • We update the standard SFDA paradigm by replacing conventional supervised source pre-training with a fully self-supervised one, demonstrating that label-agnostic representations provide a more effective starting point for domain adaptation. • We are the first to adapt and validate the principle of geometric manifold alignment, specifically the Space Similarity loss from ( Singh and Wang 2022 ) , for the task of Source-Free Domain Adaptation. • We integrate this geometric distillation into a dual-speed teacher-student architecture with a continual learning set up ( Feng et al. 2023 ) to enable robust adaptation while mitigating catastrophic forgetting. • Through extensive experiments, we show that SCoDA achieves state-of-the-art performance on the benchmark datasets, validating the efficacy of our approach. Next, we conduct a survey on the related literature. 2. Related Work We start with a brief survey on the efforts made for unsupervised SFDA, followed by the shortcomings in such methods. Then we discuss a brief literature on Manifold alignment concept and the adaptation with continual learning setup, citing the motivation of our work. 2.1. Unsupervised SFDA Unsupervised SFDA is a domain adaptation method where the goal is to adapt a pre-trained model to an unlabeled target domain without access to the knowledge obtained from the source domain data ( Kim et al. 2021 ) . SHOT ( Liang et al. 2020 ) freezes the classifier module (hypothesis) of the source model and learns the target-specific feature extraction module by exploiting both information maximization and self-supervised pseudo-labeling to implicitly align representations from the target domains to the source hypothesis. An improvement over SHOT was proposed in SHOT++ ( Liang et al. 2022 ) , where a new label transfer strategy was introduced separating the target data into two splits based on the confidence of predictions (labelling information). Then they employ semi supervised learning approach to improve the accuracy of less-confident predictions in the target domain. A2Net ( Xia et al. 2021 ) proposes an adaptive adversarial network to only access the well trained source model instead of source data during adaptation. Li et al. ( Li et al. 2023 ) focus on handling noisy pseudo-labels for SFDA. TPDS ( Tang et al. 2024a ) introduces a flow of proxy distributions that facilitates the bridging of typically large distribution shift from the source domain to a target domain. Fewer efforts are made for unsupervised SFDA. G-SFDA ( Yang et al. 2021a ) enables adaptation to an unlabeled target domain while preserving source performance, using local structure clustering for target alignment and sparse domain attention to retain source knowledge without accessing source data. Co-Learn and Co-learn++ ( Zhang et al. 2025 ) aim to adapt a source model trained on a