A Tale of Two Experts: Cooperative Learning for Source - Free Unsupervised Domain Adaptation
Jiaping YuSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected],[email protected] Muli Yang, Institute for Infocomm Research (IR)A*STARSingapore [email protected] Jiapeng JiSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected] · 2025
Read A Tale of Two Experts: Cooperative Learning for Source - Free Unsupervised Domain Adaptation by Jiaping YuSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected],[email protected] Muli Yang, Institute for Infocomm Research (IR)A*STARSingapore [email protected] Jiapeng JiSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22229.txt.
Introduction
Jiaping Yu and four co-authors tackle the problem of adapting an image model to a new domain when the original training data cannot be shared. Privacy and cost often block access to source data, yet a model trained elsewhere may falter on new kinds of images. Their 2025 method, Experts Cooperative Learning, puts two experts on equal footing: a frozen model from the source domain, lightly extended, and a large pretrained vision-language model with a tunable text prompt.
The two experts mine the unlabeled target data for points they agree on. It first retrieves likely source-like samples and complex target samples together. It then fine-tunes each expert on its own retrieved set. Finally it forces the two experts' learning outcomes to stay consistent with each other. On four benchmark data sets, the approach matched the best reported performance for this source-free setting.
Catalog record
| CREATOR | Jiaping YuSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected],[email protected] Muli Yang, Institute for Infocomm Research (IR)A*STARSingapore [email protected] Jiapeng JiSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22229
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Tale of Two Experts: Cooperative Learning for Source - Free Unsupervised Domain Adaptation Jiaping YuSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected],[email protected] Muli Yang, Institute for Infocomm Research (IR)A*STARSingapore [email protected] Jiapeng JiSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected] Jiexi YanSchool of Computer ScienceXidian UniversityXi’an 710071, China [email protected] Cheng DengSchool of Electronic EngineeringXidian UniversityXi’an 710071, China [email protected] Abstract Source-Free Unsupervised Domain Adaptation (SFUDA) addresses the realistic challenge of adapting a source-trained model to a target domain without access to source data, driven by privacy and cost concerns. Existing SFUDA methods either exploit only the source model’s predictions or fine-tune large multimodal models, yet both neglect complementary insights and the latent structure of target data. In this paper, we propose the Experts Cooperative Learning (EXCL). EXCL contains the Dual Experts framework and Retrieval - Augmentation - Interaction optimization pipeline. The Dual Experts framework places a frozen source-domain model (augmented with Conv - Adapter) and a pretrained vision–language model (with trainable text prompt) on equal footing to mine consensus knowledge from unlabeled target samples. To effectively train these plug - in modules under purely unsupervised conditions, we introduce Retrieval-Augmented-Interaction(RAIN), a three - stage pipeline that (i) collaboratively retrieves pseudo - source and complex target samples, (ii) separately fine - tunes each expert on its respective sample set, and (iii) enforces learning object consistency via a shared learning result. Extensive experiments on four benchmark datasets demonstrate that our approach matches state-of-the-art performance. Figure 1: Illustration of Dual Expert Frameworks. The source domain data is unavailable throughout the training steps, except for the source model and unlabeled target domain data. Two expert models must learn from each other to ensure the consistency of the learning target. 1 Introduction Unsupervised Domain Adaptation (UDA) aims to address the scenario where the data distributions of the source and target domains are inconsistent and the target domain samples lack labels, by jointly training a model with labeled source domain samples and unlabeled target domain samples to achieve good generalization performance on the target domain. However, due to concerns over privacy, security, and cost, there is a growing call to restrict models’ access to source domain training data. Against this backdrop, a more realistic and challenging task has emerged: Source-Free Unsupervised Domain Adaptation (SFUDA). In SFUDA, the source domain data becomes inaccessible, and the model training relies on the unlabeled samples from the target domain and the source model pre-trained on the source domain data to adapt to the target domain data. The current methods have been divided into two major categories to address the SFUDA task. The first category of methods focuses on exploring how to utilize the source domain model and target domain data to enhance the generalization of the source domain model to the target domain. For instance, these methods kurmi2021domain ; ding2022source ; tian2021vdm ; tang2022semantic primarily estimate the source domain data distribution explicitly or implicitly to construct an intermediate domain for knowledge transfer. These works xia2021adaptive ; huang2021model ; yang2021exploiting enhance the domain invariance of the target domain model through adversarial or contrastive feature representation. By introducing new loss functions or regularization methods, these works kundu2022balancing ; lao2021hypothesis ; wang2022exploring optimize model training on the target domain. The second category of methods tends to fine-tune the target domain model using pre-trained multimodal models. For example, liang2023open ; tang2024source2 ; tang2024proxy leverage the strong generalization ability of large multimodal models to help the source domain model adapt to the target domain dataset. However, these approaches typically rely solely on the source domain model for the first category of methods. Due to domain differences, the classification boundaries of the source domain model cannot adapt well to the feature distribution of the target domain, leading to performance degradation. For the second category of methods, although pre-trained models provide powerful initial representations, fine-tuning on the target domain may not fully capture the unique characteristics of the target domain, thereby limiting the model’s adaptability. At the same time, these methods fail to adequately consider the quality, representativeness, and uncertainty of target domain data, which can lead to biases in the model adaptation process. Moreover, an open question is how to fully exploit the target domain data’s intrinsic structure and latent information without source domain data. In this case, we proposed a novel method called Experts Cooperative Learning, which contains the Dual Experts Framework and Retrieval-Augmented-Interaction (RAIN) optimization pipeline. In the Dual Experts Framework, the source domain model and the Visual Language Model (VLM) leverage their respective plug-in modules (Conv-Adapter and trainable text prompt) to mine consensual information from the target domain data, thereby achieving adaptation to the target domain dataset. However, unsupervised target domain data are challenging to use for targeted optimization of these plug-in modules. To overcome this challenge, we have designed an optimization pipeline for this framework: RAIN. Specifically, we utilize the source domain model and the VLM to retrieve pseudo-source data information from the target domain data collaboratively. Subsequently, the pseudo-source and the remaining complex data are employed for targeted enhancement training of the plug-in modules. Finally, through the interaction of learning outcomes, the enhanced dual experts ensure the consistency of learning objectives, thereby improving both models’ adaptation to the target domain data. We summarize the main contributions of this paper as follows: 1. We introduce a Dual Experts framework employing a frozen source - domain model (augmented with a Conv - Adapter) and a pretrained VLM (with a learnable text prompt) to mine complementary consensus knowledge from unlabeled target data. 2. We design a three - stage RAIN optimization pipeline to (a) retrieve pseudo - source and complex target samples, (b) fine - tune each expert’s plug - in on these sets, and (c) enforce interactive consistency, leading to synchronized objectives and improved target - domain adaptation. 3. We conducted extensive experiments. The results demonstrate that our method performs well with state-of-the-art (SOTA) methods in all four datasets. 2 Related work 2.1 Source-Free Unsupervised Domain Adaptation Existing SFUDA (Source-free Unsupervised Domain Adaptation) approaches can be categorized into two distinct types. The first type focuses on extracting cross-domain factors from the source domain and transferring these factors through successive model adaptations to align feature distributions across the two domains. For instance, tang2019adaptive establishes a mapping relationship between a sample and its corresponding individual classifier (Support Vector Machine, SVM) in the source domain to ensure effective classification in the target domain. Some approaches utilize pre-trained source models to generate auxiliary factors, including prototypes tanwisuth2021prototype , source distribution estimation ding2022source , or hard samples li2021divergence to facilitate feature alignment. The second type incorporates auxiliary information derived from the unlabeled target domain. In addition to commonly used pseudo-labels chen2022self ; liang2020we , techniques leverage geometric information tang2021nearest ; tang2022semantic . Our method addresses this limitation by leveraging the knowledge encoded in pre-trained multimodal models and a source model for cooperative learning. Figure 2: An overview of EXCL. Part A contains the Dual Experts Framework and the RAIN optimization Pipeline. The retrieval stage will update the pseudo-source and complex data throughout training to help the Weiszfeld Style Loss and the Promote Semantic Consistency Loss update. In the interaction stage, both experts will exchange their softmax output of the whole