DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization
Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee · 2024
Read DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization by Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-09074.txt.
Introduction
Jin-Seop Lee, Noo-ri Kim, and Jee-Hyong Lee propose DomCLP, a contrastive learning method for unsupervised domain generalization. Self supervised models often fail on data from unseen domains. The goal is to learn features that ignore domain while keeping class content, using data from several source domains with no labels.
The authors show why standard instance discrimination suppresses shared cross domain features and amplifies domain specific ones. Their domain wise contrastive step reverses that effect and strengthens shared features. A prototype mixup step adds diversity and counters bias from strong alignment assumptions in past work. The combined method aims for representations that transfer to new domains without any target data for tuning.
Catalog record
| CREATOR | Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-09074
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DomCLP: Domain-wise Contrastive Learning with Prototype Mixup for Unsupervised Domain Generalization Jin-Seop Lee, Noo-ri Kim, Jee-Hyong Lee Abstract Self-supervised learning (SSL) methods based on the instance discrimination tasks with InfoNCE have achieved remarkable success. Despite their success, SSL models often struggle to generate effective representations for unseen-domain data. To address this issue, research on unsupervised domain generalization (UDG), which aims to develop SSL models that can generate domain-irrelevant features, has been conducted. Most UDG approaches utilize contrastive learning with InfoNCE to generate representations, and perform feature alignment based on strong assumptions to generalize domain-irrelevant common features from multi-source domains. However, existing methods that rely on instance discrimination tasks are not effective at extracting domain-irrelevant common features. This leads to the suppression of domain-irrelevant common features and the amplification of domain-relevant features, thereby hindering domain generalization. Furthermore, strong assumptions underlying feature alignment can lead to biased feature learning, reducing the diversity of common features. In this paper, we propose a novel approach, DomCLP , Domain-wise Contrastive Learning with Prototype Mixup. We explore how InfoNCE suppresses domain-irrelevant common features and amplifies domain-relevant features. Based on this analysis, we propose Domain-wise Contrastive Learning ( DCon ) to enhance domain-irrelevant common features. We also propose Prototype Mixup Learning ( PMix ) to generalize domain-irrelevant common features across multiple domains without relying on strong assumptions. The proposed method consistently outperforms state-of-the-art methods on the PACS and DomainNet datasets across various label fractions, showing significant improvements. Our code will be released. Our public code is available at https://github.com/jinsuby/DomCLP . Sungkyunkwan University, Suwon, South Korea {wlstjq0602, pd99j, john}@skku.edu Introduction Self-supervised learning (SSL) methods, particularly based on the instance discrimination task using contrastive learning with InfoNCE, have shown remarkable performance ( Oord, Li, and Vinyals 2018 ; Chen et al. 2020a ; He et al. 2020 ; Grill et al. 2020 ; Caron et al. 2020 ) . Despite these successes, there is a critical limitation in that they assume pretraining, fine-tuning, and testing data all originate from the same distribution. This assumption often does not hold in real-world scenarios, where data distribution shifts frequently occur, leading to previously unseen data. Consequently, while SSL models are capable of generating high-quality representations for in-domain data, they often struggle to generate effective representations for unseen-domain data ( Zhang et al. 2022b ; Harary et al. 2022 ) . To address this challenge, there has been growing interest in the field of unsupervised domain generalization (UDG), which aims to develop SSL models that can learn domain-irrelevant features ( Zhang et al. 2022b ) . Figure 1: (a) In the UDG environment, representations include both domain-irrelevant common features and domain-relevant features. (b) T-SNE visualization for SimCLR. In the UDG scenario as depicted in Figure 1a, there are multi-domain source datasets with only domain labels and no additional class labels. Each sample contains both domain-irrelevant common features (e.g. shape, components, etc.) and domain-relevant features (e.g. style, background, etc.). To achieve good generalization performance on unseen domains, unsupervised domain generalization (UDG) ( Zhang et al. 2022b ) aims to learn representations of domain-irrelevant feature. To learn domain-irrelevant features, most UDG approaches utilize contrastive learning based on InfoNCE to generate feature representations, and perform feature alignment based on strong assumptions to achieve domain generalization ( Zhang et al. 2022b ; Harary et al. 2022 ; Liu et al. 2023 ; Scalbert, Vakalopoulou, and Couzinié-Devy 2023 ) . To align cross-domain features, BrAD ( Harary et al. 2022 ) utilized edge-like image transforms, while DN 2 A ( Liu et al. 2023 ) employed cross-domain nearest neighbors as positive samples in contrastive learning. In BSS ( Scalbert, Vakalopoulou, and Couzinié-Devy 2023 ) , they introduced batch style standardization using Fourier transform for feature alignment. However, most approaches still struggle to effectively extract domain-irrelevant common features. Most UDG approaches rely on instance discrimination tasks, which are not well-suited for domain generalization. In contrastive learning with InfoNCE, representations are learned to distinguish between instances. If the model attempts to differentiate between instances, deep neural networks tend to learn features that are useful to discriminate instances. In UDG environments, they are easy to capture domain-relevant features rather than domain-irrelevant common features, because domain-relevant features are more helpful to distinguish instances across various domains ( Robinson et al. 2021 ; Chen, Luo, and Li 2021 ; Geirhos et al. 2020 ) . As a result, the instance discrimination task suppresses domain-irrelevant features and amplifies domain-relevant features, thereby hindering domain generalization, as shown in Figure 1b. Another limitation is that previous approaches heavily depend on strong assumptions to align features across multi-domain. Harary et al. (2022) and Liu et al. (2023) assumed that if two edge-like images or two images from different domains are similar, then the images will share domain-irrelevant common features. Scalbert, Vakalopoulou, and Couzinié-Devy (2023) assumed that domain-irrelevant common features will not be lost or distorted even if an image is style transformed using a Fourier transform. To generalize domain-irrelevant common features, they tried to align features across multiple domains based on these strong assumptions. However, these strong assumptions can lead to learning biased features or ignoring other important features. They reduce the diversity of domain-irrelevant common features, and only a limited set of common features is extracted ( Robinson et al. 2021 ; Meng et al. 2022 ; Liu et al. 2023 ) . To achieve high performance on unseen domains, we need to effectively generalize diverse domain-irrelevant common features, rather than relying on strong assumption based feature alignments that may lead to unintended bias. To address these limitations, we propose a novel approach, DomCLP , Dom ain-wise C ontrastive L earning with P rototype Mixup for unsupervised domain generalization. First, we theoretically and experimentally demonstrate that some negative terms in InfoNCE can suppress domain-irrelevant common features and amplifies domain-relevant features. Building on this insight, we introduce the D omain-wise Con trastive Learning (DCon) to enhance domain-irrelevant common features while representation learning. Second, to effectively generalize diverse domain-irrelevant common features across multi-domain, we propose the P rototype Mix up Learning (PMix). In PMix, to generalize common features from multi-domain, we interpolate common features in each domain utilizing mixup ( Zhang et al. 2017 ) . We extract prototypes of features by k-means clustering, and train the model with mixed prototypes by mixup. It allows the model to effectively learn feature representations for unseen inter-manifold spaces while retaining diverse common feature information. Through our proposed method, DomCLP, the model effectively enhances and generalizes diverse common features. We validate our approach through experiments on PACS and DomainNet datasets, achieving state-of-the-art performance with improvements of up to 11.3% on the PACS 1% label fraction and 12.32% on the DomainNet 1% label fraction. Related Works Self-supervised Learning Self-supervised learning (SSL) aims to learn semantic features without relying on label information. Recently, most SSL methods have been based on contrastive learning with the information noise-contrastive estimation (InfoNCE) objective ( Gutmann and Hyvärinen 2010 ; Oord, Li, and Vinyals 2018 ) , which has shown outstanding performance ( Chen et al. 2020a ; Chen et al. 2020b ; He et al. 2020 ; Chen et al. 2020c ; Grill et al. 2020 ; Caron et al. 2020 ; Hu et al. 2021 ;