Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective
Wei Feng, Zongyuan Ge · 2025
Read Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective by Wei Feng, Zongyuan Ge (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00573.txt.
Introduction
Wei Feng and Zongyuan Ge tackle category discovery when the data itself shifts domains. Generalized category discovery uses labeled examples of known classes to cluster unlabeled data that may hide new classes. Under distribution shift, performance usually crumbles, and unlabeled samples may also come from unknown domains. Their framework, FREE, reads the problem in the frequency domain, where domain signatures show clearly.
Those amplitude gaps first separate known from unknown domains. Two perturbation strategies follow: swapping amplitude parts across domains to teach adaptation, and perturbing within the unknown domain for robustness. Extended contrastive and clustering losses guide training, and a resampling step focuses effort on groups that are hard to cluster. Across benchmarks, the method finds known and new categories more steadily under shift.
Catalog record
| CREATOR | Wei Feng, Zongyuan Ge |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 101 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00573
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective Wei Feng, Zongyuan Ge Abstract Generalized Category Discovery (GCD) aims to leverage labeled samples from known categories to cluster unlabeled data that may include both known and unknown categories. While existing methods have achieved impressive results under standard conditions, their performance often deteriorates in the presence of distribution shifts. In this paper, we explore a more realistic task: Domain-Shifted Generalized Category Discovery (DS_GCD), where the unlabeled data includes not only unknown categories but also samples from unknown domains. To tackle this challenge, we propose a F requency-guided Gene r alized Cat e gory Discov e ry framework (FREE) that enhances the model’s ability to discover categories under distributional shift by leveraging frequency-domain information. Specifically, we first propose a frequency-based domain separation strategy that partitions samples into known and unknown domains by measuring their amplitude differences. We then propose two types of frequency-domain perturbation strategies: a cross-domain strategy, which adapts to new distributions by exchanging amplitude components across domains, and an intra-domain strategy, which enhances robustness to intra-domain variations within the unknown domain. Furthermore, we extend the self-supervised contrastive objective and semantic clustering loss to better guide the training process. Finally, we introduce a clustering-difficulty-aware resampling technique to adaptively focus on harder-to-cluster categories, further enhancing model performance. Extensive experiments demonstrate that our method effectively mitigates the impact of distributional shifts across various benchmark datasets and achieves superior performance in discovering both known and unknown categories. Figure 1: Illustration of Domain-Shifted GCD setting (DS_GCD) and the traditional GCD setting. In the DS_GCD setting, the model needs to categorize known and unknown categories from both known and unknown domains. 1 Introduction Deep learning has achieved remarkable success in visual recognition tasks under closed-world assumptions qi2023cross ; qi2025cross ; hang2020local . However, visual concepts in the real world are infinite, open-ended, and constantly evolving, while traditional deep learning models are limited to recognizing a predefined set of categories and cannot handle unseen concepts. In contrast, humans can recognize new concepts based on existing knowledge. Inspired by this, Generalized Category Discovery (GCD) vaze2022generalized ; wen2023simgcd has been proposed to enable models to simultaneously recognize both known and new categories. However, existing GCD methods typically assume that labeled and unlabeled data come from the same domain, which is often not the case in real-world scenarios. In practice, data often exhibit both label and domain shifts. For instance, in clinical applications, medical images collected from different sources may differ significantly in both appearance and statistical properties, and new disease categories may emerge unexpectedly. Current GCD methods suffer substantial performance degradation when faced with both unknown categories and domain shifts wang2024hilo . This work addresses a more realistic and challenging problem setting, referred to as Domain-Shifted Generalized Category Discovery (DS_GCD), where unlabeled data may belong to both unseen categories and entirely new domains, as illustrated in Fig. 1 . DS_GCD requires models to identify novel categories and domains using only labeled data from known categories, while generalizing across both semantic and distributional variations. This setting poses major challenges for existing methods. First, most unsupervised domain adaptation (UDA) approaches cannot be directly applied to DS_GCD, as new categories lack label supervision, and relying on unlabeled data often leads to poor performance. Second, current GCD methods lack mechanisms to address domain shifts, resulting in significant performance drops with novel domains. Figure 2: Clustering accuracy of all categories using FREE, SimGCD+FDA, and SimGCD. It can be observed that FREE outperforms SimGCD+FDA both in final clustering accuracy and convergence speed. In contrast, random transformation in SimGCD+FDA may even hinder the learning of unknown domains, potentially leading to negative transfer effects. To address the aforementioned challenges, we propose F requency-guided Gene r alized Cat e gory Discov e ry framework (FREE), a method that tackles the DS_GCD task from a frequency-domain perspective. Our work is inspired by Fourier Domain Adaptation (FDA) yang2020fda , which bridges domain gaps by leveraging Fourier Transform to exchange low-frequency components (typically associated with image style) across different domains in the frequency domain. However, directly applying the FDA to the GCD task poses two key challenges. First, FDA assumes that randomly exchanging amplitude components between images from different domains can enhance performance on the target domain. However, in the DS_GCD setting, where domain labels for individual samples are unavailable and the direction of style transfer cannot be controlled, randomly swapping amplitude components between images may lead to unstable training and negative transfer, as shown in Figure 2 , especially in the unknown domain. This is because transforming unknown domain samples into the style of the known domain may distort their original structure or semantic content. Unlike the known domain, the unknown domain lacks any supervision and relies solely on pseudo-labels to learn clustering, making the learning process more vulnerable to such instability. Furthermore, this strategy primarily focuses on global domain adaptation while overlooking intra-class style variations across domains. To address these issues, we propose a divide-and-conquer strategy that overcomes these limitations in a more controlled and class-aware manner. Specifically, we first design a frequency-based domain separation strategy, which separates known-domain and unknown-domain samples based on the amplitude differences across different sample sets. We then propose a cross-domain frequency perturbation strategy that integrates amplitude components from different domains in a class-aware manner to generate hybrid samples and bridge distribution gaps. Additionally, for unknown domains, we introduce an intra-domain frequency perturbation strategy to enhance the model’s robustness to domain-specific variations. Furthermore, we extend the self-supervised contrastive objective and semantic clustering loss to better guide the training process. These enhancements enable the model to effectively leverage cross-domain frequency knowledge and discover novel categories from unlabeled images, even under domain shift conditions. Finally, we propose a clustering-difficulty-aware resampling strategy, which dynamically evaluates the clustering difficulty of different categories and encourages the model to focus more on the harder ones. In summary, our contributions are as follows: 1) We introduce a novel frequency-domain approach to address the GCD task under domain shifts. 2) We propose frequency-based domain separation and perturbation strategies to improve domain robustness and cross-domain generalization. 3) We enhance learning with extended contrastive learning and clustering objectives. 4) We develop a clustering-difficulty-aware resampling mechanism to focus learning on challenging categories. 5) Extensive experimental results show that our method is able to remarkably discover new categories while minimizing the effects of domain shift, and performs far better than state-of-the-art methods on all datasets. 2 Related Works Category Discovery Category discovery tasks are typically categorized into Novel Class Discovery (NCD) feng2023towards and Generalized Category Discovery (GCD) feng2025neighbor settings. NCD was initially introduced by han2021autonovel , with the goal of clustering unlabeled samples from novel categories using only labeled data from known categories. Early methods primarily relied on ranking statistics and self-supervised contrastive learning to transfer discriminative knowledge from known to unknown categories han2021autonovel ; zhao2021novel . Subsequently, UNO fini2021unified utilized a self-labeling strategy based on the Sinkhorn-Knopp (SK) algorithm, integrating multiple objectives into a unified framework to significantly enhance discovery performance. Later works further explored category-level semantic relationships to improve clustering accuracy for unknown categories li2023modeling ; gu2023class . GCD extends the NCD setting by allowing unlabeled samples to contain both known and unknown