Solving the Catastrophic Forgetting Problem in Generalized Category Discovery

Xinzi Cao, Xiawu Zheng, Guanhong Wang, Weijiang Yu, Yunhang Shen, Ke Li, Yutong Lu, Yonghong Tian · 2025

Read Solving the Catastrophic Forgetting Problem in Generalized Category Discovery by Xinzi Cao, Xiawu Zheng, Guanhong Wang, Weijiang Yu, Yunhang Shen, Ke Li, Yutong Lu, Yonghong Tian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-05272.txt.

Introduction

Xinzi Cao and co-authors tackle forgetting in a learning task that must find both known and new classes. Their 2025 paper covers generalized category discovery, where unlabeled data mixes familiar and novel kinds. A model that adapts to new kinds can lose its grip on old ones, a failure called catastrophic forgetting.

Their method adds two constraints to a leading prior approach. One shapes how likely known-class samples spread in the new data, protecting old knowledge. The other forces two views of the same image to yield like predictions, which steadies learning. The combined method sharpened novel class sorting while holding known class scores. The design can plug into earlier methods, which widens its use for real recognition systems.

Catalog record

CREATORXinzi Cao, Xiawu Zheng, Guanhong Wang, Weijiang Yu, Yunhang Shen, Ke Li, Yutong Lu, Yonghong Tian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-05272

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Solving the Catastrophic Forgetting Problem in Generalized Category Discovery Xinzi Cao, Xiawu Zheng, Guanhong Wang, Weijiang Yu, Yunhang Shen, Ke Li, Yutong Lu, Yonghong Tian Abstract Generalized Category Discovery (GCD) aims to identify a mix of known and novel categories within unlabeled data sets, providing a more realistic setting for image recognition. Essentially, GCD needs to remember existing patterns thoroughly to recognize novel categories. Recent state-of-the-art method SimGCD transfers the knowledge from known-class data to the learning of novel classes through debiased learning. However, some patterns are catastrophically forgot during adaptation and thus lead to poor performance in novel categories classification. To address this issue, we propose a novel learning approach, LegoGCD , which is seamlessly integrated into previous methods to enhance the discrimination of novel classes while maintaining performance on previously encountered known classes. Specifically, we design two types of techniques termed as L ocal E ntropy Re g ularization (LER) and Dual-views Kullback–Leibler divergence c o nstraint (DKL). The LER optimizes the distribution of potential known class samples in unlabeled data, thus ensuring the preservation of knowledge related to known categories while learning novel classes. Meanwhile, DKL introduces Kullback–Leibler divergence to encourage the model to produce a similar prediction distribution of two view samples from the same image. In this way, it successfully avoids mismatched prediction and generates more reliable potential known class samples simultaneously. Extensive experiments validate that the proposed LegoGCD effectively addresses the known category forgetting issue across all datasets, e.g . , delivering a 7.74% and 2.51% accuracy boost on known and novel classes in CUB, respectively. Our code is available at: https://github.com/Cliffia123/LegoGCD . 1 Introduction (a) Baseline SimGCD Wen et al. 2023 (b) Ours LegoGCD Figure 1 : Visualization of the accuracy results in unlabeled dataset on CUB dataset Wah et al. 2011 during training. (a) shows a decrease in the accuracy of known (Old) classes ( green ) in the baseline as the accuracy of novel (New) classes ( orange ) increases. (b) demonstrates that LegoGCD solves the catastrophic forgetting problem and surpasses the baseline by a significant margin of 7.74 . Deep learning have achieved superior performance on computer vision tasks He et al. 2017 ; Ren et al. 2015 ; Szegedy et al. 2015 ; Ronneberger et al. 2015 ; Vaswani et al. 2017 ; Caron et al. 2021 , particularly on image classification He et al. 2016 ; Huang et al. 2017 ; Simonyan and Zisserman 2014 ; Sandler et al. 2018 ; Zoph et al. 2018 . However, conventional methods work in a close-world setting, where all training data comes with pre-defined classes. Consequently, deploying these models in real-world scenarios with potential novel classes becomes a considerable challenge. Furthermore, these achievements rely heavily on large-scale annotated dataset, which is not easily accessible in realistic scenarios. To address these challenges, a new paradigm of Generalized Category Discovery (GCD) Vaze et al. 2022b ; Zhang et al. 2023b ; Pu et al. 2023 ; An et al. 2023 ; Wen et al. 2023 ; Zhang et al. 2023a ; Fini et al. 2021 ; Han et al. 2022 has been proposed and attracts increasing attention in recent years. The goal of GCD is to train a classification model capable of recognizing both known and novel categories within unlabeled data. To be clear, GCD distinguishes itself from the Novel Class Discovery (NCD) Han et al. 2019 , which relies on an unrealistic assumption that all unlabeled data exclusively belongs to entirely new classes or patterns. In contrast, GCD adopts a more pragmatic assumption, acknowledging that unlabeled data encompasses a mixture of both known and novel categories. Consequently, GCD is more realistic compared to NCD, especially in real-world scenarios. Since GCD is partially based on the learned patterns, an intuitive idea is to classify the unlabeled data through a clustering-based approach Vaze et al. 2022b i.e . k -means. However, as the scale of datasets increases, the computational costs for clustering in the original GCD grow exponentially. To tackle this issue, Wen et al . introduce SimGCD Wen et al. 2023 , which replaces the clustering approach with a classifier. Specifically, SimGCD trains the classifier using a pseudo-labeling strategy, a technique that has demonstrated remarkable effectiveness in Semi-supervised Learning (SSL) Berthelot et al. 2019 ; Zhang et al. 2023a . Nevertheless, the pseudo labels of novel samples tend to be assigned as known classes due to the absence of guidance for novel class samples. In response, Wen et al . further propose to adopt class mean entropy to encourage the model to focus on novel categories, consequently generating high-quality pseudo labels for classifier training. As a result, SimGCD has achieved state-of-the-art performance and established itself as a robust baseline solution in the GCD setting. However, SimGCD Wen et al. 2023 still has a significant drawback. It encourages the model to focus more on novel classes by employing an entropy regularization, which unfortunately comes at the cost of known class accuracy, resulting in a catastrophic forgetting problem in known categories. To illustrate this issue, we have tracked the classification accuracy of known and novel categories on unlabeled data during each training epoch on CUB Wah et al. 2011 . As shown in Fig. 1(a) , the green curve represents the accuracy of known (Old) classes, while the orange curve represents the accuracy of novel (New) classes. Notably, we can easily observe this phenomenon, with the accuracy of novel classes improving, the accuracy of known categories initially increases to approximately 74% after 20 epochs but then drops to 64.44% in the end. We thus conclude that SimGCD faces catastrophic forgetting in known categories during training. To address the above issue, we propose a novel Local Entropy Regularization (LER) to preserve the knowledge of known categories. In particular, we first identify potential known samples using a threshold on their logits prediction like FixMatch Sohn et al. 2020 . Then, we employ the information entropy function to encourage the predictions of above selected known samples close to a more certain distribution, thereby increasing the confidence of samples from potential known classes. Consequently, this LER prevents known samples from being misclassified as novel classes and therefore preserves the knowledge related to known categories during learning novel categories. It’s worth noting that the model may occasionally miss potential known samples or select incorrect (novel) samples for LER. For example, when we have two augmentation view samples, \boldsymbol{x}_{i} and \boldsymbol{x}^{\prime}_{i} , from the same image, and \boldsymbol{x}_{i} has higher logits than the threshold while \boldsymbol{x}^{\prime}_{i} falls below it. In such cases, we can’t be certain whether the original image belongs to known classes, and this uncertainty may impact the effectiveness of LER. We argue that the predictions of the two view samples should be correctly aligned to ensure the quality of the chosen known samples. Therefore, we further propose a dual-view alignment scheme called Dual-views Kullback–Leibler divergence constraint (DKL), which employs Kullback–Leibler (KL) divergence to encourage the consistency of two views from the same image. To summarize, we propose a novel approach named LegoGCD, which integrates SimGCD Wen et al. 2023 with our proposed LER and DKL to address the problem of catastrophic forgetting. To validate the effectiveness of LegoGCD,

Read the complete text