SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery
Zhenqi He, Yuanpei Liu, Kai HanVisual AI Lab, The University of Hong Kong {zhenqi_he, ypliu0}@connect.hku.hk [email protected] · 2025
Read SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery by Zhenqi He, Yuanpei Liu, Kai HanVisual AI Lab, The University of Hong Kong {zhenqi_he, ypliu0}@connect.hku.hk [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18740.txt.
Introduction
Zhenqi He, Yuanpei Liu, and Kai Han tackle a sorting task. It is called generalized category discovery. Some images come labeled, and the rest do not. The unlabeled ones may belong to new classes entirely. Past methods leaned on a single level of meaning. Some leaned on hand-built class hierarchies. Both choices limit how far the methods transfer. Their framework, SEAL, uses natural hierarchies found in data.
One part of SEAL builds soft negative examples. It uses hierarchical similarity to weight them. Near-miss classes are treated more gently than distant ones. That fixes a bluntness in standard contrastive losses. Another part aligns predictions across levels of detail. SEAL reached leading results on fine-grained benchmarks. Pet breeds were one test set. A large plant specimen collection was another. It also generalized well to coarser data sets.
Catalog record
| CREATOR | Zhenqi He, Yuanpei Liu, Kai HanVisual AI Lab, The University of Hong Kong {zhenqi_he, ypliu0}@connect.hku.hk [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18740
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery Zhenqi He, Yuanpei Liu, Kai HanVisual AI Lab, The University of Hong Kong {zhenqi_he, ypliu0}@connect.hku.hk [email protected] Abstract This paper investigates the problem of Generalized Category Discovery (GCD). Given a partially labelled dataset, GCD aims to categorize all unlabelled images, regardless of whether they belong to known or unknown classes. Existing approaches typically depend on either single-level semantics or manually designed abstract hierarchies, which limit their generalizability and scalability. To address these limitations, we introduce a SE mantic-aware hier A rchical L earning framework ( SEAL ), guided by naturally occurring and easily accessible hierarchical structures. Within SEAL, we propose a Hierarchical Semantic-Guided Soft Contrastive Learning approach that exploits hierarchical similarity to generate informative soft negatives, addressing the limitations of conventional contrastive losses that treat all negatives equally. Furthermore, a Cross-Granularity Consistency (CGC) module is designed to align the predictions from different levels of granularity. SEAL consistently achieves state-of-the-art performance on fine-grained benchmarks, including the SSB benchmark, Oxford-Pet, and the Herbarium19 dataset, and further demonstrates generalization on coarse-grained datasets. Project page: https://visual-ai.github.io/seal/ 1 1 footnotetext: Equal contribution. 2 2 footnotetext: Corresponding author. 1 Introduction The field of computer vision has undergone substantial progress in various tasks, including classification simonyan2014very ; he2016deep , object detection girshick2015fast ; ren2015faster , and segmentation he2017mask ; transnuseg ; wang2020solo . Such advancements have largely been driven by access to large-scale, human-annotated datasets deng2009imagenet ; lin2014microsoft . However, models trained on these datasets are constrained to a closed-world paradigm, limiting their predictions to the predefined labels within the training set. In contrast, there exists a wealth of unlabelled data in the open world. To capitalize on the unlabelled data, a variety of Semi-Supervised Learning (SSL) techniques chapelle2009semi have been proposed, yielding notable improvements over traditional supervised learning methods. Despite substantial success in various tasks berthelot2019mixmatch ; xu2021end ; chen2021semi , most existing SSL methods are designed under the closed-set assumption, wherein the training and test datasets share an identical set of classes. Category discovery, initially introduced as Novel Category Discovery (NCD) han2019learning ; he2025category and later extended to Generalized Category Discovery (GCD) vaze2022generalized ; he2025category , has recently emerged as a compelling open-world problem, attracting significant attention. Unlike SSL, GCD tackles the challenges where the unlabelled subset may include instances from both known and unknown classes. Its primary objective is to utilise knowledge gained from labelled data to effectively categorize all samples within the unlabelled data. Concurrently, an equivalent task named Open-world Semi-Supervised Learning (OSSL) cao2022open has also been introduced. The effectiveness of GCD is rooted in the efficient transfer of knowledge from known categories to cluster samples of both known and novel categories. As a transfer clustering task han2019learning , hierarchical information has been demonstrated to be effective in GCD rastegar2023learn ; hao2023cipr and similarly in the parallel task of OSSL wang2023discover , particularly with fine-grained datasets vaze2022semantic . In rastegar2023learn , the hierarchical structure is composed of abstract concepts as an implicit binary tree, where each node represents an increasingly abstract concept derived from shared binary code prefixes, and in RastegarECCV2024 , the hierarchical structure is implicitly formed by incrementally halving the category count as the hierarchy level increases with hierarchical pseudo-labeling to provide soft supervision for the training. Similarly, CiPR hao2023cipr constructs abstract hierarchies by iteratively merging data partitions through semi-supervised clustering. The hierarchical tree in wang2023discover consists of manually defined upper and lower levels that represent different granularities, with the number of categories per level controlled by hyperparameters. More recently, HypCD liu2025hyperbolic implicitly models hierarchies via hyperbolic embeddings, achieving strong performance and underscoring the importance of hierarchical information for GCD. These methods build hierarchical levels from abstract, weakly supervised structures that may introduce noise and errors, ultimately affecting GCD performance. As illustrated in Fig. 1 (a), the ‘Siberian Tiger’ , ‘Bengal’ and ‘Egyptian Mau’ can be merged into a single category while the ‘Red Fox’ can be divided into multiple categories. Additionally, the high similarity among categories may result in some images of the ‘Basset Hound’ being incorrectly merged with those of the ‘Beagle’ . This observation naturally prompts us to consider: whether the intrinsic, semantically grounded taxonomies present in the real world can serve as more reliable guides . In botanical research, taxonomists commonly use labelled specimens of known species to classify newly collected, unlabelled samples into existing taxonomic hierarchies or to identify unseen species mace2004role ; karbstein2024species . Similarly, studies in closed-world visual classification Chang2021Labrador ; 9609630 ; du2021progressive have shown that hierarchical structures enhance classification. From an information-theoretic perspective, we further deduce that such semantic-aware hierarchies yield a tighter mutual information bound, providing a principled foundation for our design. Figure 1: Comparison of SEAL and previous methods rastegar2023learn ; wang2023discover using hierarchical learning. (a) In previous attempts, several upper and lower levels as well as abstract concepts are defined around the ground-truth level, which may cause errors in the hierarchical structure. (b) In our method, we propose to utilise the semantic information at different levels to enhance the GCD performance. To this end, we propose the SE mantic-aware hier A rchical L earning ( SEAL ) framework for GCD. Unlike previous approaches that either focus exclusively on single-granularity information vaze2022generalized ; vaze2023no ; wang2024sptnet ; wen2023parametric ; liu2025debgcd or rely on abstract hierarchical cues rastegar2023learn ; hao2023cipr ; wang2023discover ; RastegarECCV2024 ; liu2025hyperbolic , SEAL effectively leverages the naturally occurring semantic hierarchies without manual design (shown in Fig. 1 (b)) and incorporates several innovative techniques tailored specifically for this task. Firstly , we implement a multi-task training paradigm that facilitates the simultaneous discovery of categories across several different semantic levels. Secondly , we introduce a Cross-Granularity Consistency (CGC) module to align the class predictions from different granularities. Thirdly , we propose the Hierarchical Semantic-guided Soft Contrastive Learning to capture uncertainty in contrastive learning, ensuring that not all negative samples are treated equally. By effectively integrating these components into a cohesive framework, SEAL can be trained end-to-end in a single stage. In summary, we make the following contributions in this paper: (i) We propose SEAL , a novel framework specifically designed to tackle the challenging GCD task by leveraging the inherent semantic hierarchies, marking the first exploration of this aspect. (ii) Within the SEAL framework, we develop two novel components: the Cross-Granularity Consistency (CGC) module and Hierarchical Semantic-guided Soft Contrastive Learning. These components function synergistically to significantly enhance the model’s category discovery capabilities. (iii) Through extensive experimentation on public GCD benchmarks, SEAL consistently demonstrates its effectiveness and achieves superior performance, especially on fine-grained datasets. 2 Related Work Category Discovery. Novel Category Discovery (NCD) was first articulated in han2019learning , establishing a pragmatic framework for transferring knowledge from known categories to clusters of unseen categories, framed as a transfer clustering problem. Subsequently, a variety of methods have emerged to advance the research domain han2020automatically ; han2021autonovel ; jia2021joint ; zhao2021novel ; ncl ; fini2021unified . Generalized Category Discovery (GCD) extends the NCD framework by relaxing its assumptions, incorporating unlabelled data that features samples from both known and unknown classes vaze2022generalized . Recent studies cao2022open ; hao2023cipr ; pu2023dynamic ; joseph2022novel ; cendra2024promptccd ; wang2024hilo ; liu2025debgcd have explored a range of strategies to