Generalized Category Discovery via Token Manifold Capacity Learning
Luyao Tang, Kunze Huang, Chaoqi Chen, Cheng Chen · 2025
Read Generalized Category Discovery via Token Manifold Capacity Learning by Luyao Tang, Kunze Huang, Chaoqi Chen, Cheng Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14044.txt.
Introduction
Luyao Tang and co-authors help vision models sort the world into kinds they were never taught, in this 2025 paper. Generalized category discovery asks a model to cluster fresh images that mix known classes with brand-new ones. Older methods squeeze each cluster tight, which can flatten the rich variety inside a class and blur its meaning.
Their method, MTMC, instead grows the capacity of each class's token space, a measure of how much varied structure it can hold. A norm over the token spread serves as that measure, keeping representations rich and well formed. Tests on coarse and fine-grained data sets beat earlier methods at both sorting accuracy and guessing how many classes exist. The authors report less collapse of the representation space and cleaner borders between classes.
Catalog record
| CREATOR | Luyao Tang, Kunze Huang, Chaoqi Chen, Cheng Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14044
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalized Category Discovery via Token Manifold Capacity Learning Luyao Tang, Kunze Huang, Chaoqi Chen, Cheng Chen Abstract Generalized category discovery (GCD) is essential for improving deep learning models’ robustness in open-world scenarios by clustering unlabeled data containing both known and novel categories. Traditional GCD methods focus on minimizing intra-cluster variations, often sacrificing manifold capacity, which limits the richness of intra-class representations. In this paper, we propose a novel approach, Maximum Token Manifold Capacity (MTMC), that prioritizes maximizing the manifold capacity of class tokens to preserve the diversity and complexity of data. MTMC leverages the nuclear norm of singular values as a measure of manifold capacity, ensuring that the representation of samples remains informative and well-structured. This method enhances the discriminability of clusters, allowing the model to capture detailed semantic features and avoid the loss of critical information during clustering. Through theoretical analysis and extensive experiments on coarse- and fine-grained datasets, we demonstrate that MTMC outperforms existing GCD methods, improving both clustering accuracy and the estimation of category numbers. The integration of MTMC leads to more complete representations, better inter-class separability, and a reduction in dimensional collapse, establishing MTMC as a vital component for robust open-world learning. Code is here . 1 Introduction Figure 1: (a) GCD is constrained by dimensional collapse due to strong clustering, leading to mixed class features and limited representational capacity. (b) MTMC enhances the class token manifold capacity, improving representational completeness and unlocking the model’s full potential in the open world. Machine learning models encounter substantial challenges when deployed in real-world settings due to the intractability of objects in the open world [ 65 , 43 , 56 ] . The diversity of real-world objects exceeds the scope of data collected for training [ 58 ] , and labeled data covers even fewer categories. Traditional deep learning models, trained on predefined categories, are ill-equipped to handle new category samples. To enhance the reliability of model deployment in real-world scenarios, open-world learning has emerged, aiming to identify and categorize unknown samples [ 22 , 17 , 50 ] in new environments. A plethora of approaches have been proposed to identify and categorize unknown samples, such as open-set recognition (OSR) [ 17 ] and novel class discovery (NCD) [ 22 ] . However, OSR treats all unknown samples as a single category. On the other hand, NCD relies on a strong assumption that all unlabeled samples encountered come from new classes. To relax this assumption, Generalized Category Discovery (GCD) [ 50 ] permits the presence of known classes within unlabeled data. GCD relies on contrastive learning [ 10 ] or prototype learning [ 57 ] to reduce the distance between semantically identical samples in the embedding space. However, current approaches face significant challenges: (i) the compressed inter-class distribution may lead to the loss of useful information. This results in each cluster being unable to fully represent the semantic details within a class, leading to bias within the feature space, which is detrimental to category discovery. (ii) Bias prevents the inter-class decision boundaries from aligning with the boundaries between real-world categories, making it impossible for the model to accurately separate clusters during the discovery of categories (Figure 6 demonstrates that incomplete intra-class representations result in low clustering accuracy). To this premise, we challenge the status quo by raising an open question: Can deep models accurately separate new semantics during the category discovery by enhancing the completeness of intra-class representations ? The GCD aims to partition data points into distinct clusters, which are distributed on low-dimensional manifolds [ 46 , 53 ] within high-dimensional spaces. Recently, Maximum Manifold Capacity Representations [ 60 , 44 , 24 ] have sought to learn representations by examining the separability of manifolds. In this context, manifolds containing views of the same scene are both compact and low-dimensional, while manifolds corresponding to different scenes are separated. Building on manifold capacity concept, we introduce Maximum Class Token Manifold Capacity ( MTMC ). Specifically, (1) we associate low intra-class representation completeness with low manifold capacity. Our research narrows the focus from the entire feature space to the intra-class feature space, examining manifold capacity at a more granular token level . (2) We consider the representation of a sample as its manifold, with the sample representation in GCD derived from the class token provided by Vision Transformer (ViT) [ 12 ] . Under the attention mechanism, the class token refines the patch features, thus serving as a proxy for the sample manifold . (3) Given that a comprehensive and information-rich class token manifold necessitates a large capacity, we measure manifold capacity using the nuclear norm of class token and aim to maximize this norm. (4) MTMC enhances the completeness of sample representation, enabling clusters to capture more intra-class semantic details while preventing dimensionality collapse, thus improving inter-class separability accuracy. Our contributions are summarized as follows: • We introduce MTMC to enhance representation completeness and analyze its effectiveness in addressing dimensional collapse and improving von Neumann entropy. • We maximize the nuclear norm of the class token’s singular value kernel to increase its manifold capacity, enabling clusters to capture more intra-class semantic details. • MTMC is easy to implement. Experiments on coarse- and fine-grained datasets demonstrate its effectiveness in improving precision and the accuracy of category number estimation. 2 Preliminary and Motivation 2.1 Notation and Optimization of GCD For each dataset, consider a labeled subset \mathcal{D}_{l}=\{(\mathbf{x}_{i}^{l},y_{i}^{l})\}\subset\mathcal{X}\times\mathcal{Y}_{l} and an unlabeled subset \mathcal{D}_{u}=\{(\mathbf{x}_{i}^{u},y_{i}^{u})\}\subset\mathcal{X}\times\mathcal{Y}_{u} . Only known classes can be found in \mathcal{D}_{l} , while \mathcal{D}_{u} encompasses known and novel classes, translating to \mathcal{Y}_{l}=\mathcal{C}_{known} and \mathcal{Y}_{u}=\mathcal{C}_{known}\cup\mathcal{C}_{novel} . The task of models involves clustering on both the known and novel classes in \mathcal{D}_{u} . The number of novel classes represented as K_{novel} can be determined beforehand [ 50 , 42 , 64 ] . The functions f(\cdot) and g(\cdot) perform as the feature extractor and projection head, respectively. Both the feature \mathbf{h}_{i}=f(\mathbf{x}_{i}) and the projected embedding \mathbf{z}_{i}=g(\mathbf{h}_{i}) are under L-2 normalization. For compact clustering, GCD consists of supervised and unsupervised contrastive learning or prototype learning (discussed in Appendix E.1 ). While these methods achieve clustering, they overly prioritize compact clusters, overlooking incomplete intra-class representations that fail to capture the full distribution, resulting in low manifold capacity. More details are provided in Appendix A . 2.2 Manifold Capacity Theory Manifold capacity theory [ 60 , 24 ] evaluates the efficiency of neural representation coding by mapping high-dimensional data to low-dimensional manifolds representing different objects or categories. Key concepts include: (1) Manifold Radius : R\_M=\sqrt{\frac{1}{P}\sum\_{i=1}^{P}\lambda\_i^{2}} , where \lambda\_i are the eigenvalues of the covariance matrix of points on the manifold, and P is the number of points. It measures the manifold’s size relative to its centroid. (2) Manifold Dimensionality : D\_M=\frac{\left(\sum\_{i=1}^{P}\lambda\_i\right)^{2}}{\sum\_{i=1}^{P}\lambda\_i^{2}} , quantifying the manifold’s expansion along its major directions. (3) Manifold Capacity : \alpha\_C=\phi\left(R\_M\sqrt{D\_M}\right) , where \phi(\cdot) is a monotonically decreasing function. This represents the maximum number of linearly separable manifolds in a feature space. Manifold capacity, derived from radius and dimensionality, determines the number of distinguishable categories in high-dimensional space. Optimizing it enhances coding efficiency by refining the manifold’s geometric properties. 2.3 Our Thoughts on Why GCD Needs a Manifold Capacity Quest Based on above, GCD has made notable progress in clustering both