Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning
Huiyu Yi, Zhiming Xu, Dunwei Tu, Zhicheng Wang, Baile Xu, Furao Shen · 2026
Read Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning by Huiyu Yi, Zhiming Xu, Dunwei Tu, Zhicheng Wang, Baile Xu, Furao Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-11904.txt.
Introduction
Huiyu Yi and five colleagues improve class-incremental learning, where a model must absorb new classes without forgetting old ones. The popular nearest-class-mean classifier assumes features collapse to single points, but drift in incremental training leaves classes as complex manifolds instead. Their HC-SOINN classifier captures manifold topology through a local-to-global representation, while their STAR method tracks pointwise trajectories to deform the topology as features drift. Plugged into seven state-of-the-art methods, it improved all of them.
Catalog record
| CREATOR | Huiyu Yi, Zhiming Xu, Dunwei Tu, Zhicheng Wang, Baile Xu, Furao Shen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74855 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.11904
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Point-wise Neural Collapse: A Topology-Aware Hierarchical Classifier for Class-Incremental Learning Huiyu Yi, Zhiming Xu, Dunwei Tu, Zhicheng Wang, Baile Xu, Furao Shen Abstract The Nearest Class Mean (NCM) classifier is widely favored in Class-Incremental Learning (CIL) for its superior resistance to catastrophic forgetting compared to Fully Connected layers. While Neural Collapse (NC) theory supports NCM’s optimality by assuming features collapse into single points, non-linear feature drift and insufficient training in CIL often prevent this ideal state. Consequently, classes manifest as complex manifolds rather than collapsed points, rendering the single-point NCM suboptimal. To address this, we propose Hierarchical-Cluster SOINN (HC-SOINN), a novel classifier that captures the topological structure of these manifolds via a “local-to-global” representation. Furthermore, we introduce Structure-Topology Alignment via Residuals (STAR) method, which employs a fine-grained pointwise trajectory tracking mechanism to actively deform the learned topology, allowing it to adapt precisely to complex non-linear feature drift. Theoretical analysis and Procrustes distance experiments validate our framework’s resilience to manifold deformations. We integrated HC-SOINN into seven state-of-the-art methods by replacing their original classifiers, achieving consistent improvements that highlight the effectiveness and robustness of our approach. Code is available at https://github.com/yhyet/HC_SOINN . 1 Introduction Class-Incremental Learning (CIL) aims to enable models to learn new categories sequentially without catastrophically forgetting previously acquired knowledge ( 2 ; 32 ; 12 ) . While traditional Fully Connected (FC) classifiers are highly susceptible to forgetting due to severe weight bias towards new tasks ( 13 ) , the Nearest Class Mean (NCM) classifier ( 15 ) has emerged as a cornerstone of modern CIL frameworks owing to its superior robustness. This empirical preference has gained significant theoretical traction through the lens of Neural Collapse (NC) ( 14 ; 28 ) . NC theory proves that during the terminal phase of training, intra-class features collapse into their respective class means, rendering the NCM classifier equivalent to an optimal linear classifier. Consequently, NCM-based approaches have become the de facto standard for maintaining feature-classifier alignment in incremental scenarios ( 31 ; 29 ; 35 ; 22 ) . However, the assumption that models reach this ideal terminal phase is frequently violated in practical CIL settings due to three critical factors. First, models in the initial task are often not trained to the point of collapse to avoid severe overfitting. Second, during incremental steps, training on new classes is typically constrained (e.g., via regularization or distillation) to preserve old class performance, leading to insufficient convergence. Finally, the pervasive “feature drift” problem causes the representations of old classes to shift as the backbone updates ( 30 ) , further distancing the model from the collapsed state. Under these constraints, the NC1 property (zero intra-class variance) is compromised ( 14 ) . Class features no longer collapse into single points but instead reside on complex, high-dimensional manifolds. While NCM is robust to simple linear translations, it fails to represent these manifolds when they undergo non-linear drift, potentially forming “dumbbell” or “crescent” structures that deviate from the single-prototype assumption ( 1 ) . In this paper, we challenge the reliance on single-prototype classifiers and propose a topology-aware representation for CIL. We introduce HC-SOINN (Hierarchical-Cluster SOINN), a novel classifier designed to capture the intrinsic structure of class manifolds. By integrating hierarchical clustering with an improved Self-Organizing Incremental Neural Network (SOINN) ( 3 ) , HC-SOINN represents each category through a combination of “local centers” and a “global center.” During inference, these topological points are fused to provide a decision boundary that more faithfully respects the manifold’s geometry. Furthermore, our Procrustes analysis reveals that feature drift in CIL is far from a simple rigid transformation ( 5 ) , with normalized distances frequently reaching 0.3-0.4 in later tasks. This significant deviation indicates that the feature drift is predominantly non-linear, involving complex deformations that cannot be approximated by simple rigid transformations. Although the proposed HC-SOINN, with its topological representation, is inherently more resilient to such deformations than the single-prototype NCM, its capacity to withstand severe non-linear distortion remains finite. To bridge this gap, we propose STAR (Structure-Topology Alignment via Residuals). Leveraging the multi-node granularity of HC-SOINN, STAR moves beyond global alignment to perform flexible, pointwise trajectory tracking. This mechanism enables the learned manifold structure to actively deform and precisely adapt to the complex evolution of the feature space. Our contributions are summarized as follows: • We revisit NCM’s optimality under Neural Collapse and use Procrustes analysis to empirically reveal that feature drift causes significant non-linear structural deformations, exposing the failure of single-prototype methods. • We propose the HC-SOINN classifier, which utilizes a hierarchical topological structure to represent class manifolds more accurately than single-prototype methods. • We introduce STAR to shift the paradigm from drift resistance to drift adaptation. Leveraging pointwise trajectory tracking, STAR enables the topology to actively deform to match complex non-linear feature evolution. • Extensive experiments on three mainstream CIL benchmarks and integration with seven state-of-the-art methods demonstrate the effectiveness and robustness of our approach. 2 Related Works Continual Learning with Pre-trained Models leverages the robust representations of PTMs to mitigate forgetting ( 32 ) . Parameter-Efficient Fine-Tuning (PEFT) approaches, such as DualPrompt ( 26 ) , CODA-Prompt ( 18 ) , MQMK ( 21 ) , SEMA ( 25 ) , and CL-LoRA ( 6 ) , adapt backbones through task-specific prompts, attention-based weighted summation, or modularized adapters and LoRA modules. Alternatively, representation-based methods focus on exploiting frozen features; for instance, SimpleCIL ( 31 ) uses class prototypes as baselines, while APER and EASE ( 33 ) further enhance performance via embedding aggregation or expandable subspace ensembles. Beyond these, KAC introduces a non-linear classifier based on Kolmogorov-Arnold networks to refine decision boundaries and mitigate drift ( 8 ) . Neural Collapse Theory Neural Collapse theory describes a phenomenon observed during the terminal phase of training, mathematically formalized through four key properties ( 14 ) : Variability Collapse (NC1), where intra-class feature variance approaches zero as samples converge to their class means \mu_{k} ; Convergence to Simplex ETF (NC2), where these class means form a geometrically optimal Simplex Equiangular Tight Frame; Feature-Classifier Alignment (NC3), implying classifier weights align perfectly with the normalized class means; and Simplification to NCC (NC4), where the decision rule becomes equivalent to the NCM classifier. In the context of Few-Shot Class Incremental Learning (FSCIL) ( 20 ) , NC theory justifies the use of NCM-based classifiers to maintain feature-classifier alignment across incremental steps, inspiring methods that utilize fixed ETF structures or prototype alignment to integrate new categories with limited data while preserving old class knowledge ( 28 ; 22 ) . Self-Organizing Incremental Neural Network (SOINN) is a classic neural network designed for unsupervised incremental learning ( 3 ) . It represents the topology of non-stationary data streams by dynamically adjusting nodes and edges. To extend its capabilities to supervised tasks, several variants have been developed. For instance, the Enhanced SOINN adopts a single-layer structure with improved density-based noise removal ( 4 ) . Furthermore, the Adjusted SOINN Classifier incorporates class labels into the incremental learning process ( 16 ) , enabling the network to handle online classification and adapt to overlapping class distributions. 3 Preliminary & Motivation 3.1 Problem Setting Class-Incremental Learning (CIL) aims to learn from a