Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning
Katarzyna Filus, Kamil Faber, Roberto Corizzo, Christopher Kanan · 2026
Read Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning by Katarzyna Filus, Kamil Faber, Roberto Corizzo, Christopher Kanan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16374.txt.
Introduction
Filus and three others propose a diagnostic tool for concept-level forgetting in supervised continual learning, using sparse autoencoders to define a task-based feature space. Prior work tracked forgetting through task scores or coarse drift metrics, without splitting output-level access from finer internal structure changes. Their tool splits forgetting into seeming concept loss, revival, and decodability. Much lost concept information can often be won back with linear readouts, with decodability fading as more tasks arrive.
Catalog record
| CREATOR | Katarzyna Filus, Kamil Faber, Roberto Corizzo, Christopher Kanan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66271 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.16374
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning Katarzyna Filus, Kamil Faber, Roberto Corizzo, Christopher Kanan Abstract Continual learning studies how models can adapt to new tasks while retaining previously acquired knowledge. Although a broad spectrum of methods has been proposed to mitigate catastrophic forgetting, the field remains predominantly performance-driven, with limited insight into what forgetting actually corresponds to within the vision model’s representation space. Prior work has primarily analyzed forgetting through task-level performance or coarse measures of representational drift, without disentangling output-level accessibility from changes in finer-grained internal structure. To this end, we propose a diagnostic framework that leverages Sparse Autoencoders (SAEs) to define a task-anchored latent feature space, enabling analysis of how task-specific information evolves at a finer granularity, where individual SAE latents are treated as concept proxies for recurring and relatively disentangled visual patterns in the model’s internal computations. Within this framework, we decompose forgetting into apparent concept deletion, recoverability, and decodability. We show that a large portion of seemingly lost concept-level information can often be recovered under linearity assumption, with concept decodability degrading as more tasks are introduced. Overall, our findings suggest that a significant part of concept-level forgetting can be attributed to changes in the representational accessibility rather than complete information erasure. 1 Introduction The degradation of previously learned knowledge during sequential training, known as catastrophic forgetting, remains a central challenge in continual learning 28 . Various forgetting mitigation strategies have been proposed, including rehearsal-based methods 25 , regularization and distillation 20 , as well as hybrid methods 5 . Despite this progress, evaluation remains largely performance-driven, equating forgetting to decrease in task accuracy after learning new tasks, while leaving unresolved the fundamental questions of how forgetting manifests within models 22 . Recent works notice this gap and attempt to shift from the output-level evaluation to representations 14 ; 22 ; 6 ; 12 . Previous studies measure representational drift using mostly similarity measures 18 ; 14 ; 22 . Other works 6 ; 12 show that much apparent performance loss can be recovered via retrained classifiers or representation constraints. However, these analyses remain global and tied to task-level performance, and thus do not capture how fine-grained information is organized, transformed, or lost. Figure 1: The taxonomy of concept transitions for task t data after task t+1 training. Similarly to how humans recognize objects by combining recurring concepts, recent progress in mechanistic interpretability enables the study of neural representations in terms of more disentangled and interpretable features used to perform learned tasks 2 . It is particularly important because deep networks are considered to rely on superposition, where multiple concepts – understood as recurring patterns within the model computation – are encoded in overlapping neural directions, making feature-level analysis difficult 2 . Methods such as Sparse Autoencoders (SAEs) 15 and transcoders 8 map representations into higher-dimensional sparse latent spaces that can isolate more atomic and interpretable features. Although these latent dimensions do not correspond to the ground-truth semantic concepts, they often capture recurring and relatively disentangled patterns in model computations, serving as useful concept proxies that often align with abstract motifs such as furry animals or red objects 2 . In the following, we refer to them as concepts and leverage them as a structured basis for analyzing how fine-grained information evolves during continual learning. Studying forgetting at a fine-grained level should be as important as in terms of accuracy degradation, indicating whether the model retains knowledge and supports stable interpretations rather than shifts representations. However, despite the potential of mechanistic interpretability, it has been scarcely applied to this problem. The only work 22 uses transcoders to analyze forgetting as geometric transformations of features in a shared latent space. However, it focuses on how features transform, rather than analyzing task-specific information accessibility. In particular, it does not quantify concept-level activation dynamics, the decodability of individual features, or the extent to which seemingly lost information can be recovered under standard readout assumptions. As a result, the relationship between representational change and functional forgetting remains insufficiently understood. In this work, we address this gap by proposing a concept-level framework for analyzing forgetting through the accessibility of task-specific information. Using Sparse Autoencoders (SAEs), we define a fixed, task-anchored latent space in which individual features act as concept proxies, enabling fine-grained tracking of how information evolves during continual training. Within this framework, we decompose forgetting into concept deletion, recoverability, and decodability, distinguishing between information that is removed, misaligned, or still present but not directly accessible. Our results show that much of the seemingly lost concept-level information remains linearly recoverable, indicating that forgetting often reflects reduced accessibility to the underlying computational patterns rather than true erasure. At the same time, some concepts become less linearly decodable over time, making loss of accessibility functionally equivalent to forgetting under standard linear readouts. Moreover, continual strategies differ in how well they preserve the fine-grained information. Our findings provide a unified view of forgetting that links representational drift, readout limitations, and concept-level information dynamics. Our contributions can be summarized as follows: • We introduce a concept-level framework for analyzing forgetting in continual learning, leveraging SAEs to define a task-anchored latent space whose features act as concept proxies. It enables studying task-specific knowledge preservation and linear recoverability through concept activation dynamics and task- and concept-level decodability. • We define a taxonomy of concept transitions after learning subsequent tasks in continual learning: retained, seemingly deleted, recovered, decodable, and lost concepts (see Fig. 1 ). • We show that a substantial portion of task-specific information remains linearly recoverable at the level of individual concepts, and that continual learning strategies differ in how well they preserve the fine-grained knowledge and its recoverability. 2 Related work Most work in continual learning quantifies forgetting through degradation in the model’s classifier performance 27 ; 26 ; 16 ; 17 . Most works focused on developing mitigation strategies, including replay-based methods, regularization and distillation, architectural, and hybrid variants 28 ; 24 ; 10 , with forgetting typically evaluated through accuracy-based metrics. Although this line of work has established effective methods for reducing catastrophic forgetting, it offers limited insight into how forgetting manifests within internal representations. More recent studies relate forgetting to representational drift and changes in feature geometry 6 ; 12 ; 22 ; 14 . The authors of 6 shows that classifier-level forgetting can overestimate degradation of the underlying representations. In contrast, we move beyond task-level probing on raw features by using an SAE-defined, task-anchored concept space to analyze whether more fine-grained task-relevant information remains preserved and linearly decodable at the concept level. The authors of 12 enforces linear separability across tasks through backward feature projection. In contrast, we use linear mappings not as a training constraint, but as a diagnostic tool to measure how much past information remains linearly recoverable under standard continual learning strategies, and where this recoverability breaks down. In terms of understanding forgetting, mechanistic interpretability offers a complementary perspective by decomposing neural representations into more atomic and interpretable components, e.g., by using Sparse Autoencoders (SAEs) and transcoders 15 ; 8 ; 2 . Such methods construct overcomplete, sparse latent spaces that expose recurring disentangled patterns, offering a stronger basis for feature-level analysis. Yet, mechanistic interpretability