SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models

Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun · 2026

Read SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models by Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25525.txt.

Introduction

Mingxu Zhang and colleagues tackle forgetting when a language model learns task after task. Dense updates smear concepts across shared dimensions, so guarding old knowledge tends to block new learning. Their SAE-FD method anchors updates in the sparse features of a pretrained sparse autoencoder, which separates concepts and allows tighter, less disruptive regularization. Across two benchmarks and three model designs, it beats older regularization methods, reaching about 53 percent average accuracy with only slight backward transfer.

Catalog record

CREATORMingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49319
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25525

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models Mingxu Zhang, Yuhan Li, Lujundong Li, Dazhong Shen, Hui Xiong, Ying Sun Abstract Continual learning enables large language models to adapt to evolving tasks without retraining from scratch, yet catastrophic forgetting remains a central obstacle. Among continual learning methods, regularization-based approaches are widely used to constrain model updates and reduce forgetting, operating in weight space, gradient space, or output space. However, these dense representation spaces suffer from feature superposition, where multiple concepts are encoded in overlapping dimensions, making it difficult to selectively protect previously learned knowledge without impeding new-task learning. To address this issue, we propose SAE-FD (Sparse Autoencoder Feature Distillation), which anchors model representations in the sparse feature space of a pre-trained Sparse Autoencoder, where dense activations are decomposed into a sparse overcomplete basis that reduces representational entanglement, enabling more targeted regularization with less interference to new-task learning. Experiments on two continual learning benchmarks across three model architectures show that SAE-FD consistently outperforms existing regularization-based methods, achieving up to 52.70% average accuracy with only − - 0.46% backward transfer. 1 Introduction Large language models (LLMs) have demonstrated strong capabilities across diverse domains, including molecular science 33 ; 32 and social science 4 . In practice, deploying LLMs for specific application scenarios often requires fine-tuning on specific data. When models must be sequentially adapted to multiple tasks over time, catastrophic forgetting 12 becomes a central challenge: training on a new task degrades performance on previously learned tasks due to the overwriting and interference of internal representations. To mitigate forgetting, existing continual learning (CL) methods regularize model updates in different representation spaces. Weight-space methods such as EWC 12 penalize changes to important parameters; gradient-space methods such as O-LoRA 26 and GORP 24 project updates to be orthogonal to previous task subspaces; output-space methods such as LwF 13 and SEEKR 9 distill model outputs or attention patterns. A common limitation is that all three families operate in dense representation spaces where features representing different tasks are superimposed 6 , making it difficult to selectively protect old-task knowledge without constraining new-task learning. Sparse Autoencoders (SAEs) 5 ; 1 , developed for mechanistic interpretability, decompose dense activations into sparse, overcomplete feature vectors ( D\gg d ) that disentangle superimposed representations. The high sparsity and the overcomplete basis provide a promising decoupled space for continual learning regularization, where the reduced entanglement allows more targeted protection of previously learned knowledge. However, this application is non-trivial, as it requires training model-specific SAEs, designing loss functions that maintain gradient flow through the sparse encoding, and adapting regularization strength across different tasks and training stages. To address these challenges, we introduce SAE-FD (Sparse Autoencoder Feature Distillation), a continual learning method that performs representation anchoring in SAE feature space. SAE-FD operates in three stages. First, we train a Gated SAE on diverse text activations from the base model to obtain a high-quality sparse feature decomposition. Second, after fine-tuning on each task using LoRA, we capture per-token SAE feature activations on a small set of anchor samples, creating a compact snapshot of the model’s learned representations. Third, during subsequent task training, we apply a sparse-aware distillation loss that combines cosine direction preservation with active-feature magnitude matching. The total training loss combines the task loss with the distillation loss weighted by a coefficient \lambda ; since both losses vary in magnitude across tasks and training steps, a fixed \lambda cannot maintain a consistent regularization effect. We therefore introduce an adaptive mechanism that adjusts \lambda to target a fixed contribution ratio of the distillation loss, providing strong protection at the start of each new task and relaxing as the model converges. By operating in the sparse feature space, SAE-FD reduces the interference between regularization and new-task learning, achieving a better plasticity-stability tradeoff than dense-space alternatives. In conclusion, our contributions are: • We propose SAE-FD, the first continual learning method that performs regularization in the sparse, decoupled feature space of SAEs, enabling more targeted preservation of previously learned knowledge than baselines. • We design an adaptive \lambda mechanism that dynamically adjusts regularization strength during training by targeting a fixed contribution ratio of the distillation loss to the total loss, providing stronger protection when forgetting risk is high and relaxing as the model stabilizes. • Experiments on two CL benchmarks across three model families show that SAE-FD achieves 52.70% average accuracy with only - 0.46% backward transfer on TRACE, outperforming the strongest published baseline by +2.3% AA with 34% less forgetting. 2 Related Work Continual Learning for LLMs. Parameter-efficient fine-tuning via LoRA 10 has become the dominant LLM adaptation strategy, enabling a family of LoRA-based CL methods. O-LoRA 26 maintains orthogonal subspaces across tasks; GORP 24 projects gradients into low-rank subspaces optimized for minimal interference; TreeLoRA 17 organizes task-specific LoRA modules in a hierarchical structure with knowledge distillation; N-LoRA 30 addresses parameter collision between tasks; InfLoRA 14 constructs interference-free low-rank subspaces. Prompt-based methods including L2P 29 , DualPrompt 28 , and Progressive Prompts 21 prepend learnable tokens to model inputs, but their effectiveness is limited on generation tasks. Replay-based methods maintain a memory buffer of previous-task exemplars; SEEKR 9 combines selective attention distillation with replay, achieving strong results but requiring stored training data and full fine-tuning. Representation Distillation for CL. A complementary line of work preserves previous-task knowledge through distillation at various levels of model internals. At the output level, LwF 13 distills logits from a frozen teacher, and DER++ 2 stores logits alongside replay exemplars. At the attention level, SEEKR 9 identifies and distills the most task-relevant attention patterns, and DATA 15 decomposes attention for task adaptation. At the hidden state level, GeRe 34 enforces activation consistency on anchor samples, demonstrating that feature-level distillation outperforms logit-level distillation, though their ablation shows that raw L1/L2 matching of dense activations is suboptimal. SAE-FD distills in the sparse, decoupled feature space of an SAE, reducing representational entanglement before applying regularization. Sparse Autoencoders. SAEs learn to decompose neural network activations into sparse, overcomplete features 5 ; 1 , and recent variants such as Gated SAEs 20 achieve high-quality reconstruction with sparser features, scaling reliably to production-scale models 22 ; 8 . The superposition hypothesis 6 , which posits that networks represent more features than they have dimensions by exploiting sparsity, provides theoretical grounding for why SAE-based decomposition can disentangle overlapping representations. Beyond interpretability, SAEs have been applied to controllable generation, where 31 show that SAE features can serve as decoupled intervention handles for steering molecular properties in LLMs, demonstrating the practical utility of SAE-based disentanglement. To our knowledge, no prior work has explored SAE feature space as a regularization target for continual learning. 3 Preliminaries Continual Learning. We consider a sequence of T tasks \{\mathcal{D}_{1},\ldots,\mathcal{D}_{T}\} , where each task \mathcal{D}_{t}=\{(x_{i}^{t},y_{i}^{t})\} consists of input-output pairs. The model is trained sequentially on each task. A central challenge in this setting is catastrophic forgetting: as the model adapts to later tasks, performance on earlier tasks may degrade due to interference between task-specific representations. To quantify this, we use two standard metrics. Let a_{i,t} denote the performance on task t after training through task i : \displaystyle\text{AA}_{T} \displaystyle=\frac{1}{T}\sum_{t=1}^{T}a_{T,t} (1) \displaystyle\text{BWT}_{T} \displaystyle=\frac{1}{T-1}\sum_{t=1}^{T-1}(a_{T,t}-a_{t,t}) (2) Average Accuracy (AA) measures the overall performance across

Read the complete text