Exemplar-Free Continual Learning for State Space Models

Isaac Ning Lee Leila Mahmoodi Trung Le Mehrtash Harandi Monash University National University of Singapore · 2025

Read Exemplar-Free Continual Learning for State Space Models by Isaac Ning Lee Leila Mahmoodi Trung Le Mehrtash Harandi Monash University National University of Singapore (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18604.txt.

Description

Isaac Lee and co-authors present Inf-SSM, a method that lets state space models learn new tasks in sequence without forgetting old ones and without storing old examples, in a 2025 paper. State space models track long sequences well, but their inner state keeps evolving, so updates for a new task can wreck what earlier tasks built. Saving past data is the usual fix, and this method avoids it.

Instead of pinning the model's weights, Inf-SSM pins the geometry of how its state unfolds over an endless horizon. That guard is written as a matrix equation that normally costs heavy computation. The authors exploit the model's structure to cut the cost sharply. The regularizer can be added to existing continual learning methods. On hard image benchmarks, it cut forgetting a great deal while raising accuracy across the task sequence.

Catalog record

CREATORIsaac Ning Lee Leila Mahmoodi Trung Le Mehrtash Harandi Monash University National University of Singapore
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE116 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18604

Browse the full library · About the Tower of Babel