InTAct: Interval-based Task Activation Consolidation for Continual Learning
Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek · 2025
Read InTAct: Interval-based Task Activation Consolidation for Continual Learning by Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17439.txt.
Description
Patryk Krukowski and five co-authors present InTAct, a continual learning method with a mathematical promise: a network can learn new tasks without its behavior on old ones drifting. Most continual learning methods fight catastrophic forgetting heuristically; the few with guarantees constrain the weights themselves, which is expensive in high dimensions. InTAct instead works at the level of neuron activations, which are far fewer than weights.
The method identifies the activation intervals where earlier tasks live and constrains updates inside those intervals, while letting the network adapt freely elsewhere. Because predictions stay stable within the nested intervals, functional invariance is guaranteed in a tractable way. The approach is architecture-agnostic, and combined with prompt-based methods it reached state-of-the-art results on challenging continual learning benchmarks. The authors argue activation space is the right place to buy guarantees cheaply.
Catalog record
| CREATOR | Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 95 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-17439