InTAct: Interval-based Task Activation Consolidation for Continual Learning

Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek · 2025

Read InTAct: Interval-based Task Activation Consolidation for Continual Learning by Patryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17439.txt.

Description

Patryk Krukowski and five co-authors present InTAct, a continual learning method with a mathematical promise: a network can learn new tasks without its behavior on old ones drifting. Most continual learning methods fight catastrophic forgetting heuristically; the few with guarantees constrain the weights themselves, which is expensive in high dimensions. InTAct instead works at the level of neuron activations, which are far fewer than weights.

The method identifies the activation intervals where earlier tasks live and constrains updates inside those intervals, while letting the network adapt freely elsewhere. Because predictions stay stable within the nested intervals, functional invariance is guaranteed in a tractable way. The approach is architecture-agnostic, and combined with prompt-based methods it reached state-of-the-art results on challenging continual learning benchmarks. The authors argue activation space is the right place to buy guarantees cheaply.

Catalog record

CREATORPatryk Krukowski, Jan Miksa, Piotr Helm, Jacek Tabor, Paweł Wawrzyński, Przemysław Spurek
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE95 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-17439

Browse the full library · About the Tower of Babel