A Proposal for Networks Capable of Continual Learning

Zeki Doruk Erden & Boi Faltings · 2025

Read A Proposal for Networks Capable of Continual Learning by Zeki Doruk Erden & Boi Faltings (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22068.txt.

Introduction

Zeki Doruk Erden and Boi Faltings ask how a learning system can keep old skills while it learns new ones in a 2025 paper. Networks taught by the standard step-by-step method tend to lose past responses when their settings shift. The pair propose a fresh design that holds on to past answers by its very build.

Trials cover a small world model and a classic digit set. The new design kept learning fresh tasks with no replay of old samples and no set task borders. The cost is higher: the math runs heavier, and the design still has limits in what it can express at this stage.

Catalog record

CREATORZeki Doruk Erden & Boi Faltings
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22068

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Proposal for Networks Capable of Continual Learning Zeki Doruk Erden & Boi Faltings Abstract We analyze the ability of computational units to retain past responses after parameter updates, a key property for system-wide continual learning. Neural networks trained with gradient descent lack this capability, prompting us to propose Modelleyen , an alternative approach with inherent response preservation. We demonstrate through experiments on modeling the dynamics of a simple environment and on MNIST that, despite increased computational complexity and some representational limitations at its current stage, Modelleyen achieves continual learning without relying on sample replay or predefined task boundaries. 1 Introduction Modern machine learning relies on neural networks (NNs) to model complex systems, achieving remarkable success in areas like image recognition, language processing, and complex behavior. Yet as these problems are solved, crucial shortcomings at their core start to gain attention ( Clune (2019) ; Zador (2019) ; Marcus (2018) ; LeCun (2022) ). Importantly, they struggle with continual learning—the ability to learn new tasks without forgetting previous ones, as essential for open-ended learning, reflecting real-world dynamics. This limitation necessitates frequent, resource-intensive retraining, making scalability impractical. In this paper, we analyze a property essential for system-wide continual learning: the ability of computational units to retain past responses after parameter updates. We show that neurons in gradient-based NNs lack this capability and propose the early design of an alternative approach, Modelleyen , where units inherently preserve past responses. We also extend this approach to larger networks and, through experiments on modeling a simple environment and on MNIST, demonstrate that despite higher computational complexity and some performance limitations due to representational limitations at its current stage, we achieve system-wide continual learning without requiring replay or task boundaries. It also offers the advantage of human-comprehensible internal representations, as shown in additional results. 2 Related work The inability of continual learning is a core limitation of current ML systems ( Hadsell et al. (2020) ; Zhuang et al. (2020) ; Qu et al. (2021) ). Solutions proposed for for this problem often don’t fully resolve the fundamental limitations of NNs in this regard but aim to mitigate their effects. Many continual learning methods rely on simplifying assumptions, like externally defined task boundaries and task change information ( Rusu et al. (2016) ; Jacobson et al. (2022) ; Wang et al. (2022) , or storing and replaying past observations Buzzega et al. (2020) ; Galashov et al. (2023) ), which bias learning toward previous tasks without enabling true continual learning ( Kirkpatrick et al. (2017) ). One exception is methods generating multiple experts and assigning tasks based on a generative model ( Erden & Faltings (2024) ; Lee et al. (2020) ), though these still assume distinct tasks by decomposing the system into experts, relying on the availability of bulk data for each expert’s generative model – an assumption that does not hold in real-world learning. Thus, the continual learning problem merely shifts to the generative model for task identification rather than the entire system. Over the past decades, evolutionary biology has highlighted the crucial role of local variation and selection in shaping biological structures and driving their evolution Gerhart & Kirschner (2007) ; Marc (2005) ; West-Eberhard (2003) , including neural processes in the brain Marc (2005) ; Edelman (1993) . Our method centers on this mechanism, which is vital for achieving continual learning. 3 Local preservation of past responses System-wide preservation of past response is difficult to define and analyse, likely impossible without explicit retained knowledge of past samples. For that, we turn our attention to a local preservation of past response, which acts as a useful tool for system-wide continual learning, if not a prerequisite. We look at computational units whose response y can be phrased as y=f(\sum w_{i}x_{i}+b) where w_{i} are the weights, x_{i} are the inputs and b is the bias of the unit, and f(\cdot) is a nonlinear activation function. Neurons in NNs are examples of such units (as well as CSVs in our design, see next section). Assume that the unit, in the past, has been exposed to an input instance \textbf{X}_{t}=[x_{0t},...x_{Nt}] at some step t . With the standing weights \textbf{W}=[w_{0},w_{1},...w_{N}] , its response to this input given as y_{t}=f(\sum w_{i}\cdot x_{it}+b) . Suppose that after an arbitrary process of learning in response to other input instances, the parameters of the unit are updated to \textbf{W}^{\prime}=[w^{\prime}_{0},...w^{\prime}N] w^{\prime}_{i}=w_{i}+\Delta w_{i} and b^{\prime}=b_{i}+\Delta b at step t=T . New response to past input \textbf{X}_{t} is now y^{\prime}_{t}=f(\sum w^{\prime}_{i}x_{it}+b^{\prime}) . We want to ensure y^{\prime}_{t}=y , or f(\sum w^{\prime}_{i}x_{it}+b^{\prime})=f(\sum w_{i}x_{it}+b) . This is generally valid (and valid for monotonous activations used in NNs only) if \sum(w_{i}+\Delta w_{i})x_{it}+(b+\Delta b)=\sum w_{i}x_{it}+b , or if: \sum\Delta w_{i}x_{it}+\Delta b=0 (1) Eq. 1 is the condition for local preservation of information. A learning update rule defining \Delta w_{i} and \Delta b should satisfy this condition if response of the unit to previous inputs \textbf{X}_{t} , t\leq T is to be preserved. Some of the terms in the sum of Eq. 1 may be 0 due to \Delta w_{i} being 0, as defined by the learning rule, based on the current value of w_{i} . So we can rewrite 1 as: \sum_{i:\Delta w_{i}\neq 0}\Delta w_{i}x_{it}+\Delta b=0 (2) Note that this equation is in general dependent on x_{it} of t\leq T for i:\Delta w_{i}\neq 0 , which we assume to be not explicitly available. The only possibility for an update rule to satisfy the condition is to make it independent of x_{it} . This would only be possible if \forall i:\Delta w_{i}\neq 0 , and \forall t\leq T , x_{it} can be readily deduced from the available information, in which case w_{i} . This can be written as: \forall i:\Delta w_{i}\neq 0,\ \forall t\leq T,\ w_{i}=g(x_{it})\rightarrow x_{it}=g^{-1}(w_{i}) (3) So that 1 becomes independent of x_{it} : \sum_{i:\Delta w_{i}\neq 0}\Delta w_{i}g^{-1}(w_{i})+\Delta b=0 . It follows from 3 that: x_{i0}=x_{i1}=...=x_{iT},\ \forall t\leq T,\forall i:\Delta w_{i}\neq 0\ (4) These conditions do not apply to neural networks trained by gradient descent because there is (1) no direct correspondence between a learned weight and the input value it modulates (Eq. 3 ), and (2) no guarantee that previously encountered inputs will be identical for a given weight (Eq. 4 ). In contrast, our design meets these conditions, as outlined at the end of the following section. 4 Modelleyen: Learning basic environment dynamics Our proposal, termed Modelleyen , is designed to model sequential observations from an environment, but can be applied to any prediction task. It learns the environment’s structure with minimal exposure, enabling information reuse and continual learning while maintaining consistency with past experiences. At the core of our method is a local variation and selection process Gerhart & Kirschner (2007) as essential for continual learning and structured environment modelling. Below, we outline Modelleyen’s core mechanism, which relies on the immediate succession of activities in discrete state variables to model simple environmental dynamics. The next section extends this mechanism to observation spaces that can be represented as networks. Due to space constraints, we provide an overview of key definitions, the basic learning process, and core continual learning properties. For a full description, see Appendix A.2.1 and Algorithms 1 and 2 . Definition 1

Read the complete text