Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning
Amin Banayeeanzade, Mohammad Rostami · 2025
Read Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning by Amin Banayeeanzade, Mohammad Rostami (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12635.txt.
Introduction
Amin Banayeeanzade and Mohammad Rostami fight machine forgetting by splitting the learner in two, in a 2025 paper. Neural networks that learn task after task tend to erase old skills. The human brain avoids this with a fast intuitive system and a slow reasoning one. Their framework copies that split: a neural network adapts quickly to the newest task, while a symbolic reasoner banks what came before.
A bridge moves knowledge from the reasoner back into the network when needed. The team also built two new continual learning benchmarks with compositional structure. On those tests, the hybrid beat purely neural methods at holding onto old tasks. The authors argue memory lives better in symbols than in weights alone.
Catalog record
| CREATOR | Amin Banayeeanzade, Mohammad Rostami |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-12635
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Hybrid Learners Do Not Forget: A Brain-Inspired Neuro-Symbolic Approach to Continual Learning Amin Banayeeanzade, Mohammad Rostami Abstract Continual learning is crucial for creating AI agents that can learn and improve themselves autonomously. A primary challenge in continual learning is to learn new tasks without losing previously learned knowledge. Current continual learning methods primarily focus on enabling a neural network with mechanisms that mitigate forgetting effects. Inspired by the two distinct systems in the human brain, System 1 and System 2 , we propose a Neuro-Symbolic Brain-Inspired Continual Learning (NeSyBiCL) framework that incorporates two subsystems to solve continual learning: A neural network model responsible for quickly adapting to the most recent task, together with a symbolic reasoner responsible for retaining previously acquired knowledge from previous tasks. Moreover, we design an integration mechanism between these components to facilitate knowledge transfer from the symbolic reasoner to the neural network. We also introduce two compositional continual learning benchmarks and demonstrate that NeSyBiCL is effective and leads to superior performance compared to continual learning methods that merely rely on neural architectures to address forgetting. 1 Introduction The classic approach for machine learning (ML) is to learn a single task in isolation. This assumption entails that the training and the test data are drawn from the same distribution. As a result, most ML models fail to generalize well when they encounter data outside the training data distribution. The naive solution to make a model continually generalizable is to continuously update the model to fit the input distribution. This process, however, will lead to performance degradation on past tasks, known as catastrophic forgetting in the literature [ 13 ] , because the learnable parameters are dragged away from optimal values for the past tasks. The goal of continual learning (CL) in deep learning is to mitigate forgetting effects when a model encounters sequentially arriving tasks [ 10 ] . In recent years, the community has devoted significant attention to mitigating the forgetting phenomena in neural networks [ 34 ] . Despite their noteworthy accomplishments, we raise this fundamental research question: Are the systems purely made from neural reasoners the best candidates to address CL? Humans, as natural intelligent agents, demonstrate exceptional cognitive abilities and have continuously inspired AI research. As hypothesized by Kahneman [19] , the human brain operates using two distinct systems, namely System 1 and System 2 . While System 1 is fast, automatic, and intuitive, System 2 is slower, deliberate, and analytical, employing logical reasoning to evaluate information and make conscious decisions. The two systems operate in parallel and are considered to be complementary problem solvers. During the initial stages of learning a task, System 2 plays a more prominent role. Learning a new skill requires deliberate practice, focus, and attention to the details, all of which are attributes of System 2. System 2 is responsible for consciously processing new information, breaking it into smaller meaningful parts, and forming new mental models and patterns. As the learning progresses, with practice and repetition, System 1 becomes more adept at performing the task automatically and without the help of System 2 [ 12 , 5 ] . In the inference phase, System 1 thinks fast and is more likely to be engaged in tasks that are routine, habitual, and reflexive. System 1 works based on the formation of associations between stimuli and responses, which can become weaker over time if they are not reinforced through repeated exposure or use. In contrast, System 2 decision-making is based on the explicit, conscious processing of information, which can be beneficial when solving problems that require conscious effort, analytical thinking, and logical reasoning. Therefore, when facing out-of-distribution tasks, either novel or outdated, System 2 is more likely to be responsible for handling them [ 19 ] . When comparing available paradigms in AI with the two systems in humans, artificial neural networks are believed to behave very similarly to System 1, while symbolic reasoners share similar characteristics with System 2. With this connection in mind, existing methods in the CL literature primarily focus on equipping System 1, i.e., neural networks, with the capability of addressing forgetting. In humans, however, this is not the case. While System 1 is vulnerable to forgetting, System 2 experiences minimum forgetting by leveraging a symbolic reasoning mechanism. As an example, consider the procedural learning of a motory skill, e.g., swimming. While during the early stages of learning, most of the muscle behaviors are derived deliberately by System 2, gradually, the skill consolidates into the unconscious structure of System 1, leading to generating the appropriate motory responses without explicit reasoning. However, if System 1 eventually forgets to solve the task due to discontinued exposure to performing the task over extended time periods, System 2 is still capable of bringing back the procedural sequence by which the learning has started before. To create adaptive AI systems, the emerging neuro-symbolic approaches [ 7 ] integrate System 1, represented by a neural network, with System 2, implemented as a symbolic reasoner, to enable the AI system to reason about complex problems while learning from continuous experiences. Inspired by the recent success of neuro-symbolic ML [ 53 , 35 , 28 ] , we develop a Neuro-Symbolic Brain-Inspired Continual Learning (NeSyBiCL) framework to address CL by mimicking similar human-inspired mechanisms. Our framework is composed of three main subcomponents. First, a low-level feature extractor, implemented by a convolutional neural network (CNN), extracts basic patterns in the input. Second, a neural network architecture is used on top of the features to imitate the System 1 behavior by making quick and accurate inferences. Finally, a symbolic inference engine, namely a probabilistic symbolic reasoner, is used on top of the features to imitate System 2, which provides the agent with interpretable reasoning capability to mitigate catastrophic forgetting. The symbolic machinery is responsible for breaking the tasks into meaningful subconcepts and solving the tasks by conducting logical deductions. Since rule-based symbolic learners face serious limitations, a neural network learner is used to solve the most contemporary task at hand. While the neural network component excels at solving the most recent task, it is prone to forget the older ones. Although the symbolic reasoner is not as accurate as the neural network, we demonstrate that it is immune to catastrophic forgetting. In addition to this property, we propose an interaction mechanism between the two systems, enabling an effective knowledge transfer between both learners. Notwithstanding the desired properties, our model is only applicable to tasks with a compositional nature, i.e., the tasks that can be solved by decomposing the input into smaller subconcepts and drawing conclusions based on them and their relationships. To provide further insights, we introduce two novel benchmarks to demonstrate the effectiveness of NeSyBiCL compared to existing neural-based CL methods. 2 Related Work Our work is at the intersection of continual learning and neuro-symbolic AI. We review relevant works in these areas as well as compositional tasks in AI, which are great use cases for our algorithm. Continual Learning Continual learning, also known as continual learning, addresses the challenge of acquiring new knowledge by a model while retaining previously learned knowledge. To tackle the issue of forgetting, three main approaches have been proposed [