Task-Agnostic Experts Composition for Continual Learning

Luigi Quarantiello, Andrea Cossu, Vincenzo Lomonaco · 2025

Read Task-Agnostic Experts Composition for Continual Learning by Luigi Quarantiello, Andrea Cossu, Vincenzo Lomonaco (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15566.txt.

Introduction

Luigi Quarantiello, Andrea Cossu, and Vincenzo Lomonaco propose building continual-learning systems by composing expert models instead of training one network per task. Their 2025 paper draws on compositionality, the human ability to split hard problems into simpler parts. A system that reuses specialists could learn new tasks more cheaply and sustainably than one retrained from scratch.

Their Expert Composition method ensembles a set of expert models with no task-specific training, a zero-shot assembly. On a benchmark designed to stress compositional skill, the method reached much higher accuracy than baseline algorithms. It also used fewer computing resources to get there. The authors present the result as a step toward modular, efficient artificial intelligence.

Catalog record

CREATORLuigi Quarantiello, Andrea Cossu, Vincenzo Lomonaco
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15566

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Task-Agnostic Experts Composition for Continual Learning Luigi Quarantiello, Andrea Cossu, Vincenzo Lomonaco Abstract Compositionality is one of the fundamental abilities of the human reasoning process, that allows to decompose a complex problem into simpler elements. Such property is crucial also for neural networks, especially when aiming for a more efficient and sustainable AI framework. We propose a compositional approach by ensembling zero-shot a set of expert models, assessing our methodology using a challenging benchmark, designed to test compositionality capabilities. We show that our Expert Composition method is able to achieve a much higher accuracy than baseline algorithms while requiring less computational resources, hence being more efficient. [[email protected] ] *1 [[email protected] ] [[email protected] ] keywords Compositionality, Continual Learning † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: TCAI 25: Trustworthy and Collaborative Artificial Intelligence Workshop, June 09–11, 2025, Pisa, Italy † † address: Computer Science Department, University of Pisa, Largo Bruno Pontecorvo 3, 56127, Pisa - Italy † † corresp: Corresponding author. 1 Introduction itemize To go towards a more sustainable AI framework, compositionality is one of the main properties. Compositionality enables knowledge reuse, which reduces the training. Compositional models are more resilient and robust to distribution shifts, which are common in real world. In this paper, we explore in particular the composition of knowledge coming from different models Current AI research has given significant attention on developing large foundation models Bommasani et al. 2021 . Such architectures, typically composed by numerous non-linear layers and billions of parameters, exhibit remarkable results across diverse application domains. Nonetheless, these achievements come with a substantial computational demand, thereby impacting noticeably on the environment through increased carbon emissions Patterson et al. 2021 . For this reason, we believe that a change of direction should be promoted, by focusing on the development of more efficient and sustainable learning-based solutions. One possible approach in this sense comes by applying the principle of compositionality. The standard definition of compositionality Werning et al. 2012 reads as follows: {displayquote} The meaning of a compound expression is a function of the meanings of its parts and of the way they are syntactically combined. In other words, it refers to the ability of recognizing the whole as the sum of its parts. This concept has been studied in the context of AI for more than 30 years Braham and Hamblen 1990 ; Hussain 1995 , starting by taking inspiration from biological properties of the human brain, which appears to function in a highly compositional way. In computer vision applications, in particular, it can be seen as the capacity of classifying an image by detecting the objects present in the scene. One of the main advantages that comes from applying the compositionality property is the one of knowledge reuse. In fact, one could have a set of pretrained models, each able to recognize particular patterns or objects, and reuse them for different applications, selecting only the needed networks and composing their answers. Such composition could be zero-shot , i.e. directly applying the models without further training, or few-shot , allowing for a short training procedure with only a small set of samples. Following this approach leads to a much more sustainable AI framework, since it allows to effectively exploit the knowledge acquired by the models, reusing them multiple times, while reducing the need for long and expensive training processes. Compositional models, given that they focus on small sub-components, are also by design invariant to larger scale changes, such as variations in the background of an image or in the relations among objects. Additionally, by simply adjusting the models composition, such architectures are able to fit seamlessly to different applications and data distributions; these makes them much more versatile and robust than their “ monolithic ” counterparts. In this work, we mainly explore the composition of knowledge coming from different expert models. We will show that, by enforcing compositionality, we are able to largely surpass the performance of known approaches, using a compositional benchmark to asses our method. 2 Experts Composition itemize Our objective is to reuse and compose the knowledge of a set of pretrained experts. We employ the CGQA benchmark from cfst ; description of the CR_GQA and CGQA datasets. To show the benefits of enforcing compositionality to neural models, our proposed methodology consists mainly in two steps: first, the training of expert models, and then their application in a compositional scenario. Ideally, especially considering that the number of pretrained models available online is always increasing, it could be possible to simply download the needed networks, thus avoiding the training process at all. Unfortunately, in the case of our work, we could not find adequate ready-to-use models. Hence, we opted to train our own networks, nonetheless keeping them as small as possible to maintain an efficient approach. To train the experts, we employed cropped images from the GQA Hudson and Manning 2019 dataset. We selected 21 objects from the dataset, we used the information about their bounding boxes to crop the original images and we resized them to a resolution of 98x98 pixels (Figure 1 , left). Figure 1: Left: training of the expert models, each on a different subset of cropped images from GQA. Right: the experts composition is tested on CGQA. figure \captionof figureCropped images from GQA. \captionof figureImages from CGQA. We assigned 3 unique classes to each model, thus resulting in 7 experts. Each model is trained to specialize on its classes, while a generic “ other ” class label is used as a container of the remainder of classes. In this way, an expert can both provide an answer when prompted with samples from its classes, but also detect when an image is out of its scope. These trained models were then tested on the highly compositional CGQA benchmark Liao et al. 2024 , that contains samples made by a 2x2 grid, in which 2 cropped object images taken from GQA are inserted (Figure 1 , right). CGQA was originally designed for Continual Learning Parisi et al. 2019 , a Machine Learning paradigm in which models are trained on streams of different tasks in dynamic environments; nonetheless, the benchmark can be easily adapted for offline testing. In the paper, two main scenarios are presented, the task-incremental and class-incremental learning cases. We will compare our method against the second one, which is the most challenging setting, since no task identifier is provided and a single-head classifier must be employed. Our approach can be formalized as follows: for each query image, we extract its composing quadrants; then, the prediction for each quadrant is computed as: \forall\;\text{expert},\quad p=\argmax\;\text{expert}(q)\\ where q is a quadrant from an image. Lastly, the entire prediction for a compositional image is given by the concatenation of the predictions on its quadrants: p_{\text{comp}}=p_{1}\cup p_{2} In this fashion, no additional training procedure is needed to recognize the objects composition, since the experts are employed zero-shot . This results in a substantial saving in training effort with respect to related approaches, which instead need to be fine-tuned for each experience, incurring also in loss of accuracy due to the catastrophic

Read the complete text