A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents
Luigi Quarantiello 1 4 , Elia Piccoli 1 , Jack Bell 1 , Malio Li 1 , Giacomo Carfì 1 , Eric Nuertey Coleman 1, Gerlando Gramaglia 1 , Lanpei Li 1 3 , Mauro Madeddu 1 , Irene Testa 1 , Vincenzo Lomonaco 2 · 2025
Read A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents by Luigi Quarantiello 1 4 , Elia Piccoli 1 , Jack Bell 1 , Malio Li 1 , Giacomo Carfì 1 , Eric Nuertey Coleman 1, Gerlando Gramaglia 1 , Lanpei Li 1 3 , Mauro Madeddu 1 , Irene Testa 1 , Vincenzo Lomonaco 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18608.txt.
Introduction
Luigi Quarantiello and his ten co-authors argue for a new way to build foundation models for robots. Foundation models, trained on huge data, now perform impressively across language, vision, and robot control, because the rich representations they learn transfer between tasks. Their weakness is flexibility: when the world changes, adapting them usually means retraining the whole enormous model.
The authors propose joining two ideas. Continual learning lets a model absorb new experience over time without forgetting the old, while compositionality builds behavior from reusable parts that can be recombined. Together, they argue, these principles could yield robotic agents that are smarter and more efficient, adjusting to dynamic real-world settings by updating or rearranging parts instead of starting over. The paper is a position and roadmap piece, laying out why this combined direction deserves pursuit.
Catalog record
| CREATOR | Luigi Quarantiello 1 4 , Elia Piccoli 1 , Jack Bell 1 , Malio Li 1 , Giacomo Carfì 1 , Eric Nuertey Coleman 1, Gerlando Gramaglia 1 , Lanpei Li 1 3 , Mauro Madeddu 1 , Irene Testa 1 , Vincenzo Lomonaco 2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 9 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18608
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Compositional Paradigm for Foundation Models: Towards Smarter Robotic Agents Luigi Quarantiello 1 4 , Elia Piccoli 1 , Jack Bell 1 , Malio Li 1 , Giacomo Carfì 1 , Eric Nuertey Coleman 1, Gerlando Gramaglia 1 , Lanpei Li 1 3 , Mauro Madeddu 1 , Irene Testa 1 , Vincenzo Lomonaco 2 Abstract The birth of Foundation Models brought unprecedented results in a wide range of tasks, from language to vision, to robotic control. These models are able to process huge quantities of data, and can extract and develop rich representations, which can be employed across different domains and modalities. However, they still have issues in adapting to dynamic, real-world scenarios without retraining the entire model from scratch. In this work, we propose the application of Continual Learning and Compositionality principles to foster the development of more flexible, efficient and smart AI solutions. I Introduction Recent advancements in Artificial Intelligence have led to the emergence of Foundation Models (FMs). Such architectures [ 1 ] , leveraging billions of parameters, terabytes of training data and the most powerful (and energy-hungry) machines in the world, are able to encode broad knowledge and skills in their hidden spaces. Ultimately, these features are employed to solve the most diverse tasks, encompassing different modalities and objectives. The most impressive results, in particular from the user standpoint, comes from the large language models powering different chatbots, which excel at web search, code/text generation and brainstorming. They show an incredible fluency (at least for widely spoken languages) and a great capability in elaborating knowledge and information from multiple sources. Beyond language, FMs have demonstrated superior capabilities in other settings too, such as Computer Vision ( e.g. ViT for image classification), and multi-model learning ( e.g. CLIP for joint image-text understanding). This led to networks with richer representations, which are more flexible and applicable to multiple scenarios, ultimately showing better performance. However, recent releases show that noticeable parameters and complexity increases lead only to marginal performance improvements. In this work, we propose a alternative vision on AI that encompasses Continual Learning (CL) and Compositionality, to enhance FMs with an increased adaptability and flexibility to address more complex, dynamic scenarios. II Large Models for Robotics Given their strong performance across different tasks, Transformer-based models have been applied to robotics too. Early efforts, such as Robotics Transformer (RT), proved the feasibility of high-capacity, data-driven control policies. RT-1 [ 4 ] demonstrated scalable generalization from large, diverse real-robot datasets, while RT-2 [ 16 ] injected internet-scale vision-language priors directly into action policies, establishing the “ foundation model paradigm ” for robotic control. In order to further scale these agents, the researchers focused on different aspects. A first key contribution was the release of Open X-Embodiment dataset [ 11 ] , that unifies demonstrations across 22 robot types to enable cross-embodiment transfer. On top of that, a great effort has been put to in the development of benchmarks, e.g. Meta-World [ 15 ] , and realistic yet highly efficient, GPU accelerated simulators, like ManiSkill3 [ 14 ] . These tools enable efficient training and systematic evaluation of generalization and multi-task performance across different robotics tasks. More recently, given the increased computational power and huge breakthroughs of language models, a parallel—or perhaps orthogonal—line of research has focused on the models architecture. Some pioneering work [ 13 , 6 ] tried to leverage CLIP’s embeddings to align natural language instruction in the robotics domain. As pre-trained models evolved, research aimed at modifying architectures to have specific components that handle different data modalities. While scaling architectures to billions of parameters helped in achieving better agents performance, a huge effort was spent to adapt general pre-trained models to specific robotics tasks. This effort lead to the development of Vision Language Action (VLA) models, in particular OpenVLA [ 9 ] . By fine-tuning a 7B-Llama backbone, the agent obtains a 16.5% increase in absolute task success rate compared to RT-2-X. III The Need for Continual Learning and Compositionality In the previous sections, we examined Transformer-based models and their most recent applications, particularly in robotics. As it often happens in Computer Science, many of these improvements has been driven by scaling, increasing model size, training data, or both, at the cost of growing computational and energy demands. This alone has yielded impressive results: language models are able to handle complex requests and process longer inputs, offering a more human-like experience and demonstrating reduced biases and improved safety, while robotic agents have gained more complex, multi-skill capabilities. However, we are at a point of diminishing returns, where enlarging models, datasets or computational resources is not enough to obtain noticeable results [ 10 ] . Furthermore, despite being trained on massive datasets, FMs still struggle to generalize across tasks over time, especially when data distributions shift. Their performance relies on statistical correlations rather than deeper reasoning, so scaling data [ 2 ] or using Parameter-Efficient Fine-Tuning methods only yields limited improvements. Applying CL principles and methods to FMs can represent a viable solution to give such models an higher degree of flexibility, allowing for updates on the pre-training distribution and efficient adaptation to domain-specific tasks [ 3 ] . TABLE I: Image Classification task. Our composition of adapters greatly outperforms competitors, while being more efficient Method Accuracy (%) Training Time (s) SD-LoRA 47.56 \pm 3.77 318.22 InfLoRA 36.02 \pm 3.26 196.89 HAM ( Ours ) 55.17 \pm 1.04 170.61 For these reasons, we argue that advancing to the next generation of AI requires a paradigm shift towards systems that are more flexible, adaptable and computationally efficient. Compositionality— divide & conquer —is a key enabling factor of the next revolution. It allows multiple specialized models to coexist, share knowledge and collectively solve complex tasks more effectively than single models. Such model compositions are inherently dynamic; by changing the way in which the models are orchestrated, such systems could easily adapt to ever-changing scenarios, being more flexible than a monolithic solution. Lastly, such combinations may be performed with little to no training phases, exploiting FMs full potential while being more efficient than the static counterparts. IV Experimental Results TABLE II: Robotic manipulation task. Our method outperforms more complex baselines, with reduced training times Method Reward/Step Success Rate Training Time InstructRL 0.14 0.0 40 h OpenVLA 0.10 0.0 92 h WSA ( Ours ) 0.60 0.91 14 h We explored the idea of CL and Compositionality in different domains, performing some preliminary experiments. For image classification tasks, we studied the adaptation of the ViT model to a stream of tasks with the composition of multiple LoRA adapters [ 7 ] . For each experience, we train a separate module, which is then merged with previously learned LoRAs to obtain a single final model. We employ a hierarchical merging process: a first combination is performed within LoRAs grouped by similarity, while group adapters are later merged into a single module, so to have a unified model capable of solving every seen task [ 5 ] . We found that such method is effective in reducing the merging process interference. Table I shows the results obtained on CUB200 dataset with 50 tasks: our methods achieves an higher accuracy, while being faster in