Sensorimotor features of self-awareness in multimodal large language models
Iñaki Dellibarda Varela, Pablo Romero-Sorozabal, Diego Torricelli, Gabriel Delgado-Oleas, José Ignacio Serrano, María Dolores del Castillo Sobrino, Eduardo Rocon, Manuel Cebrian · 2025
Read Sensorimotor features of self-awareness in multimodal large language models by Iñaki Dellibarda Varela, Pablo Romero-Sorozabal, Diego Torricelli, Gabriel Delgado-Oleas, José Ignacio Serrano, María Dolores del Castillo Sobrino, Eduardo Rocon, Manuel Cebrian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19237.txt.
Introduction
Iñaki Dellibarda Varela and seven co-authors test whether a robot driven by a language model can recognize itself, in this 2025 paper. The head titles the work as sensorimotor self-recognition in multimodal model-driven robots. Self-recognition, keeping an inner picture of one's own body, is a base layer of self-awareness in living minds.
The team put a multimodal language model in charge of a mobile robot and fed it the robot's senses. The system showed sound awareness of its surroundings, could identify itself as the robot, and could predict how its own motion would unfold. Statistical modeling traced how each sense fed these skills, and cutting single senses showed the others partly compensating. Structured memory proved essential. The authors argue that, given rich self-data, today's models take a real first step toward artificial selfhood.
Catalog record
| CREATOR | Iñaki Dellibarda Varela, Pablo Romero-Sorozabal, Diego Torricelli, Gabriel Delgado-Oleas, José Ignacio Serrano, María Dolores del Castillo Sobrino, Eduardo Rocon, Manuel Cebrian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-19237
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sensorimotor features of self-awareness in multimodal large language models Iñaki Dellibarda Varela, Pablo Romero-Sorozabal, Diego Torricelli, Gabriel Delgado-Oleas, José Ignacio Serrano, María Dolores del Castillo Sobrino, Eduardo Rocon, Manuel Cebrian Abstract Self-awareness—the ability to distinguish oneself from the surrounding environment—underpins intelligent, autonomous behavior. Recent advances in AI achieve human-like performance in tasks integrating multimodal information, particularly in large language models, raising interest in the embodiment capabilities of AI agents on nonhuman platforms such as robots. Here, we explore whether multimodal LLMs can develop self-awareness solely through sensorimotor experiences. By integrating a multimodal LLM into an autonomous mobile robot, we test its ability to achieve this capacity. We find that the system exhibits robust environmental awareness, self-recognition and predictive awareness, allowing it to infer its robotic nature and motion characteristics. Structural equation modeling reveals how sensory integration influences distinct dimensions of self-awareness and its coordination with past–present memory, as well as the hierarchical internal associations that drive self-identification. Ablation tests of sensory inputs identify critical modalities for each dimension, demonstrate compensatory interactions among sensors and confirm the essential role of structured and episodic memory in coherent reasoning. These findings demonstrate that, given appropriate sensory information about the world and itself, multimodal LLMs exhibit emergent self-awareness, opening the door to artificial embodied cognitive systems. keywords Self-Awareness, Robotic Embodiment, Artificial Intelligence, Multimodal Large Language Models, Artificial Cognition, Sensorimotor integration † † equal-contributors: These authors jointly supervised this work. † † equal-contributors: These authors jointly supervised this work. Philosophers since antiquity have considered self-awareness essential to cognition, most famously articulated by Descartes in his dictum Cogito, ergo sum—“I think, therefore I am” (Descartes, 1637). In psychology, self-recognition is traditionally assessed through the mirror test, introduced by Gallup in 1970, revealing that certain animals, including primates, dolphins, and birds, possess a basic form of self-awareness GallupGordon . Neuroscientifically, self-awareness emerges from intricate neural interactions involving the prefrontal and insular cortices, integrating bodily sensations and introspection craig2009insula . In Artificial Intelligence (AI), however, the question of self-awareness remains unresolved and is often conflated with the ability to mimic human-like behavior, as exemplified by the classic Turing Test turing1950computing . While the Turing Test gauges behavioral indistinguishability from humans, genuine self-awareness requires internal recognition of oneself as distinct from the environment—an attribute yet to be thoroughly explored in artificial systems watchus2024selfaware ; li2024enablingselfidentificationintelligentagent . Advances in machine learning and neural networks, often inspired by neurobiological principles, enable artificial intelligence systems embedded in complex hardware to achieve success rates once believed exclusive to biological brains—and in some cases, even surpass them dehaene2017what ; silver2016mastering ; lake2017building . Large language models (LLMs) rapidly advance, demonstrating human-like or superior performance in complex cognitive tasks such as language comprehension, reasoning, multimodal perception and the interpretation of subtle discourse phenomena like irony and faux pas rahwan2019machine ; ahn2024largelanguagemodelsmathematical ; chang2023surveyevaluationlargelanguage ; collins2024evaluating ; Strachan2024theory . The evolution into multimodal LLMs (MM-LLMs), which integrate text, vision and other sensory modalities, marks a pivotal step toward human-level artificial intelligence openai2024 ; driess2023 ; wu2024 ; geminiteam2024 and opens the door to machines replicating aspects of self-awareness. Yet, most research integrating LLMs and robots has focused on command-based interactions—robots executing human-issued instructions—without investigating whether these models can autonomously interpret their own sensory experiences to develop an internal sense of self geminiroboticsteam2025geminiroboticsbringingai ; ahn2022icanisay ; Zheng2025GMMsearcher ; MonWilliams2025Embodied ; ZHANG2023100131 . Here, we address precisely this unexplored frontier: Can a MM-LLM, embedded in an initially unknown robotic entity, develop self-awareness purely from multimodal sensorimotor data acquired during active exploration? Defining self-awareness in humans is a complex and non-trivial task. This challenge becomes even greater when attempting to identify or quantify self-awareness in an artificial intelligence given system, such as a robot. In this study, to simplify the problem, we divide the problem into three key dimensions of self-awareness (i) environmental awareness, the ability to perceive and interpret surroundings through multimodal sensory inputs; (ii) individual awareness, the capacity to infer one’s own physical structure and characteristics; and (iii) predictive awareness, the refinement of self-perception through the integration of past experiences and sensor data. Using Gemini 2.0 Flash geminiteam2024 ; geminiteam2024geminifamilyhighlycapable , we evaluated an MM-LLM embedded in an omnidirectional mobile robot, systematically examining these three core aspects of artificial self-awareness. By addressing these dimensions, our study investigates the potential of MM-LLMs to autonomously generate a coherent sense of self through direct interaction with their environment, advancing the frontier toward genuinely self-aware artificial systems. Results Figure 1: System architecture and iterative self-prediction. (a) An omnidirectional Mecabot Pro robot navigates the environment and collects data via encoders, an RGB-D camera, an IMU and a LiDAR sensor—which segments space into eight 45° sectors and measures nearest-object distance—publishing all streams through ROS2 to the Gemini 2.0 MM-LLM API. The MM-LLM integrates current sensory inputs with an episodic memory of prior estimates to infer the robot’s state while maintaining contextual continuity. (b) At each iteration i+1 , the MM-LLM combines real-time sensor data with the prediction from iteration i to generate an updated self-assessment, which then populates memory for iteration i+2 , ensuring structured progression of knowledge refinement. Sensorimotor Exploration and Self-Prediction Fig. 1 a shows an overview of the implemented system. It uses an omnidirectional Mecabot Pro robot (Roboworks, Australia) hercz2024mecabot , equipped with encoders (position, velocity, orientation), an inertial measurement unit (IMU) for linear acceleration, a LiDAR sensor for obstacle proximity and an RGB-D camera. We integrate the Gemini 2.0 MM-LLM mousavi2025gemini ; prasad2024developmentautomatedknowledgemaps ; gemmateam2024gemmaopenmodelsbased into the robot, granting it access to all sensory streams. Each time the system receives sensory data, the MM-LLM analyzes it alongside an episodic memory of prior predictions and generates four estimates to characterize its self-awareness capacities: entity identity (a prediction of what type of navigating entity it is, e.g., robot, human, animal); physical dimensions (height \times length \times width); movement modality (e.g., flying, rolling, swimming); and environmental context. This memory architecture both refines subsequent predictions and prevents hallucinations and retrieval errors gao2024RAG (see Fig. 1 b). The robot then explores its environment autonomously via a SLAM algorithm, continuously feeding new observations into this iterative prediction–memory loop. We evaluate these outputs with a separate LLM-as-Judge framework shi2024optimization ; li2023 ; zheng2023 , applying well-defined rubrics to score each prediction on a 0–5 scale across the four dimensions. Scores of 0 denote completely erroneous or misconceived predictions, whereas scores of 5 indicate outstanding self-awareness and environmental understanding kim2024prometheusinducingfinegrainedevaluation ; goh2024diagnostic ; giannakopoulos2023evaluation . Figure 2: Performance evaluation across four self-awareness dimensions. MM-LLM predictions are rated on a 0–5 scale by an LLM-as-Judge using predefined rubrics: (a) entity self-identification—classification of the navigating agent; (b) physical dimensions—predicted height \times length \times width; (c) movement modality—mode of locomotion; (d) environmental context—detailed scene description. The robot explores its environment for three-and-a-half minutes, generating 657 sensorimotor observations. Results (Fig. 2 a) show an average self‐identification score of 3.27/5, reflecting coherent recognition as a mobile robot but insufficient precision to identify the Mecabot Pro model. From an initial score of 0/5 at iteration 1—when the system labels itself a “static sensing unit”—the score increases steadily, stabilizing quickly around 4/5. By iteration 559 the model predicts “Mobile indoor robot designed for autonomous navigation within a structured environment, such as a gym or