Multimodal Large Language Models for Real-Time Situated Reasoning

Giulio Antonio Abbo, Senne Lenaerts, Tony Belpaeme · 2026

Read Multimodal Large Language Models for Real-Time Situated Reasoning by Giulio Antonio Abbo, Senne Lenaerts, Tony Belpaeme (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01880.txt.

Description

Giulio Antonio Abbo and colleagues test whether multimodal models can guide a vacuum robot, combining GPT-4o with a TurtleBot 4 in a home. The model watches the home through vision and judges whether starting cleaning fits now, reasoning about activities, social norms and user preferences. Decisions align with residents' values such as cleanliness, comfort and safety, and a home demo shows context and values inferred from limited visual input. The work shows promise for robotic autonomy and awareness, while flagging consistency, bias and real time performance as challenges.

Catalog record

CREATORGiulio Antonio Abbo, Senne Lenaerts, Tony Belpaeme
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE13201
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.01880

Browse the full library · About the Tower of Babel