Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning

Baining Zhao ∗ , Ziyou Wang ∗ , Jianjie Fang ∗ , Chen Gao † , Fanghang Man , Jinqiang Cui , Xin Wang , Xinlei Chen † , Yong Li , Wenwu Zhu Tsinghua University Project Page Code · 2025

Read Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning by Baining Zhao ∗ , Ziyou Wang ∗ , Jianjie Fang ∗ , Chen Gao † , Fanghang Man , Jinqiang Cui , Xin Wang , Xinlei Chen † , Yong Li , Wenwu Zhu Tsinghua University Project Page Code (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-12680.txt.

Introduction

Baining Zhao and co-authors introduce Embodied-R in 2025, a framework meant to give foundation models stronger spatial reasoning from video. People can watch a stream of views from their own eyes and judge how places and objects relate. Models often lack that skill, and the team splits the job between a large vision-language model that perceives and a small language model that reasons.

Training used reinforcement learning with rewards that check whether the thinking and the final answer agree in logic. After training on only five thousand video samples, their small reasoning model matched leading large models on spatial tasks, both familiar and new. The paper also probes answer length, reward design, and how fine-tuning and reinforcement learning differ.

Catalog record

CREATORBaining Zhao ∗ , Ziyou Wang ∗ , Jianjie Fang ∗ , Chen Gao † , Fanghang Man , Jinqiang Cui , Xin Wang , Xinlei Chen † , Yong Li , Wenwu Zhu Tsinghua University Project Page Code
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-12680

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning Baining Zhao ∗ , Ziyou Wang ∗ , Jianjie Fang ∗ , Chen Gao † , Fanghang Man , Jinqiang Cui , Xin Wang , Xinlei Chen † , Yong Li , Wenwu Zhu Tsinghua University Project Page Code Abstract . Humans can perceive and reason about spatial relationships from sequential visual observations, such as egocentric video streams. However, how pretrained models acquire such abilities, especially high-level reasoning, remains unclear. This paper introduces Embodied-R, a collaborative framework combining large-scale Vision-Language Models (VLMs) for perception and small-scale Language Models (LMs) for reasoning. Using Reinforcement Learning (RL) with a novel reward system considering think-answer logical consistency, the model achieves slow-thinking capabilities with limited computational resources. After training on only 5k embodied video samples, Embodied-R with a 3B LM matches state-of-the-art multimodal reasoning models (OpenAI-o1, Gemini-2.5-pro) on both in-distribution and out-of-distribution embodied spatial reasoning tasks. Embodied-R also exhibits emergent thinking patterns such as systematic analysis and contextual integration. We further explore research questions including response length, training on VLM, strategies for reward design, and differences in model generalization after SFT (Supervised Fine-Tuning) and RL training. Figure 1. Embodied spatial reasoning: tasks and thinking process. Challenging tasks from public embodied video datasets are identified, encompassing both indoor and outdoor scenarios. We introduce slow-thinking to improve reasoning performance. 1. Introduction On the path toward Artificial General Intelligence (AGI) ( Fei et al. 2022 ) , we hope that pre-trained foundation models can not only perform tasks such as dialogue and image understanding in the cyber world ( Achiam et al. 2023 ; Team et al. 2023 ) but also develop human-like embodied spatial cognition in the three-dimensional physical world, enabling them to perceive, think, and move ( Aubin et al. 2022 ; Liu et al. 2024a ) . The fundamental way humans achieve spatial cognition is through continuous, dynamic visual observations, akin to video streams ( Intriligator and Cavanagh 2001 ; Liu et al. 2023 ) . For example, by observing their surroundings, humans can infer their position relative to nearby objects. Similarly, based on historical visual observations, humans can determine the actions they should take to reach a target destination. Visual spatial cognition can be divided into two levels: perception and reasoning ( Wang et al. 2024c ) . Perception refers to “what is seen", characterized by direct, low-level tasks such as object recognition, edge detection, or color differentiation ( Wang et al. 2024b ) . Reasoning, on the other hand, involves “what is understood" and “what actions to take", which are indirect and higher-level tasks requiring logical inference and knowledge integration ( Zhao et al. 2025 ) . Examples of reasoning include “Where did I come from?" (e.g., recalling historical movement trajectories ( Mu et al. 2023 ) ), “Where am I?" (e.g., inferring the spatial relationships between nearby objects and distances ( Azuma et al. 2022 ) ), and “Where do I want to go?" (e.g., planning actions and deciding movements to reach a destination ( Chen et al. 2024a ) ). While most existing research focuses on improving the perception capabilities of foundation models ( Chen et al. 2024b ; Bai et al. 2025 ) , with notable progress, their spatial reasoning abilities remain limited ( Chen et al. 2024c ; Yang et al. 2024a ) , and methods for enhancement are largely unexplored. Specifically, video-based spatial reasoning poses several challenges, as follows: • Reasoning is always built upon perception ( Foglia and Wilson 2013 ; Liu et al. 2024a ) . For the studied problem, continuous visual observations impose higher demands on perception. Reasoning cannot be well achieved with faulty perceptions or hallucinations ( Wang et al. 2024a ) . It is challenging to reason when it is already hard to perceive from the videos. • Video data naturally involves complex spatio-temporal relationships, requiring the discovery of object associations across frames and the extraction of semantics relevant to the reasoning task ( Fei et al. 2024 ) . For instance, to navigate to a destination outside the current field of view, one must infer their location from historical visual observations, build a mental map of the environment, develop a high-level plan to determine the direction, and finally decide on specific actions to execute. Existing supervised fine-tuning (SFT) training methods lack supervision for the reasoning process, making it difficult to handle such reasoning tasks ( Zhao et al. 2025 ) . • Embodied visual observations have distinct characteristics. First, understanding disembodied videos, such as movies or TV shows, primarily emphasizes the content within the video, often from a broad and objective perspective ( Li et al. 2024b ) . In contrast, egocentric videos focus on understanding the relationship between the observer and the surrounding environment, often from a constrained first-person perspective ( Grauman et al. 2022 ) . Second, embodied continuous visual observations are generated over time, indicating that embodied perception should rely on sequential inputs rather than aggregating all visual observations for a single input after a prolonged period ( Liu et al. 2024b ) . Finally, due to the continuity of motion in the physical world, egocentric visual observations also exhibit spatial continuity, meaning there is significant redundancy and repetition between frames. Consequently, directly applying existing multimodal large language models (MLLMs) to embodied videos leads to issues, including loss of generalization and input token limits caused by excessive redundant frames ( Abdin et al. 2024 ; Lin et al. 2023 ) . Recently, the impressive performance of OpenAI’s o1/o3 ( OpenAI 2024b ) and DeepSeek-R1 ( Guo et al. 2025 ) in solving complex reasoning problems(e.g., mathematics, coding, science, etc.) has drawn attention to reinforcement learning (RL) techniques. By incorporating the chain-of-thought (CoT) reasoning process into post-training, large language models (LLMs) demonstrate a "slow-thinking" mode, where they reason thoroughly before generating responses ( Team et al. 2025 ; Xie et al. 2025 ) . Inspired by this, we attempt to introduce “slow thinking" into embodied video-based spatial reasoning tasks, as shown in Figure 1 . This brings a new challenge: the trade-off between model size and computational cost. Existing studies suggest a strong correlation between multimodal understanding/perception capabilities and model size ( Xu et al. 2023 ; Gao et al. 2024 ; Chandrasegaran et al. 2024 ) . Since reasoning builds on perception, larger vision-language foundation models should be used as the starting point for training. However, increasing model size leads to often unacceptable computational costs. Additionally, video inputs map to long token sequences, further raising computational demands. Is there a way to leverage the perception capabilities of large-scale models while developing embodied reasoning abilities at a lower computational cost? Inspired by neuroscience ( Zilles and Amunts 2010 ) , spatial perception and reasoning involve distinct brain regions: visual perception occurs in the visual areas of the occipital lobe ( Clarke and Miklossy 1990 ) , basic spatial understanding in the parietal lobe ( Fogassi et al. 2005 ) , and complex spatial reasoning in the prefrontal cortex ( Donoso et al. 2014 ) . This inspired the design of a collaborative framework with two main components: a large-scale vision-language model (VLM) for perception and

Read the complete text