RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation

Zixuan Chen, Jing Huo, Yangtao Chen, Yang Gao · 2025

Read RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation by Zixuan Chen, Jing Huo, Yangtao Chen, Yang Gao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06605.txt.

Introduction

Zixuan Chen, Jing Huo, Yangtao Chen, and Yang Gao build a world model for long robot tasks in a 2025 paper. Long manipulation jobs are hard for learning systems because rewards are sparse and scenes are complex. Their pipeline has the robot recognize, sense, plan, then act. A large language model writes dense rewards for each sub-stage from the task instruction, so the robot can tell where it is in a long job.

Key moments are spotted and fed into a multi-view vision encoder that sharpens stage awareness. A learned world model then plans ahead. On two standard benchmarks the system beat prior methods, with success rates up by roughly a quarter to a third.

Catalog record

CREATORZixuan Chen, Jing Huo, Yangtao Chen, Yang Gao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-06605

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation Zixuan Chen, Jing Huo, Yangtao Chen, Yang Gao Abstract Efficient control in long-horizon robotic manipulation is challenging due to complex representation and policy learning requirements. Model-based visual reinforcement learning (RL) has shown great potential in addressing these challenges but still faces notable limitations, particularly in handling sparse rewards and complex visual features in long-horizon environments. To address these limitations, we propose the Recognize-Sense-Plan-Act (RSPA) pipeline for long-horizon tasks and further introduce RoboHorizon , an LLM-assisted multi-view world model tailored for long-horizon robotic manipulation. In RoboHorizon, pre-trained LLMs generate dense reward structures for multi-stage sub-tasks based on task language instructions, enabling robots to better recognize long-horizon tasks. Keyframe discovery is then integrated into the multi-view masked autoencoder (MAE) architecture to enhance the robot’s ability to sense critical task sequences, strengthening its multi-stage perception of long-horizon processes. Leveraging these dense rewards and multi-view representations, a robotic world model is constructed to efficiently plan long-horizon tasks, enabling the robot to reliably act through RL algorithms. Experiments on two representative benchmarks, RLBench and FurnitureBench, show that RoboHorizon outperforms state-of-the-art visual model-based RL methods, achieving a 23.35% improvement in task success rates on RLBench’s 4 short-horizon tasks and a 29.23% improvement on 6 long-horizon tasks from RLBench and 3 furniture assembly tasks from FurnitureBench. 1 Introduction A general-purpose robotic manipulator for real-life applications should be capable of performing long-horizon tasks composed of multiple sub-task phases, such as kitchen organization or warehouse picking. For instance, kitchen organization requires a robot to complete tasks like sorting food items, placing them into the refrigerator, and cleaning the countertops, while warehouse picking might involve identifying orders, picking items, and packing them. But how can we design such a comprehensive robotic system? Traditionally, long-horizon robotic tasks are tackled using the ”Sense-Plan-Act” (SPA) pipeline Marton 1984 ; Paul 1981 ; Murphy 2019 , which involves perceiving the environment, planning tasks based on a dynamic model, and executing actions through low-level controllers. A common approach to implementing this pipeline involves using visual and language encoders to extract task-relevant features for representation learning, followed by training control policies with model-based visual reinforcement learning (RL) Dalal et al. 2021 ; Yamada et al. 2021 ; Dalal et al. 2024 . While the above solutions are somewhat effective, they still face significant challenges in complex long-horizon tasks: (1) Language and visual encoders struggle to capture the hierarchical structure and dependencies of multi-stage sub-tasks in long-horizon tasks; and (2) Environmental feedback in such tasks is often sparse, while RL policies heavily relies on the rational reward structure. The former limits the robot’s ability to fully understand task dynamics and environmental context, while the latter further hinders the development of stable and effective long-horizon manipulation policies. Figure 1 : The proposed RSPA pipeline for long-horizon robotic manipulation. Figure 2 : RoboHorizon overview, using the long-horizon robotic manipulation task “take shoes out of box” in RLBench as the illustration example, following the proposed RSPA pipeline . Our key insight is that achieving stable execution of long-horizon tasks in model-based visual RL relies on enabling robots to accurately understand tasks , perceive multi-stage interactions between the robot and objects in the environment , and learn stable control policies through a structured reward system . How can robots be equipped with these capabilities? We propose leveraging pre-trained large language models (LLMs) and visual demonstrations captured by multi-view cameras to empower robots, primarily because: 1) LLMs have made significant advancements in robotics, demonstrating capabilities such as step-by-step planning Liang et al. 2023 ; Zeng et al. 2022 ; Ahn et al. 2022 ; Snell et al. 2022 , goal-oriented dialogue Zeng et al. 2022 ; Ahn et al. 2022 ; Huang et al. 2022 , sub-goals Huang et al. 2023 ; Chen et al. 2024a and reward generation Chiang et al. 2019 ; Yu et al. 2023 for robotic tasks based on language instructions. and 2) Observations from multi-camera views can significantly enhance a robot’s visual manipulation capabilities, and this setup is becoming increasingly common in real-world applications. Execution trajectories captured from different viewpoints often share similar environmental dynamics and physical structures. Previous studies have explored learning control policies from multi-view offline data using model-based RL Seo et al. 2023b or imitation learning (IL) Goyal et al. 2023 ; Shridhar et al. 2023 ; Ke et al. 2024 . Building on the above insight, we extend the traditional SPA pipeline into the Recognize-Sense-Plan-Act (RSPA) pipeline, as illustrated in Fig. 1 , specifically designed to address the challenges in long-horizon robotic manipulation. At the core of this pipeline is RoboHorizon , an LLM-assisted multi-view world model that enables robots to execute tasks effectively in complex, long-horizon robotic scenarios. To construct RoboHorizon, we first leverage pre-trained LLMs to generate a reasonable reward system for long-horizon tasks with sparse rewards. Unlike previous methods that use LLMs to generate reward signals directly applied to motion controllers Chiang et al. 2019 ; Yu et al. 2023 , which fail to address the complexities of multi-stage, long-horizon decision-making, our approach utilizes LLMs to divide long-horizon tasks into multi-stage sub-tasks. For each stage, dense stepwise rewards and intermediate rewards are generated, along with task reward code integrated into the environment interface, enabling robots to fundamentally recognize the long-horizon tasks they need to execute. Next, by collecting multi-view demonstrations of long-horizon manipulation tasks, we move beyond previous methods that directly perform representation learning on multi-view long-horizon demonstrations Seo et al. 2023b ; Goyal et al. 2023 ; Shridhar et al. 2023 . Instead, we integrate the multi-view masked autoencoder (MAE) architecture Seo et al. 2023b ; He et al. 2021 with the keyframe discovery method in multi-stage sub-tasks James and Davison 2022 , proposing a novel long-horizon key-horizon multi-view representation learning method. This method enables robots to more accurately sense interactions between the robotic gripper and objects during critical multi-task stages. Building on dense reward structures and the learned key-horizon multi-view representations, we construct RoboHorizon, enabling robots to effectively and efficiently plan action trajectories for long-horizon tasks. Finally, we leverage imagined trajectories generated by the world model to train RL policies, enabling effective low-level act capabilities for robots. We evaluate RoboHorizon on 4 short-horizon and 6 long-horizon tasks from the RLBench James et al. 2020 and 3 furniture assembly tasks from the FurnitureBench Heo et al. 2023 , both representative testbeds for robotic manipulation under sparse rewards and multi-camera settings. RoboHorizon outperforms state-of-the-art model-based visual RL methods, with a 25.35% improvement on short-horizon RLBench tasks and 29.23% on long-horizon RLBench tasks and FurnitureBench assembly tasks. Our contributions can be summarized as follows: (1) We introduce a novel Recognize-Sense-Plan-Act (RSPA) pipeline for long-horizon robotic manipulation, which tightly integrates LLMs for dense reward structures generation (Recognize), key-horizon multi-view representation learning for interaction perceiving (Sense), a world model for action trajectories planning (Plan), and RL policies for robot control (Act). (2) Based on the Recognize-Sense-Plan-Act (RSPA) pipeline, we propose RoboHorizon, a robot world model specifically designed for long-horizon manipulation tasks, built upon LLM-generated dense reward structures and key-horizon multi-view representations. It enables efficient long-horizon task planning and ultimately ensures the stable execution of RL policies. (3) We provide a comprehensive empirical study

Read the complete text