RIG: Synergizing R easoning and I magination in End-to-End G eneralist Policy
Zhonghan Zhao, Wenwei Zhang, Haian Huang, Kuikun Liu, Jianfei Gao, Gaoang Wang, Kai Chen · 2025
Read RIG: Synergizing R easoning and I magination in End-to-End G eneralist Policy by Zhonghan Zhao, Wenwei Zhang, Haian Huang, Kuikun Liu, Jianfei Gao, Gaoang Wang, Kai Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-24388.txt.
Introduction
Zhonghan Zhao and co-authors present RIG, a 2025 robot policy that joins reasoning with imagination. A generalist robot must act in messy open worlds. Thinking before acting helps, and so does picturing what an action will do. Past systems bolted one skill or the other onto separate models, which slowed learning and hurt transfer. RIG is, the authors say, the first single policy trained end to end with both skills.
Training data was built in stages, weaving reasoning text and imagined next images into recorded agent runs. The joint training ties reasoning, action, and world change together. It improved sample efficiency by more than seventeen times over past work. At run time, the policy first reasons, then imagines the outcome. That preview lets it review and correct itself before it moves. Tests showed gains in robustness and transfer.
Catalog record
| CREATOR | Zhonghan Zhao, Wenwei Zhang, Haian Huang, Kuikun Liu, Jianfei Gao, Gaoang Wang, Kai Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-24388
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RIG: Synergizing R easoning and I magination in End-to-End G eneralist Policy Zhonghan Zhao, Wenwei Zhang, Haian Huang, Kuikun Liu, Jianfei Gao, Gaoang Wang, Kai Chen Abstract Reasoning before action and imagining potential outcomes (i.e., world models) are essential for embodied agents operating in complex open-world environments. Yet, prior work either incorporates only one of these abilities in an end-to-end agent or integrates multiple specialized models into an agent system, limiting the learning efficiency and generalization of the policy. Thus, this paper makes the first attempt to synergize R easoning and I magination in an end-to-end G eneralist policy, termed RIG . To train RIG in an end-to-end manner, we construct a data pipeline that progressively integrates and enriches the content of imagination and reasoning in the trajectories collected from existing agents. The joint learning of reasoning and next image generation explicitly models the inherent correlation between reasoning, action, and dynamics of environments, and thus exhibits more than 17 × 17\times sample efficiency improvements and generalization in comparison with previous works. During inference, RIG first reasons about the next action, produces potential action, and then predicts the action outcomes, which offers the agent a chance to review and self-correct based on the imagination before taking real actions. Experimental results show that the synergy of reasoning and imagination not only improves the robustness, generalization, and interoperability of generalist policy but also enables test-time scaling to enhance overall performance. 1 Introduction To navigate the complexities of open-world environments, two quintessential human faculties are de facto to embodied agents: imagination of prospective outcomes and reasoning. Although reasoning endows agents with the ability to deconstruct task objectives into executable plans through logical inference, it inherently operates within the constraints of perceptual history. This limitation underscores the complementarity of world models that learn the environmental dynamics, which not only allows the agent to predict action consequences but also facilitates risk-aware decision-making by evaluating hypothetical trajectories. The synergistic integration of reasoning and imagination constitutes an indispensable foundation for more intelligent and robust embodied agents operating in dynamically evolving environments. However, these two abilities are typically implemented in separate models. Specifically, reasoning mainly exists in large vision language models (VLMs) [ 43 , 33 , 35 , 44 ] that parse visual input and produce textual insights and actions ( Fig. 1 (a)), which lack explicit future prediction mechanisms. In contrast, world models [ 21 , 16 ] specialize in predicting future frames from video data ( Fig. 1 (b)), which suffer from data inefficiency due to the implicit learning of concepts, physical laws, and environment dynamics. Recent attempts [ 48 , 42 , 45 ] combine reasoning and imagination by connecting VLMs and visual generative models (VGMs). Yet, the integrated system ( Fig. 1 (c)) prevents end-to-end optimization of the agent, leaving the mutual benefits between reasoning and world models underexplored. Figure 1 : Comparison between conventional agents and RIG. RIG produces reasoning, actions, and imagination within a single Transformer. To bridge these gaps, this paper makes the first attempt to synergize R easoning and I magination in an end-to-end G enralist policy, termed RIG ( Fig. 1 (d)), RIG learns textual reasoning, low-level action control, and image generation through the sequence-to-sequence modeling objective within an autoregressive Transformer, as we hypothesize that the explicit modeling of the logic and motivation behind actions and their consequences could make RIG capture open-world dynamics more comprehensively and improve the sample efficiency of training. We develop RIG by adopting a progressive data collection strategy because existing datasets typically lack trajectories that contain interleaved image observations, precise actions, and high-quality textual reasoning. Based on initial trajectories collected from humans [ 11 ] and existing agents [ 20 ] that contain only actions and image frames, we first use VLM to insert textual rationales before each action on the trajectory and train RIG- basic with the reasoning-enriched trajectories. During inference, RIG- basic generates actions purely from textual and visual inputs, without leveraging imagined future frames, as decisions are executed immediately based on current observations. To further leverage visual imagination in reasoning to further improve the robustness of the policy, we collect unsuccessful trajectories from RIG- basic and adopt GPT-4o to review and revise these trajectories. Then, the suboptimal trajectories are taken as dreamed trajectories and combined with their corresponding revisions to form dream-review style trajectories for training RIG (also noted as RIG- lookahead for clarity). In contrast to RIG- basic that conduct reasoning without imagination, RIG- lookahead learns to first generate a trajectory by taking the predicted images as the environment states, and then review the hypothetical trajectory in reasoning, and predict revised action that changes the environment. Such a design provides scalability at inference time, where the number of steps in the dream trajectory can be scaled so that the agent can more comprehensively understand the effectiveness of the action and make future-aware decisions. We extensively evaluate RIG in the diverse, open-world Minecraft environment. Experimental results show that RIG upgrades the state-of-the-art results on embodied tasks, image generation, and reasoning benchmarks by 3.29\times , 2.42\times , and 1.33\times , respectively. Such a superior performance is achieved by training RIG on only 111 hours of videos, which is 17 \times fewer than previous works that rely on 2000 hours of videos. Moreover, when scaling the training data, environmental interactions, and the lookahead steps during reasoning, the generalization ability and robustness of RIG consistently improve, which implies the potential of synergizing reasoning and imagination in embodied agents. Our main contributions are summarized as follows: • We introduce an end-to-end generalist policy that synergistically integrates explicit reasoning and visual imagination. • We propose a progressive data collection strategy coupled with straightforward language model-based training to efficiently implement our method. • Our method naturally supports test-time scaling, enabling dynamic lookahead reasoning that enhances action robustness and reduces trial-and-error during inference. 2 Related Work Embodied Agents in Minecraft. Minecraft presents a significantly open-ended and complex environment [ 18 , 13 , 11 , 36 , 6 ] for embodied agents. Early approaches leveraged explicit world models to predict future states [ 16 , 7 ] but lack textual reasoning capabilities. Inspired by large language models (LLMs) [ 5 , 32 ] , subsequent methods combined LLMs with low-level controllers to address long-horizon tasks. For example, Voyager [ 33 ] and STEVE [ 43 ] used LLMs for high-level planning integrated with code databases, while others like Jarvis-1 [ 35 ] paired LLMs with pre-trained low-level policy models such as VPT [ 3 ] . However, these methods typically lack a world model to explicitly anticipate future visual outcomes. More recently, MineDreamer [ 48 ] integrates a world model and a policy controller, yet treats vision generation and policy control as separate modules, limiting coherent multi-modal reasoning. In contrast, RIG first attempts to explore an end-to-end generalist policy that simultaneously learns textual reasoning, visual imagination, and low-level action predictions to achieve high generalization ability and sample efficiency. World Models for Embodied Agents. Learning robust world models is essential for embodied agents to effectively plan and act within simulated environments [ 27 , 19 ] . Early approaches primarily focused on action-conditioned video prediction or latent imagination for