Imagine, Verify, Execute: Memory-Guided Agentic Exploration with Vision-Language Models
Seungjae Lee a∗ , Daniel Ekpo a∗ , Haowen Liu a Furong Huang a,b† , Abhinav Shrivastava a† , Jia-Bin Huang a† a University of Maryland, College Park, b Capital One {sjaelee, daniekpo, hwl, furongh, abhinav, jbhuang}@umd.edu * Equal contribution † Equal advising Project Page: https://ive-robot.github · 2025
Read Imagine, Verify, Execute: Memory-Guided Agentic Exploration with Vision-Language Models by Seungjae Lee a∗ , Daniel Ekpo a∗ , Haowen Liu a Furong Huang a,b† , Abhinav Shrivastava a† , Jia-Bin Huang a† a University of Maryland, College Park, b Capital One {sjaelee, daniekpo, hwl, furongh, abhinav, jbhuang}@umd.edu * Equal contribution † Equal advising Project Page: https://ive-robot.github (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07815.txt.
Introduction
Lee and co-authors give a robot a curiosity loop modeled on human exploring. Robots learning in open settings rarely get clear goals or rewards, and vision-language models can suggest actions that sound smart but cannot physically be done. Their system, IVE, turns camera views into scene graphs, imagines new arrangements, checks whether each is plausible, and only then acts.
In simulated and real tabletop tests, IVE visits a far wider spread of states than reinforcement learning baselines, by a factor of four to nearly eight in diversity. The experience it gathers also trains skills that match or beat those learned from human demonstrations. The authors argue that verifying imagination before acting is what makes the difference.
Catalog record
| CREATOR | Seungjae Lee a∗ , Daniel Ekpo a∗ , Haowen Liu a Furong Huang a,b† , Abhinav Shrivastava a† , Jia-Bin Huang a† a University of Maryland, College Park, b Capital One {sjaelee, daniekpo, hwl, furongh, abhinav, jbhuang}@umd.edu * Equal contribution † Equal advising Project Page: https://ive-robot.github |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-07815
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Imagine, Verify, Execute: Memory-Guided Agentic Exploration with Vision-Language Models Seungjae Lee a∗ , Daniel Ekpo a∗ , Haowen Liu a Furong Huang a,b† , Abhinav Shrivastava a† , Jia-Bin Huang a† a University of Maryland, College Park, b Capital One {sjaelee, daniekpo, hwl, furongh, abhinav, jbhuang}@umd.edu * Equal contribution † Equal advising Project Page: https://ive-robot.github.io/ Abstract Exploration is essential for general-purpose robotic learning, especially in open-ended environments where dense rewards, explicit goals, or task-specific supervision are scarce. Vision-language models (VLMs), with their semantic reasoning over objects, spatial relations, and potential outcomes, present a compelling foundation for generating high-level exploratory behaviors. However, their outputs are often ungrounded, making it difficult to determine whether imagined transitions are physically feasible or informative. To bridge the gap between imagination and execution, we present IVE ( I magine, V erify, E xecute), an agentic exploration framework inspired by human curiosity. Human exploration is often driven by the desire to discover novel scene configurations and to deepen understanding of the environment. Similarly, IVE leverages VLMs to abstract RGB-D observations into semantic scene graphs, imagine novel scenes, predict their physical plausibility, and generate executable skill sequences through action tools. We evaluate IVE in both simulated and real-world tabletop environments. The results show that IVE enables more diverse and meaningful exploration than RL baselines, as evidenced by a 4.1 to 7.8× increase in the entropy of visited states. Moreover, the collected experience supports downstream learning, producing policies that closely match or exceed the performance of those trained on human-collected demonstrations. Keywords: Exploration, Agentic System, Vision-Language Model Figure 1: Comparison of human, RL, and IVE exploration strategies. (a) Humans explore by seeking novel scene configurations and understanding the environment [ 1 , 2 ] , often enhanced by goal verbalization [ 3 ] . (b) RL agents explore using a range of techniques, including intrinsic reward or goal sampling, to maximize the coverage of visited states. (c) IVE (ours) leverages VLMs to structure exploration via scene description, exploration, verification, memory, and action tools, each aligned with key aspects of human exploration. 1 Introduction Figure 2: Autonomous scene exploration with IVE. IVE enables autonomous exploration of the scene with diverse objects, e.g., a tangram ( top ) or common objects ( bottom ). Exploration is a fundamental capability for general-purpose robotic learning, particularly in open-ended environments where dense rewards, explicit goals, or demonstrations are scarce. In such settings, agents must autonomously discover diverse and meaningful interactions to support downstream learning and generalization [ 4 , 5 , 6 , 7 ] . Reinforcement learning (RL) has been a dominant paradigm, often using intrinsic rewards to promote novelty [ 8 , 9 , 10 ] or goal sampling to broaden state-space coverage [ 11 , 12 ] . While effective in simulation or low-dimensional tasks, RL methods often struggle in real-world robotic settings where environments are high-dimensional, semantically rich, and subject to physical constraints and safety risks. In these settings, the undirected or stochastic behaviors often encouraged by RL not only inefficient but also potentially hazardous. Vision-language models (VLMs) offer a promising alternative. Their broad semantic knowledge and reasoning capabilities have enabled advances in robotic perception, reasoning, and high-level decision making [ 13 , 14 , 15 , 16 ] . Building on these strengths, VLMs also offer a promising foundation for guiding exploration by generating hypothetical transitions, or quantifying novelty [ 17 , 18 , 19 , 20 ] . However, their imagination often lacks grounding in physical dynamics: transitions may appear semantically plausible yet prove physically infeasible, redundant, or unsafe to execute [ 21 , 22 , 23 ] . Moreover, VLMs operate without structured memory of prior interactions, making it difficult to reason about which states have already been visited or which actions have been attempted. This absence of memory and grounding often leads to redundant, implausible, or low-diversity generations that hinder effective exploration and downstream learning. To address this challenge, we introduce IVE ( I magine, V erify, E xecute), a fully automated, VLM-guided system for agentic exploration, inspired by human exploration—by generating self-directed goals, reacting to new information, and refining their understanding through experience [ 1 , 2 , 3 , 24 ] . IVE enables agents to imagine novel future configurations, predict their feasibility based on recent interaction history, and execute selected behaviors via a library of skills. IVE integrates imagination, verification, memory, and action execution into a closed loop, enabling exploration that is both semantically rich and physically grounded. The experience generated by IVE is not only physically grounded and semantically rich, but also directly reusable for downstream policy learning and world model learning. In this work, we make the following key contributions: • Curiosity-Driven Exploration via Imagination and Verification. We introduce IVE, that combines memory-guided imagination with physical plausibility prediction to emulate human-like curiosity in embodied agents, achieving a 4.1 to 7.8× increase in state entropy over RL baselines. • Reward-Free Data Collection with VLMs. We develop a fully automated, vision-language model-guided agentic system for generating semantically meaningful interaction data—without requiring external rewards, demonstrations, or predefined goals, achieving 82% to 122% of the scene diversity exhibited by expert humans. • Validation Across Downstream Tasks. We provide extensive experiments in both simulated and real-world tabletop environments, demonstrating that IVE improves exploration diversity and enables stronger policy learning and world model training compared to RL-based baselines. 2 Related Work Exploration for Robotic Learning . Exploration is a fundamental challenge in enabling robots, and plays a critical role in building accurate models of environmental dynamics, discovering affordances, and identifying effective strategies for control. To tackle the challenge of exploration, many prior methods have often leveraged RL via intrinsic reward or goal sampling [ 25 ] . Intrinsic reward methods encourage exploration using prediction error [ 8 , 26 , 9 ] , entropy [ 10 , 27 ] , or state visitation [ 28 , 29 ] , but often lack semantic understanding about the task, leading the agent to focus on perceptual novelty rather than task-relevant behaviors. In contrast, goal sampling methods guide exploration by sampling goals using temporal distance [ 30 , 31 , 32 ] , uncertainty [ 33 , 12 ] , or coverage [ 11 , 34 , 35 , 36 ] . While often more directed than intrinsic rewards, these methods still lack mechanisms to identify meaningful goals, and struggle in high-dimensional observation spaces, where learning a reliable latent representation or estimating uncertainty becomes challenging. Vision-language models for Robotics . VLMs have emerged as powerful tools for bridging visual perception and language-driven understanding. VLMs have demonstrated strong generalization across diverse tasks, enabling applications in robotic task generation [ 37 ] , autonomous data collection [ 38 ] , evaluation [ 39 ] , and serving as high-level planners for low-level action tools [ 22 , 40 ] . While VLMs offer rich semantic understanding, leveraging them specifically for guiding exploration remains relatively underexplored. Recent efforts include using VLMs to improve goal-conditioned policies [ 38 ] or explore by ranking observations based on semantic interestingness [ 20 ] . Similarly, we use a VLM in our work to guide exploration by prompting the VLM to imagine and propose physically plausible actions