LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation

Mohammad Khoshnazar, Andrew Melnik, Michael Beetz · 2026

Read LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation by Mohammad Khoshnazar, Andrew Melnik, Michael Beetz (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29864.txt.

Introduction

Mohammad Khoshnazar, Andrew Melnik, and Michael Beetz ask whether short future videos can guide multi-step robot manipulation. They formalize Future-Experience Conditioning, which steers closed-loop policies with a latent future clip. Future clips use three stages: a reasoner over a task ontology, a digital-twin rollout of intended motion, and a diffusion model. The reasoner works from the scene. The diffusion model renders a robot-consistent clip with no segmentation at runtime. Generated futures beat no-future conditioning while mismatched futures hurt, and cloning plus reinforcement-learning leads.

Catalog record

CREATORMohammad Khoshnazar, Andrew Melnik, Michael Beetz
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28396
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29864

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLM-Guided Future Hypotheses for Horizon-Aware Exploration in Multi-Step Robot Manipulation Mohammad Khoshnazar, Andrew Melnik, Michael Beetz Abstract Multi-step robot manipulation requires acting under uncertainty about how the scene will evolve, making exploration and policy adaptation challenging. We study whether short-horizon, task-consistent future videos can provide useful structured priors for control and reinforcement-learning fine-tuning. We formalize this idea through Future-Experience Conditioning (FEC) , a simple interface that conditions closed-loop policies on a latent representation of a short future video. In our simulation setup, future clips are generated in three stages, an LLM reasoner operating over a task ontology initialized from the current scene state, a robot-free digital-twin rollout of the intended object motion, and a mask-free video diffusion model that synthesizes a robot-consistent future clip without requiring segmentation at inference. We instantiate this future-conditioning interface primarily with BC and BC+RL , and compare against a future-conditioned Streaming Flow Policy (SFP) [ 18 ] baseline on RoboCasa and CALVIN under NoFuture , GTFuture , GenFuture , and WrongFuture . Generated futures improve performance over no-future conditioning, while mismatched futures degrade it, and our BC+RL instantiation achieves the strongest overall results. An average BC+RL learning-curve analysis across 8 CALVIN tasks further shows that GTFuture improves fastest, GenFuture improves earlier and to a higher level than NoFuture , and WrongFuture remains at zero throughout training. These results suggest that short-horizon future videos can serve as useful structured priors for exploration and policy adaptation under imperfect future predictions. https://enact2026.github.io/ Fig. 1: Overview of our Future-Experience Conditioning (FEC) pipeline. From the task prompt and current scene state, an ontology-guided LLM infers the target object, interaction part, desired state transition, and rollout constraints. This drives a robot-free digital-twin rollout (“transparent” video). A mask-free video diffusion model then inpaints the robot to generate a task-consistent future clip. The policy conditions on future latents together with recent observations. I Introduction Many manipulation tasks require more than reactive control. To open a drawer, move a light switch, or push an object into a drawer, a robot must account for how present actions affect future interaction states over multiple steps. Success depends on delayed consequences such as contact timing, articulation response, and task progress. These challenges are broadly reflected in recent surveys of deep reinforcement learning and contact-rich robotic manipulation [ 14 , 9 ] . Recent generative video models make it possible to use a short visual hypothesis of how the scene and manipulated object should evolve. If informative, such a future can improve contact timing, disambiguate intent, and help recovery from drift. The challenge is that the future signal used during training differs from the generated frames available at test time and may be noisy or misaligned temporally. More broadly, this question sits at the intersection of visual foresight, latent world models, and large-scale robot policy learning [ 10 , 13 , 11 , 12 , 5 , 4 , 27 ] . In this work, we study short-horizon future hypotheses as a mechanism for horizon-aware exploration and policy adaptation in multi-step manipulation. We formalize this idea with Future-Experience Conditioning (FEC) , which conditions a closed-loop policy on a latent representation of a short-horizon future clip. Our framework takes a task command and current scene state, initializes a task ontology, and uses an LLM reasoner operating over this ontology to infer what object should be manipulated, which interaction part is relevant, and how the object state should change. It then produces a robot-free digital-twin rollout for the intended object motion and uses a mask-free video diffusion model to inpaint the robot into this rollout as a future video. This video is encoded into a future latent and provided, together with recent observations, to the controller for action generation (Fig. 1 ). Generated future clips provide a form of horizon-aware exploration , allowing the controller to condition on plausible near-future task evolution rather than acting blindly. By comparing correct and mismatched future hypotheses, we study when such structured priors help or hinder closed-loop manipulation. This paper focuses on the role of future-conditioned visual priors in simulation . Our goal is not to claim real-world transfer, end-to-end digital-twin construction from raw sensory input alone, or continuous online twin updates during closed-loop execution. Instead, we study whether generated short-horizon future videos can improve policy execution and reinforcement-learning-based adaptation under controlled train and test future mismatch. Our main controller is BC+RL , while a future-conditioned SFP model is trained as a comparison baseline under the same future-conditioning interface. I-A Contributions • Future hypotheses as structured priors: We study short-horizon, task-consistent future videos as conditioning signals for multi-step robot manipulation and reinforcement-learning fine-tuning. • Future-Experience Conditioning (FEC): We introduce a simple future-conditioning interface that compresses short-horizon future videos into temporally binned latent representations for closed-loop policy conditioning. • Empirical study with BC+RL as the main instantiation: On RoboCasa and CALVIN, we evaluate BC, BC+RL, and a future-conditioned SFP comparison baseline under NoFuture, GTFuture, GenFuture, and WrongFuture. Our BC+RL instantiation is strongest overall, and an average BC+RL learning-curve analysis across 8 CALVIN tasks shows that informative generated futures can improve learning speed during RL fine-tuning while mismatched futures hinder adaptation. II Related Work Exploration is central to multi-step decision making in reinforcement learning and robotics, especially in manipulation where success depends on discovering interaction-relevant contacts and delayed consequences. Recent surveys review deep RL for robotic manipulation and contact-rich interaction settings, as well as broader reinforcement-learning applications across robotics domains [ 14 , 9 , 28 ] . Our work is most closely related to approaches that guide behavior with structured predictions of future task evolution. Visual anticipation methods predict future observations for planning and control [ 10 ] , while latent world models learn compact dynamics for longer-horizon behavior [ 13 , 11 , 12 ] . Sequence-modeling and generative control perspectives such as Decision Transformer and Diffuser are also closely related to the broader idea of exploiting predicted future structure for action selection [ 7 , 17 ] . More recent robotics approaches use generated futures either as intermediate visual goals [ 22 ] or in closed-loop replanning pipelines [ 6 ] . Our setting differs in focusing on imperfect future supervision , where the future signal used during training differs from the generated future available at test time. For future generation, we build on diffusion-based video modeling and inpainting [ 15 , 16 , 21 ] , specifically CogVideoX [ 29 ] and VideoPainter [ 3 ] , to construct a mask-free VDM conditioned on a robot-free digital-twin rollout. Our work is also adjacent to recent vision-language and digital-twin style manipulation systems that use high-level reasoning or imagined goal states for rearrangement and control [ 20 , 1 ] . For robot control, our main controller is a behavior-cloning pipeline followed by reinforcement-learning fine-tuning (BC+RL), together with a BC-only variant. We additionally adapt Streaming Flow Policy (SFP) [ 18 ] to the same future-conditioning interface as a comparison baseline. More broadly, the paper is related to recent visuomotor and multimodal robot policies including Diffusion Policy, RT-1, RT-2, CLIPort, PerAct, VIMA, and Octo [ 8 , 5 , 4 , 25 , 26 , 19 , 27 ] . Our formulation is also related

Read the complete text