Act2Goal: From World Model To General Goal-conditioned Policy

Pengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen, Wenzhi Zhao, Rongjun Jin, Guanghui Ren, Jianlan Luo · 2025

Read Act2Goal: From World Model To General Goal-conditioned Policy by Pengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen, Wenzhi Zhao, Rongjun Jin, Guanghui Ren, Jianlan Luo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-23541.txt.

Introduction

Pengfei Zhou and seven co-authors present Act2Goal, a robot control system that takes its task from a picture of the goal state. A visual goal is compact and unambiguous, but earlier goal-conditioned policies planned only one step at a time and lost track of long tasks. Act2Goal instead pairs the policy with a visual world model that imagines how the scene should evolve.

Given the current camera view and the goal image, the world model generates an intermediate sequence of visual states. A scheme called Multi-Scale Temporal Hashing then splits that imagined path into dense nearby frames for fine control and sparse distant frames that anchor the overall plan. On real robots, success on hard out-of-distribution tasks rose from thirty percent to ninety percent after minutes of self-guided practice.

Catalog record

CREATORPengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen, Wenzhi Zhao, Rongjun Jin, Guanghui Ren, Jianlan Luo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-23541

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Act2Goal: From World Model To General Goal-conditioned Policy Pengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen, Wenzhi Zhao, Rongjun Jin, Guanghui Ren, Jianlan Luo Abstract Specifying robotic manipulation tasks in a manner that is both expressive and precise remains a central challenge. While visual goals provide a compact and unambiguous task specification, existing goal-conditioned policies often struggle with long-horizon manipulation due to their reliance on single-step action prediction without explicit modeling of task progress. We propose Act2Goal, a general goal-conditioned manipulation policy that integrates a goal-conditioned visual world model with multi-scale temporal control. Given a current observation and a target visual goal, the world model generates a plausible sequence of intermediate visual states that captures long-horizon structure. To translate this visual plan into robust execution, we introduce Multi-Scale Temporal Hashing (MSTH), which decomposes the imagined trajectory into dense proximal frames for fine-grained closed-loop control and sparse distal frames that anchor global task consistency. The policy couples these representations with motor control through end-to-end cross-attention, enabling coherent long-horizon behavior while remaining reactive to local disturbances. Act2Goal achieves strong zero-shot generalization to novel objects, spatial layouts, and environments. We further enable reward-free online adaptation through hindsight goal relabeling with LoRA-based finetuning, allowing rapid autonomous improvement without external supervision. Real-robot experiments demonstrate that Act2Goal improves success rates from 30% to 90% on challenging out-of-distribution tasks within minutes of autonomous interaction, validating that goal-conditioned world models with multi-scale temporal control provide structured guidance necessary for robust long-horizon manipulation. Project page: https://act2goal.github.io/ 1 1 footnotetext: Equal contribution. 2 2 footnotetext: Corresponding author. I Introduction Learning robotic manipulation policies requires task specifications that are both expressive and precise. While natural language offers a flexible interface for diverse tasks, it often lacks the necessary granularity for fine-grained manipulation, a limitation that is more pronounced in complex, multi-stage scenarios. Visual goals offer a more precise alternative by directly encoding object configurations, spatial relations, and terminal constraints, avoiding linguistic ambiguity and explicit reward engineering. Goal-conditioned policies (GCPs) map the current observation and a target visual goal directly to actions [ 1 , 2 , 3 , 4 ] . While these methods perform well in short-horizon settings and exhibit reasonable generalization [ 5 , 6 ] , their performance degrades in long-horizon tasks. This limitation arises because standard GCPs operate via direct action prediction [ 7 ] , lacking an explicit representation of task progress, intermediate feasibility, or long-horizon consistency. This issue is further exacerbated when GCPs are trained on narrowly scoped demonstration data. Without an explicit model of visual transitions toward the goal, such policies tend to overfit demonstrated state–action mappings and must rely on dense supervision to compensate for the lack of structured intermediate guidance. This limitation becomes particularly pronounced in long-horizon or out-of-distribution settings, where maintaining coherent progress toward the goal requires reasoning beyond locally observed transitions. Fig. 1: Method Overview. The model receives a visual goal (left), imagines how to achieve it via a goal-conditioned world model (top), and executes the planned actions in the real world (right). Fig. 2: System Overview. We propose Act2Goal, a goal-conditioned policy that integrates a visual world model with multi-scale temporal control to address long-horizon manipulation. After large-scale offline imitation learning, the model shows high performance on seen settings and strong generalization to unseen scenarios. The reward-free online autonomous improvement stage further improve model’s performance through rollout-goal relabel-optimize loop. To generalize beyond demonstrations, GCPs must therefore explicitly model the visual dynamics required to reach a desired goal. Without such mechanism, the policy cannot distinguish between actions that make meaningful progress toward the goal and those that merely match local state–action correlations observed in demonstrations. Recent advances in world models provide a promising pathway to address these limitations [ 8 , 9 , 10 ] . Vision–language world models demonstrate that generative prediction of future visual states—conditioned on task instructions—can effectively support planning and decision-making [ 11 , 12 , 13 ] . Building on this insight, we consider a goal-conditioned analogue: goal-conditioned world model. Instead of predicting the future from high-level language, a goal-conditioned world model produces a plausible sequence of intermediate states bridging the gap between the current observation and a desired visual goal. This capability remedies a fundamental limitation of conventional GCPs by introducing an explicit representation of scene evolution over time, enabling a visually grounded and task-consistent trajectory toward the specified goal. However, predicting a plausible visual path is only part of the solution. Executing long-horizon manipulation tasks reliably requires addressing a deeper control challenge. A general GCP must balance global consistency—maintaining fidelity to the long-term goal—with local reactivity—responding robustly to perturbations and correcting errors in closed-loop execution. Full-trajectory planning provides global coherence but is brittle under deviations; short-horizon control improves robustness but easily loses directional alignment in extended tasks. This inherent tension poses a fundamental barrier to deploying GCPs as general-purpose robotic controllers. To bridge this gap, we introduce Act2Goal, a general goal-conditioned policy that integrates a goal-conditioned world model and a novel temporal decomposition mechanism termed Multi-Scale Temporal Hashing (MSTH). MSTH decomposes the generated visual trajectory into dense proximal frames for fine-grained control and sparse, horizon-adaptive distal frames that anchor the global plan. This multi-scale structure enables the policy to reason over long-horizon objectives while reacting quickly to local disturbances during closed-loop execution. We further couple the goal-conditioned world model with an action expert via layer-wise cross-attention, allowing intermediate visual representations to guide low-level motor control in an end-to-end differentiable architecture. Initialized from large-scale human demonstration data, Act2Goal demonstrates strong zero-shot generalization across unseen objects, rearrangements, and environments. Beyond offline training, Act2Goal supports reward-free online autonomous improvement via Hindsight Experience Replay (HER) [ 14 ] . By relabeling its own rollouts as additional goal-achieving trajectories and updating the policy efficiently through LoRA-based finetuning [ 15 ] , the system rapidly adapts to new real-world scenarios without external supervision. In real-robot experiments, this enables Act2Goal to substantially improve performance on challenging long-horizon, out-of-distribution tasks, increasing success rates from 0.30 to 0.90 within minutes of autonomous interaction. These results validate the central premise of this work: that goal-conditioned world models, combined with multi-scale temporal reasoning, provide the structured intermediate guidance necessary for robust generalization and closed-loop execution in long-horizon manipulation. Our main contributions are threefold. First, we present a novel end-to-end goal-conditioned policy that integrates a visual world model with motor control, enabling robust zero-shot generalization across unseen objects, environments, and goals. Second, we introduce Multi-Scale Temporal Hashing (MSTH), a temporal representation that decomposes trajectories into proximal and distal frames to balance long-horizon planning with closed-loop local control. Third, we develop a reward-free online adaptation mechanism based on HER-style goal relabeling with LoRA-based finetuning, enabling rapid autonomous improvement on out-of-distribution tasks. II Related Works Our Act2Goal policy combines a goal-conditioned world model with an action expert capable of online autonomous improvement. Accordingly, we review related work in goal-conditioned policies, world models for robotic control, and online autonomous improvement. II-A Goal-conditioned Policy Goal-conditioned policy learning aims to train agents to reach diverse formats of goals, e.g., visual goals [ 2 , 3 , 4 , 16 , 17 , 18 , 19 ] , tracking points [ 20 , 21 ] and motion field [ 22 ]

Read the complete text