Target-Bench : Can World Models Achieve Mapless Path Planning with Semantic Targets?

Dingrui Wang, Hongyuan Ye, Zhihao Liang, Zhexiao Sun, Zhaowei Lu, Yuchen Zhang, Yuyu Zhao, Yuan Gao, Marvin Seegert, Finn Schäfer, Haotong Qin, Wei Li, Luigi Palmieri, Felix Jahncke, Mattia Piccinini, Johannes Betz, TUM Bosch AI Center ETH NJU https://target-bench.github.io/ · 2025

Read Target-Bench : Can World Models Achieve Mapless Path Planning with Semantic Targets? by Dingrui Wang, Hongyuan Ye, Zhihao Liang, Zhexiao Sun, Zhaowei Lu, Yuchen Zhang, Yuyu Zhao, Yuan Gao, Marvin Seegert, Finn Schäfer, Haotong Qin, Wei Li, Luigi Palmieri, Felix Jahncke, Mattia Piccinini, Johannes Betz, TUM Bosch AI Center ETH NJU https://target-bench.github.io/ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-17792.txt.

Introduction

Dingrui Wang, Zhihao Liang, Hongyuan Ye, and thirteen co-authors introduce Target-Bench, a test of whether video world models can plan a path toward a named target without a map. Models that generate realistic video may still lack the semantic and spatial reasoning that planning requires.

The benchmark holds hundreds of robot collected scenes across many target types, with measured reference paths. Scoring tracks whether generated motion heads toward the target and stays directionally consistent. The best ready made model scores low overall, revealing a wide gap. Training on a small real robot data set improves planning a great deal.

Catalog record

CREATORDingrui Wang, Hongyuan Ye, Zhihao Liang, Zhexiao Sun, Zhaowei Lu, Yuchen Zhang, Yuyu Zhao, Yuan Gao, Marvin Seegert, Finn Schäfer, Haotong Qin, Wei Li, Luigi Palmieri, Felix Jahncke, Mattia Piccinini, Johannes Betz, TUM Bosch AI Center ETH NJU https://target-bench.github.io/
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-17792

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Target-Bench : Can World Models Achieve Mapless Path Planning with Semantic Targets? Dingrui Wang, Hongyuan Ye, Zhihao Liang, Zhexiao Sun, Zhaowei Lu, Yuchen Zhang, Yuyu Zhao, Yuan Gao, Marvin Seegert, Finn Schäfer, Haotong Qin, Wei Li, Luigi Palmieri, Felix Jahncke, Mattia Piccinini, Johannes Betz, TUM Bosch AI Center ETH NJU https://target-bench.github.io/ Abstract While recent world models generate highly realistic videos, their ability to perform robot path planning remains unclear and unquantified. We introduce Target-Bench , the first benchmark specifically designed to evaluate world models on mapless path planning toward semantic targets in real-world environments. Target-Bench provides 450 robot-collected video sequences spanning 45 semantic categories with SLAM-based ground truth trajectories. Our evaluation pipeline recovers camera motion from generated videos and measures planning performance using five complementary metrics that quantify target-reaching capability, trajectory accuracy, and directional consistency. We evaluate state-of-the-art models including Sora 2, Veo 3.1, and the Wan series. The best off-the-shelf model (Wan2.2-Flash) achieves only 0.299 overall score, revealing significant limitations in current world models for robotic planning tasks. We show that fine-tuning an open-source 5B-parameter model on only 325 scenarios from our dataset achieves 0.345 overall score—an improvement of more than 400% over its base version and 15% higher than the best off-the-shelf model. Figure 1 : Target-Bench provides a dataset collected with a quadruped robot, and a benchmark for evaluating world models in mapless path planning toward text-specified goals with implicit semantic meaning. In Target-Bench, world models receive a camera frame and a textual prompt describing the target state, and predict a future video depicting the trajectory toward the goal. A world decoder then extracts the planned path from this video, which is compared against the maneuver executed by a human-operated quadruped. † † footnotetext: {*} Equal contribution; author order settled via Mario Kart. 1 Introduction “If one day, we bring a humanoid robot to this conference venue, a place it has never seen before, and I can simply say ‘Please bring this bottle of water to someone in the audience,’ and it can smoothly walk over and do it by itself, I think that would be the robot’s ChatGPT moment.” — Xingxing Wang, Head of Unitree, at the World Robot Conference 2025. Embodied AI has been advancing rapidly, while its core challenges are becoming clearer. Despite major improvements (better actuators and sensors) in robotic hardware, AI software remains the main bottleneck limiting robots from realizing their full potential 22 . As a result, the robotics and embodied AI community is seeking to develop AI systems that are more robust and more generalizable. Recent breakthroughs in World Models (WMs) 4 ; 7 ; 35 ; 29 ; 32 have drawn attention from the community. WMs learn to predict how the world evolves over time 10 ; 18 . Given an initial observation (e.g., an image frame) and a condition (e.g., a text prompt or an action), models such as Veo3.1 7 and Genie3 4 can generate future frames with high spatio-temporal consistency. More importantly, they exhibit remarkable reasoning abilities in visual semantics and causal relationships 5 ; 35 ; 9 . This potential has sparked interest within the embodied AI community in applying WMs to robotic applications 19 ; 39 ; 26 ; 33 . Recent approaches from Unitree, such as UnifoLM-WMA-0 30 , aim to bridge WMs with low-level robot control. The underlying philosophy of these approaches is that if a model can accurately predict how the world evolves, its predictions can serve as a plan to guide the robot actions. However, a key question remains: How accurate must these predictions be to count as useful for planning? More broadly, how can we quantitatively assess a WM ’s reasoning, task-solving, and planning ability? Existing evaluation frameworks focus mainly on reconstruction quality, from visual fidelity to physical consistency 20 ; 8 , while assessing mapless path planning driven by semantic understanding remains an open challenge 22 . Bridging this gap requires a benchmark that evaluates not just how realistic a WM ’s predictions look, but whether they contain the semantic reasoning needed for robotic planning tasks. To address this challenge, we introduce Target-Bench . To our knowledge, it is the first benchmark evaluating WMs for mapless path planning toward semantic targets in unstructured real-world environments. The target state is specified by user text and can have implicit semantic meaning (Fig. 1 ). Our contributions are: • An open-source dataset of 450 videos (112,500 frames) collected with a quadruped robot, covering 45 semantic target categories in diverse indoor and outdoor environments, with ground truth trajectories and human annotations for explicitly- and implicitly-stated targets. • The first systematic evaluation pipeline for WMs in mapless path planning with textual semantic goals. • A world decoder that extracts camera motion from generated videos, comparing spatial reconstruction methods and introducing a new scale recovery technique. • A comparative study of open-source and proprietary WMs , including the first fine-tuning of an open-source model on a small real-world dataset for path planning, demonstrating improved generalization to unseen environments over proprietary models. 2 Related Work World Models. Early approaches such as World Models 10 and Dreamer 11 introduced latent state-space models, demonstrating that generative prediction has the potential to support planning and control 9 . More recently, foundation-scale methods pretrained on large video corpora have emerged: Cosmos 25 compared diffusion and autoregressive paradigms; DIAMOND 2 improved visual fidelity; V-JEPA 2 3 enhanced efficiency by forecasting in latent space. Advances in video generation, including Sora 2 29 , Veo 3.1 7 ; 35 , and Wan 32 , have further improved prompt adherence and physics-aware dynamics, enabling long-horizon predictions. Interactive models such as Genie-1 6 , Genie-2 27 , and Genie-3 4 provide controllable environments for use in robotics and games. Unitree’s recent UnifoLM-WMA-0 30 aims to extract the reasoning capabilities of WMs and translate them into real-world actions. However, the ability of WMs to perform robot task planning in real-world scenarios is largely untested. Benchmarks for World Models. Recent works have proposed benchmarks to evaluate generative WMs , often relying on Vision Language Model (VLM) -based judges to assess the quality and consistency of generated scenes. VBench 15 targets 16 fine-grained video quality dimensions, while WorldModelBench 20 focuses on physics consistency (e.g., Newtonian motion). Compared to VBench, WorldScore 8 emphasizes controllability and dynamics in next-scene prediction. World-in-World 38 enables closed-loop evaluation but uses WMs as simulators and does not test their planning abilities. Overall, existing benchmarks primarily assess spatio-temporal or physical consistency, and do not consider high-level robot planning. Recent advances in spatio-temporal scene reconstruction 34 ; 36 ; 14 further highlight the limits of using only VLM -based scoring for WM evaluation. We extend this literature by assessing not only spatio-temporal coherence but also path planning abilities. (a) Quadruped robot hardware setup and communication network. (b) SLAM pipeline for state estimation. Figure 2 : Robot setup and SLAM pipeline. (a) All trajectories. (b) Word cloud of captions. (c) Semantic target categories. (d) Target-Bench Evaluation Results. Figure 3 : Visualization of dataset structure and semantics. Semantic Navigation Datasets. Several datasets address semantic robot navigation, with social compliance in real-world scenarios. EgoWalk 1 provides egocentric data with stereo images, odometry, and traversability labels, but suffers from

Read the complete text