AirScape: An Aerial Generative World Model with Motion Controllability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelli
Baining Zhao ∗ , Rongze Tang ∗ , Mingyuan Jia ∗ , Ziyou Wang ∗ , Fanghang Man , Xin Zhang , Yu Shang , Weichen Zhang , Chen Gao † , Wei Wu , Xin Wang , Xinlei Chen † , Yong Li Tsinghua University Project page: https://embodiedcity.github.io/AirScape/ · 2025
Read AirScape: An Aerial Generative World Model with Motion Controllability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelli by Baining Zhao ∗ , Rongze Tang ∗ , Mingyuan Jia ∗ , Ziyou Wang ∗ , Fanghang Man , Xin Zhang , Yu Shang , Weichen Zhang , Chen Gao † , Wei Wu , Xin Wang , Xinlei Chen † , Yong Li Tsinghua University Project page: https://embodiedcity.github.io/AirScape/ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08885.txt.
Introduction
Baining Zhao and a team of co-authors build a world model for drones in a 2025 paper. A world model lets an agent predict what it will see after it moves. AirScape is the first such model made for aircraft with six degrees of freedom, predicting future views from current images and intended motions.
The team built a data set of eleven thousand video and intention pairs, with over a thousand hours spent labeling drone actions. They trained a foundation model in two phases so it obeys physical limits in space and time. In tests, AirScape beat rival foundation models at imagining 3D scenes, with over fifty percent better scores on motion alignment. The project page is public.
Catalog record
| CREATOR | Baining Zhao ∗ , Rongze Tang ∗ , Mingyuan Jia ∗ , Ziyou Wang ∗ , Fanghang Man , Xin Zhang , Yu Shang , Weichen Zhang , Chen Gao † , Wei Wu , Xin Wang , Xinlei Chen † , Yong Li Tsinghua University Project page: https://embodiedcity.github.io/AirScape/ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-08885
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AirScape: An Aerial Generative World Model with Motion Controllability DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence Baining Zhao ∗ , Rongze Tang ∗ , Mingyuan Jia ∗ , Ziyou Wang ∗ , Fanghang Man , Xin Zhang , Yu Shang , Weichen Zhang , Chen Gao † , Wei Wu , Xin Wang , Xinlei Chen † , Yong Li Tsinghua University Project page: https://embodiedcity.github.io/AirScape/ Abstract . How to enable robots to predict the outcomes of their own motion intentions in three-dimensional space has been a fundamental problem in embodied intelligence. To explore more general spatial imagination capabilities, here we present AirScape, the first world model designed for six-degree-of-freedom aerial agents. AirScape predicts future observation sequences based on current visual inputs and motion intentions. Specifically, we construct an dataset for aerial world model training and testing, which consists of 11k video-intention pairs. This dataset includes first-person-view videos capturing diverse drone actions across a wide range of scenarios, with over 1,000 hours spent annotating the corresponding motion intentions. Then we develop a two-phase training schedule to train a foundation model—initially devoid of embodied spatial knowledge—into a world model that is controllable by motion intentions and adheres to physical spatio-temporal constraints. Figure 1. In three-dimensional space, AirScape can predict the sequence of observations that would result if a six-degree-of-freedom aerial agent executed a series of actions to achieve an intention, based on current visual observations. AirScape can handle diverse actions (translation, rotation, and their combinations), environments (rural, urban), viewpoints (top-down, horizon), and lighting conditions (daytime, dusk, nighttime), simulating embodied observation characteristics such as perspective and parallax. 1. Introduction The unprecedented advancements in generative models ( Cao et al. 2024 ) have catalyzed a paradigm shift in the development of world models, enabling generation and simulation of the real-world environment, with inputs of texts and actions. The simulation reflects a kind of high-level capability, counterfactual reasoning, which enables the simulation and prediction of possible outcomes based on hypothetical conditions or decisions ( Ding et al. 2024 ) . By comparing results under different assumptions, world models support better decision-making in unknown or complex environments, which particularly important for downstream applications such as embodied robotics ( Wu et al. 2023 ; Li et al. 2025 ) , autonomous driving ( Gao et al. 2023 ; Gao et al. 2024a ) , etc. Moreover, the world model can enhance the agents’ spatial intelligence ( Gupta et al. 2021 ) , with the human-like critical ability in understanding the environment. Specifically, when an agent operates in a three-dimensional real-world space, given its current observation, we expect it to predict how its first-person perspective will change after performing actions or tasks, which implicitly indicates how its spatial relationship with the surrounding environment will evolve. This goes beyond basic spatial perception and understanding, enabling navigation and task planning in complex and unknown real-world scenarios. Current research on spatial world models primarily focuses on humanoid robots and autonomous driving applications ( Guan et al. 2024 ; Sakagami et al. 2023 ) . However, world models for humanoid robots emphasize manipulation and indoor environment modeling, while those for autonomous driving focus on predicting driving behaviors and modeling road dynamics. Both humanoid robots and vehicles operate mostly in two-dimensional planes with limited action spaces. In contrast, we attempt to explore a more general world model designed for three-dimensional spaces, as presented in Figure 1 . On one hand, constructing a world model for aerial spaces allows for the exploration of embodied spatial counterfactual reasoning in 3D real-world environments with six degrees of freedom (6DoF). On the other hand, with the development of the low-altitude economy, the increasing intelligence of aerial agents, such as drones, will drive their widespread applications. Vision is one of the fundamental perceptual modalities, and the latest visual observations inherently contain spatial information. Compared to flight control variables or specific trajectory coordinates, we argue that expressing action intentions in textual form aligns more closely with human reasoning processes and offers greater flexibility. Text-based instructions can represent high-level navigation instructions, such as "move to the boat ahead" or "follow the car in front," as well as low-level commands, such as "rotate 90 degrees to the left." When a series of actions is executed in the current spatial context to fulfill an intention, the most direct outcome is a sequence of visual observations. This input-output structure aligns with the framework of video generation models conditioned on both graphical and textual inputs. Recently, video generation foundation models, represented by diffusion and autoregressive models, have rapidly advanced and are becoming important tools for implementing world models. Inspired by the scaling law, large foundation models exhibit generalization capabilities. Video generation models can efficiently model dynamic changes in temporal sequences, directly simulating visual information of the environment. This capability aligns closely with the requirements of world models for modeling and predicting future spatio-temporal states. However, constructing spatial world models still faces the following challenges: • Lack of aerial datasets : Training world models requires first-person perspective videos and corresponding textual prompts about aerial agents’ actions or tasks. Existing datasets are either third-person views or ground-based perspectives from robots or vehicles. • Distribution gap between foundation models and spatial world models : In terms of text input, existing open-source foundation models focus on generating videos from detailed textual descriptions, whereas world models rely on concise instructions or action intents. In terms of video, training data for open-source foundation models mostly consists of third-person videos with limited visual changes, while embodied first-person perspectives typically have narrower fields of view and larger visual changes, increasing training difficulty. • Diversity in generation : Drones operate in 6DoF with high flexibility. Compared to ground vehicles, generated scenes include lateral translation, in-place rotation, camera gimbal adjustments, and combinations of multiple actions, making generation more challenging. The aerial spatial world model is required to simulate more complex changes in relative position, perspective variation, and parallax effects. To address these issues, we first introduce an 11k dataset for training aerial world models. We collect videos from three public drone datasets, segment and filter them, and annotate each video clip with its corresponding actions and intents using large multimodal models (LMMs) and human refinement ( Section 3 ). Subsequently, we design a two-stage training schedule: fine-tuning video generation foundation models to adapt to the text and video distributions; leveraging the discriminative capabilities of LMMs, rejection sampling and self-play training are employed to further improve generation outputs that violate spatial physical constraints. ( Section 4 ). Experimental results demonstrate that the proposed spatial world model can predict changes in embodied perspectives and spatial relationships when performing various actions or tasks ( Section 5 ). The main contributions of this paper are as follows: • The first dataset for training and testing generative world models in aerial spaces , containing 11k video clips with corresponding prompts, including actions, intents, or task instructions. • The first generative world model in aerial spaces , capable of predicting visual observations from controllable intentions in three-dimensional spaces. • Experimental analysis shows that, compared to general video generation models