Enhancing Physical Consistency in Lightweight World Models
Dingrui Wang, Zhexiao Sun, Zhouheng Li, Cheng Wang, Youlun Peng, Hongyuan Ye, Baha Zarrouki, Wei Li, Mattia Piccinini, Lei Xie, Johannes Betz, https://physics-wm.github.io/ · 2025
Read Enhancing Physical Consistency in Lightweight World Models by Dingrui Wang, Zhexiao Sun, Zhouheng Li, Cheng Wang, Youlun Peng, Hongyuan Ye, Baha Zarrouki, Wei Li, Mattia Piccinini, Lei Xie, Johannes Betz, https://physics-wm.github.io/ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-12437.txt.
Introduction
World models let a robot or car imagine the near future, but small models that fit on edge hardware often imagine sloppy physics. Dingrui Wang and ten co-authors present PIWM in 2025, a compact physics-informed world model that works in a bird's-eye view of the scene.
Two tricks carry the design. A soft mask used in training sharpens how moving objects are modeled and predicted, and a warm start step at inference lifts prediction quality even with no task-specific training. At equal size the model beats its baseline's weighted score by a wide margin, and the smallest version still beats the largest baseline while running faster.
Catalog record
| CREATOR | Dingrui Wang, Zhexiao Sun, Zhouheng Li, Cheng Wang, Youlun Peng, Hongyuan Ye, Baha Zarrouki, Wei Li, Mattia Piccinini, Lei Xie, Johannes Betz, https://physics-wm.github.io/ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-12437
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Physical Consistency in Lightweight World Models Dingrui Wang, Zhexiao Sun, Zhouheng Li, Cheng Wang, Youlun Peng, Hongyuan Ye, Baha Zarrouki, Wei Li, Mattia Piccinini, Lei Xie, Johannes Betz, https://physics-wm.github.io/ Abstract A major challenge in deploying world models is the trade-off between size and performance. Large world models can capture rich physical dynamics but require massive computing resources, making them impractical for edge devices. Small world models are easier to deploy but often struggle to learn accurate physics, leading to poor predictions. We propose the Physics-Informed BEV World Model (PIWM), a compact model designed to efficiently capture physical interactions in bird’s-eye-view (BEV) representations. PIWM uses Soft Mask during training to improve dynamic object modeling and future prediction. We also introduce a simple yet effective techniques— Warm Start for inference to enhance prediction quality with zero-shot model. Interactive experiments show that at the same parameter scale (400 M ), PIWM surpasses the baseline by 60.6% in weighted overall score. Moreover, even when compared with the largest baseline model (400 M ), the smallest PIWM (130 M Soft Mask) achieves a 7.4% higher weighted overall score with a 28% faster inference speed. I Introduction The Genie series of models [ 3 , 4 , 5 ] has significantly advanced research on world models. Beginning with the original Genie and Genie 2, these systems introduced the ability to generate interactive environments from data, enabling agents to learn and act within diverse virtual worlds. The most recent iteration, Genie 3, demonstrates emergent physical reasoning capabilities: it can simulate gravity, collisions, and object interactions without relying on an explicit physics engine. This capability indicates that scaling large world models can lead to a deeper, implicit understanding of the laws of physics, a critical step towards more general embodied intelligence. Among the world models, bird’s-eye-view (BEV) world models [ 6 , 7 ] are particularly promising for motion prediction and future trajectory modeling. By representing the environment in a top-down view [ 8 , 9 ] , BEV models can capture spatial relationships and object interactions more effectively than first-person or ego-centric views, making them suitable for navigation tasks in robotics. Running world models online is constrained by edge hardware performance. While devices such as the NVIDIA Jetson Orin Nano Super (67 TOPS) [ 10 ] , AGX Orin (200–275 TOPS) [ 11 ] , or Tesla HW4 (500-720 TOPS) [ 12 ] support moderate models, state-of-the-art video/world models like HunyuanVideo [ 13 ] and Veo 3 [ 14 ] with hundreds of billions of parameters remain infeasible outside GPU/TPU clusters. More compact models, e.g., DIAMOND (0.4B) runs at 21 FPS on an RTX 4080 Laptop GPU (542 TOPS) but only 9.5 FPS on 100-TOPS edge devices. Both rates fall below the human perceptual smoothness threshold ( \sim 24 FPS), which challenges real-time deployment. While further shrinking the world model to run smoothly on edge devices is feasible, a fundamental challenge remains: can compact models capture the richness of physical dynamics without sacrificing reasoning ability? Reducing model size alone often leads to oversimplification and loss of fidelity. Addressing this gap requires shifting focus from scale to intelligent design—leveraging physics-informed data, inductive biases, and targeted training. Such models can move beyond pattern memorization, enabling robust generalization and effective reasoning under resource constraints. We address this challenge by extending the 400M-parameter world model DIAMOND [ 1 ] to better capture physical consistency (Fig. ), by introducing a new Physics-Informed World Model (PIWM) with the following contributions: • Soft Mask. We propose a method that extracts spatial semantic information, emphasizing the existence of dynamic objects while preserving action sensitivity. Our Soft Mask improves temporal and perceptual video consistency and achieves higher human-judged physics scores than the baseline [ 1 ] . Experiments further show that our method enables parameter reduction for edge deployment without compromising physical consistency. • Warm Starting. We introduce a zero-shot Warm Start method, that injects contextual information at inference time to improve generation stability at small scales. It can be directly plugged into any pretrained diffusion-based world model, and yields FID gains over the baseline [ 1 ] . • Open-source models and dataset. We collect 2,000 episodes in HighwayEnv [ 2 ] with an MCTS agent, yielding 2 million BEV frames with aligned states and actions. Rewards and actions are carefully designed to evenly cover interaction and lane distributions, thereby supporting future exploration in physical consistency for the community. II Related Work World Models have demonstrated the potential to learn complex world dynamics. Large-scale models such as the Genie series [ 3 , 4 , 5 ] and Veo 3 [ 14 ] show the benefits of scale for long-horizon prediction and richer environment simulation, while recent works [ 1 , 15 , 16 ] target lightweight models that can run with limited resources. DIAMOND [ 1 ] trains an RL agent fully inside a 0.4B-parameter diffusion world model. Inspired by these advances, researchers have applied world models to autonomous driving [ 17 , 18 , 19 , 20 ] . Within this domain, Bird’s eye view (BEV) has emerged as a promising research direction, leading to attempts of interpreting BEV features as a world model. BEVDiffuser [ 21 ] designs a diffusion model to denoise the BEV feature maps, gaining a significant performance improvement, while Li et al. [ 7 ] introduce WoTE, a BEV world model for efficient and real-time future prediction and trajectory evaluation. Edge computing. Large models are often unsuitable for edge deployment in autonomous driving due to high computational and memory demands. Although compression techniques offer efficiency gains, they pose critical limitations in safety-critical settings. Quantization [ 22 , 23 , 24 ] accelerates inference, but amplifies numerical errors under distribution changes. Pruning [ 25 , 26 , 27 ] risks removing weights vital for rare scenarios, harming robustness. Knowledge distillation [ 28 , 29 , 30 ] compresses models but often fails to preserve fine-grained spatiotemporal reasoning and precise decision boundaries, degrading performance in tasks. Thus, achieving compact, efficient, and reliable models for real-time edge deployment remains a key challenge. This motivates the design of small models yet capable of accurate and robust dynamic reasoning under resource constraints. Encoding Physics into World Models. Because large models are hard to deploy on edge devices, distillation [ 31 , 32 ] is widely used but often yields weaker or unstable performance. We instead train a compact world model tailored for edge constraints. Beyond efficiency, physics understanding is a key criterion for world models [ 33 , 34 ] , indicating whether they capture causal dynamics rather than merely reproducing appearance. DrivePhysica [ 35 ] improves physics awareness in driving world models by enforcing motion in a reference system, temporal consistency, and correct spatial relations. Using pretrained depth and semantic models, World4Drive [ 36 ] can gain a deep understanding of the spatial and semantic properties of the physical world. Yet maintaining strong physics reasoning in compact models remains open. Here, we show that PIWM effectively strengthens physical understanding under tight resource budgets. III Methodology Fig. 2 : Illustration of soft masks and hard mask. In each 3D cubic shape in the mask figures, the color gradient indicates the mask value