PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance
Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Fei-Yue Wang, Zhibo Chen · 2026
Read PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance by Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Fei-Yue Wang, Zhibo Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18639.txt.
Introduction
Wang and co-authors build OrthoPhys to make generated videos obey basic physics. Video models see only flat views of motion that unfolds in 3D space. OrthoPhys first creates four synced orthogonal views of the moving subject. Cross-view attention enforces 3D coherence across those views. The steady subject then guides synthesis of the full scene with background. Training uses PhysMV, a new set of 40,000 scenes with four views each. Tests show clear gains in physical realism and space-time coherence over prior video methods.
Catalog record
| CREATOR | Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Fei-Yue Wang, Zhibo Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37605 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18639
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PhysVideo: Physically Plausible Video Generation with Cross-View Geometry Guidance Cong Wang, Hanxin Zhu, Xiao Tang, Jiayi Luo, Xin Jin, Long Chen, Fei-Yue Wang, Zhibo Chen Abstract Recent progress in video generation has led to substantial improvements in visual fidelity, yet ensuring physically consistent motion remains a fundamental challenge. Intuitively, this limitation can be attributed to the fact that real-world object motion unfolds in three-dimensional space, while video observations provide only partial, view-dependent projections of such dynamics. To address these issues, we propose PhysVideo , a two-stage framework that first generates physics-aware orthogonal foreground videos and then synthesizes full videos with background. In the first stage, Phys4View leverages physics-aware attention to capture the influence of physical attributes on motion dynamics, and enhances spatio-temporal consistency by incorporating geometry-enhanced cross-view attention and temporal attention. In the second stage, VideoSyn uses the generated foreground videos as guidance and learns the interactions between foreground dynamics and background context for controllable video synthesis. To support training, we construct PhysMV , a dataset containing 40K scenes, each consisting of four orthogonal viewpoints, resulting in a total of 160K video sequences. Extensive experiments demonstrate that PhysVideo significantly improves physical realism and spatial-temporal coherence over existing video generation methods. Home page: https://anonymous.4open.science/w/Phys4D/ . 1 Introduction Recent advances in video generation have significantly expanded the capability of visual synthesis models, enabling the generation of high-fidelity and temporally coherent videos conditioned on multimodal inputs such as text and images ( 2 ; 50 ; 21 ; 47 ; 38 ) . In particular, text- and image-conditioned video generation (TI2V) has emerged as a powerful paradigm for synthesizing dynamic visual content that aligns semantic intent with visual appearance. As perceptual quality improves, these models are increasingly seen as a foundation for interactive content creation, simulation-based reasoning, and embodied AI in dynamic visual environments. Figure 1 : Comparison of our method with prior video generation paradigms . (a) Data-driven generative models. (b) Physics-engine-based methods. (c) Our physics-aware two-stage framework utilizing foreground multi-view videos. Despite this progress, current video generation models still struggle to produce physically plausible and spatially consistent motion ( 1 ; 31 ) . While visually convincing at the pixel level, generated videos often violate basic physical principles, exhibiting implausible accelerations, inconsistent object interactions, or motion patterns that are insensitive to intrinsic object properties such as mass or elasticity, as shown in Fig. 1 .(a). These issues become particularly evident in background-aware synthesis, where realistic interactions between moving foreground objects and the surrounding scene are essential. We attribute these limitations to a mismatch between pixel-space motion modeling and physically constrained motion. Existing video generators typically learn dynamics as appearance transformations across frames, yet physically plausible motion and deformation are governed by geometry and material properties and should remain consistent across viewpoints. Explicit 3D representations and physics-based simulation ( 43 ; 25 ; 27 ) can produce valid trajectories, but they often rely on multi-stage pipelines that are difficult to scale and hard to integrate into end-to-end training, and they can hinder efficient feed-forward generation under flexible multimodal conditioning. Instead, we consider a setting that directly outputs synchronized multi-view foreground motion (e.g., four viewpoints) conditioned on physical attributes, and evaluates physical realism through both input-consistency with the specified physical properties and cross-view consistency, without explicitly outputting 3D structure or performing explicit 3D modeling. In this work, we take a foreground-first view of physically grounded motion: we generate synchronized multi-view foreground dynamics under explicit physical attributes, and then use them to guide final video synthesis. Building on this idea, we propose PhysVideo , a physics-driven framework that enables geometry-aware motion modeling for video generation without explicitly reconstructing 3D geometry. PhysVideo adopts a two-stage design. In the first stage, Phys4View generates multi-view foreground motion conditioned on a single image, a textual prompt, and explicit physical attributes. By injecting physical parameters directly into the attention mechanism and conditioning on control videos constructed from foreground masks and multi-view priors, Phys4View incorporates physical and geometric cues into motion generation. We further introduce a geometry-aware cross-view attention module together with temporal attention to enhance cross-view alignment and spatiotemporal coherence, enabling simultaneous generation of synchronized four-view videos. In the second stage, VideoSyn synthesizes background-aware image-to-video results by leveraging the generated multi-view motion as guidance, allowing the model to learn data-driven foreground–background interactions such as contact, occlusion, and context-consistent motion. To support training of the four-view video generation model, we employ a physics engine together with 3D-GS object representations to construct a physics-driven multi-view dataset, PhysMV, which comprises 10K 3D objects and 40K orthogonal video sets, totaling 160K videos. Our contributions can be summarized as follows: • We introduce PhysVideo , a feed-forward framework for physics-driven, geometry-aware motion modeling in video generation, conditioned on images, text, and explicit physical attributes. • We propose Phys4View , which integrates physics-aware attention with spatiotemporal and cross-view modeling to generate temporally coherent and geometrically consistent multi-view motion. • We develop VideoSyn for background-aware image-to-video synthesis, where multi-view motion guidance enables data-driven learning of realistic foreground–background interactions without explicit physical simulation at inference. • We construct a large-scale physics-oriented multi-view video dataset using a physics engine. We further show that PhysVideo, leveraging multi-view foreground motion generation followed by motion-aware video synthesis, achieves superior physical realism, temporal coherence, and cross-view consistency compared to prior video generation methods. 2 Related Works 2.1 Controllable Video Generation Video generation models trained on large-scale text–video paired datasets have demonstrated remarkable capabilities in synthesizing high-quality videos ( 15 ; 2 ; 21 ; 47 ) . Previous studies have shown that pretrained models can be additionally guided by various control signals, including camera motion ( 9 ; 14 ) , point trajectories ( 11 ; 13 ; 3 ) , and anchor-frame videos ( 5 ; 37 ) , enabling more controllable video generation. Meanwhile, other research efforts have explored leveraging different modalities to guide motion-aware video synthesis. Some approaches ( 23 ; 19 ) employ optical flow videos as guidance, using flow maps to characterize motion dynamics and subsequently synthesize RGB videos based on them. Other methods ( 24 ; 29 ) use depth videos as conditioning inputs to guide the generation of temporally coherent video sequences. In addition, several text-driven approaches ( 44 ) attempt to infer more fine-grained descriptions of motion dynamics from textual instructions, thereby enabling more precise control over motion behaviors during video generation. However, these methods generally lack the modeling of physical laws and often produce results that violate basic principles of physical plausibility. To address this limitation, our method generates physics-aware multi-view videos as motion representations and subsequently renders them into plausible video outputs. Figure 2 : Pipeline of PhysVideo . PhysVideo generates physically plausible videos via a two-stage pipeline: 1) generating physics-aware orthogonal foreground videos, 2) generating a plausible video with the guidance of foreground motion. 2.2 Physics-grounded Video Generation Physics engines typically use the 3D representation of foreground objects as input to simulate and generate the 3D state at each time step, based on defined material and motion properties ( 43 ; 4 ) . Leveraging the renderable 3D-GS representation ( 20 ) , some methods ( 43 ) propose using the Material Point Method (MPM) solver to update the 3D Gaussian