Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach

Yunuo Chen, Junli Cao, Vidit Goel, Sergei Korolev, Chenfanfu Jiang, Jian Ren, Sergey Tulyakov, Anil Kag · 2025

Read Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach by Yunuo Chen, Junli Cao, Vidit Goel, Sergei Korolev, Chenfanfu Jiang, Jian Ren, Sergey Tulyakov, Anil Kag (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03639.txt.

Description

Yunuo Chen and colleagues teach video generators some three-dimensional common sense by annotating training videos with moving points in space. Current models morph objects and break physics because they lack shape awareness. Writing in twenty twenty-five, the team augments ordinary clips with three-dimensional point trajectories, aligned to the pixels. A latent diffusion model fine-tuned on the enriched set learns to track flat objects with true spatial coordinates. On top of that, shape and motion regularization penalizes non-physical deformation in the output.

Contact-rich scenes are the showcase. When solid objects interact, the spatial information keeps hands and tools coherent. The method plugs into existing video diffusion models rather than replacing them. The authors frame plausibility as a geometry problem: give the model points to respect, and the pictures behave.

Catalog record

CREATORYunuo Chen, Junli Cao, Vidit Goel, Sergei Korolev, Chenfanfu Jiang, Jian Ren, Sergey Tulyakov, Anil Kag
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03639

Browse the full library · About the Tower of Babel