OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation

Weiqi Li, Shijie Zhao, Chong Mou, Xuhan Sheng, Zhenyu Zhang, Qian Wang, Junlin Li, Li Zhang, Jian Zhang · 2024

Read OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation by Weiqi Li, Shijie Zhao, Chong Mou, Xuhan Sheng, Zhenyu Zhang, Qian Wang, Junlin Li, Li Zhang, Jian Zhang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-09623.txt.

Description

Weiqi Li and eight co-authors wrote a 2024 paper on OmniDrag, a method for controlling motion in omnidirectional videos generated from a single image. Virtual reality needs immersive 360 degree video that creators can direct. Text prompts alone give loose control and often produce content errors.

Drag based motion control works for flat video, but spherical motion breaks it, causing distortion. OmniDrag is presented as the first method to offer both scene level and object level motion control for this format. It builds on pretrained video diffusion models, adding an omnidirectional control module tuned together with the temporal layers. A new spherical motion estimator extracts control signals faithfully and lets users guide movement on the sphere.

Catalog record

CREATORWeiqi Li, Shijie Zhao, Chong Mou, Xuhan Sheng, Zhenyu Zhang, Qian Wang, Junlin Li, Li Zhang, Jian Zhang
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-09623

Browse the full library · About the Tower of Babel