Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua · 2025

Read Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model by Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09533.txt.

Description

Fei Shen and six co-authors build a system that generates talking-face videos which stay consistent over long stretches. Diffusion models can make a face speak from audio or text, but in long clips the head drifts, expressions slip, and lip motion falls out of sync. Their model, MCDM, leans on motion history to keep the performance steady.

One part of the system draws on archived clips plus a reference frame to hold the speaker's identity. A second part predicts present motion, covering head movement, lip sync, and expression together. A memory-efficient attention design stores and updates motion features so errors do not pile up over time. The team also releases TalkingFace-Wild, a dataset of over 200 hours of footage in ten languages. Tests show the model holds identity and motion together well in long generations.

Catalog record

CREATORFei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09533

Browse the full library · About the Tower of Babel