Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model
Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua · 2025
Read Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model by Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09533.txt.
Description
Fei Shen and six co-authors build a system that generates talking-face videos which stay consistent over long stretches. Diffusion models can make a face speak from audio or text, but in long clips the head drifts, expressions slip, and lip motion falls out of sync. Their model, MCDM, leans on motion history to keep the performance steady.
One part of the system draws on archived clips plus a reference frame to hold the speaker's identity. A second part predicts present motion, covering head movement, lip sync, and expression together. A memory-efficient attention design stores and updates motion features so errors do not pile up over time. The team also releases TalkingFace-Wild, a dataset of over 200 hours of footage in ten languages. Tests show the model holds identity and motion together well in long generations.
Catalog record
| CREATOR | Fei Shen, Cong Wang, Junyao Gao, Qin Guo, Jisheng Dang, Jinhui Tang, Tat-Seng Chua |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09533