Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation
Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski · 2025
Read Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation by Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07239.txt.
Description
Liu and five co-authors generated speech-matched gestures on video in this paper from twenty twenty-five. Avatars feel alive when their hands move with their words. Older systems miss the exact beats or meanings that should trigger a gesture. Their framework attacks the problem in three parts.
A chronological alignment links the speech and motion streams in time. Gesture tokens are then distilled with speech context baked in, so patterns carry meaning. Finally, a structure-aware step reconnects keypoints along the body's edges for cleaner video. Experiments showed realistic, well-synced gesture videos. Long sequences and later editing of gestures were also supported. The authors aim the system at lifelike avatars and interfaces.
Catalog record
| CREATOR | Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 62 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-07239