Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation

Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski · 2025

Read Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation by Pinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-07239.txt.

Description

Liu and five co-authors generated speech-matched gestures on video in this paper from twenty twenty-five. Avatars feel alive when their hands move with their words. Older systems miss the exact beats or meanings that should trigger a gesture. Their framework attacks the problem in three parts.

A chronological alignment links the speech and motion streams in time. Gesture tokens are then distilled with speech context baked in, so patterns carry meaning. Finally, a structure-aware step reconnects keypoints along the body's edges for cleaner video. Experiments showed realistic, well-synced gesture videos. Long sequences and later editing of gestures were also supported. The authors aim the system at lifelike avatars and interfaces.

Catalog record

CREATORPinxin Liu, Pengfei Zhang, Hyeongwoo Kim, Pablo Garrido, Ari Shapiro, Kyle Olszewski
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-07239

Browse the full library · About the Tower of Babel