Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation
Xinshun Wang, Peiming Li, Ziyi Wang, Zhongbin Fang, Zhichao Deng, Songtao Wu, Jason Li, Mengyuan Liu · 2026
Read Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation by Xinshun Wang, Peiming Li, Ziyi Wang, Zhongbin Fang, Zhichao Deng, Songtao Wu, Jason Li, Mengyuan Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02401.txt.
Description
Xinshun Wang and 7 co-authors unify seeing human motion and generating it inside one model, in a 2026 arXiv paper. The field is split: perception models read video but only output text, while generation models cannot take raw visual input. Motion vocabularies built from skeletons alone also lose the link to images.
Their framework, Superman, starts with a vision-guided motion tokenizer. Because 3D skeletons align geometrically with visual data, the tokenizer learns jointly from both and builds a shared motion vocabulary. A single multimodal model trained on that vocabulary then handles pose estimation from video, motion prediction, and filling gaps between poses. On standard benchmarks such as Human3.6M, the unified system reaches top or competitive scores across all tasks. One model replaces a fragmented pipeline.
Catalog record
| CREATOR | Xinshun Wang, Peiming Li, Ziyi Wang, Zhongbin Fang, Zhichao Deng, Songtao Wu, Jason Li, Mengyuan Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-02401