SUGAR: Learning Skeleton Representation with Visual-Motion Knowledge for Action Recognition
Qilang Ye, Yu Zhou, Lian He, Jie Zhang, Xuanming Guo, Jiayu Zhang, Mingkui Tan, Weicheng Xie, Yue Sun, Tao Tan, Xiaochen Yuan, Ghada Khoriba, Zitong Yu · 2025
Read SUGAR: Learning Skeleton Representation with Visual-Motion Knowledge for Action Recognition by Qilang Ye, Yu Zhou, Lian He, Jie Zhang, Xuanming Guo, Jiayu Zhang, Mingkui Tan, Weicheng Xie, Yue Sun, Tao Tan, Xiaochen Yuan, Ghada Khoriba, Zitong Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10091.txt.
Description
Qilang Ye and twelve co-authors present SUGAR, a method that lets a large language model recognize human actions from skeleton data. Skeletons are compact and privacy friendly, but a text-trained model cannot read joint coordinates natively, nor tell similar actions apart without help. SUGAR first mines big video models as a knowledge base for visual and motion information about each action.
That prior knowledge supervises the learning of discrete skeleton representations, which the language model then reads with its pre-trained weights untouched to produce action labels and descriptions. A temporal query projection module handles long skeleton sequences. Benchmarks on skeleton action classification favored the approach, and zero-shot tests showed more versatility than linear methods, suggesting language models can serve as skeleton recognizers when given the right translation.
Catalog record
| CREATOR | Qilang Ye, Yu Zhou, Lian He, Jie Zhang, Xuanming Guo, Jiayu Zhang, Mingkui Tan, Weicheng Xie, Yue Sun, Tao Tan, Xiaochen Yuan, Ghada Khoriba, Zitong Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-10091