HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models
Xiao Wang, Jingyun Hua, Weihong Lin, Yuanxing Zhang, Fuzheng Zhang Jianlong Wu, Di Zhang Liqiang Nie · 2025
Read HAIC: Improving Human Action Understanding and Generation with Better Captions for Multi-modal Large Language Models by Xiao Wang, Jingyun Hua, Weihong Lin, Yuanxing Zhang, Fuzheng Zhang Jianlong Wu, Di Zhang Liqiang Nie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20811.txt.
Description
Xiao Wang and seven colleagues improve how multimodal language models understand human actions in video, in a paper from twenty twenty-five. Model performance on human action videos is limited by scarce high-quality data. The team built a two-stage annotation pipeline to fix that. It gathers clear action videos and captions them in a standardized format.
Captions use human attributes to distinguish individuals and detail actions and interactions chronologically. Two datasets result: a training set of about one hundred twenty-six thousand video-caption pairs, and a benchmark with hundreds of manual annotations plus two thousand question-answer pairs. Training on their data improved action understanding across four benchmarks. It also improved text-to-video generation results. Both datasets were released publicly for other researchers.
Catalog record
| CREATOR | Xiao Wang, Jingyun Hua, Weihong Lin, Yuanxing Zhang, Fuzheng Zhang Jianlong Wu, Di Zhang Liqiang Nie |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20811