HumanOmni-Speaker: Identifying Who said What and When
Detao Bai, Shimin Yao, Weixuan Chen, Xihan Wei, Zhiheng Ma · 2026
Read HumanOmni-Speaker: Identifying Who said What and When by Detao Bai, Shimin Yao, Weixuan Chen, Xihan Wei, Zhiheng Ma (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21664.txt.
Description
Bai and co-authors tackle a hard video task: who said what, and when, in multi-person scenes. Omni-modal models often cheat here, leaning on visual bias in test sets instead of true sound-vision links. Sparse frame sampling also erases lip motion. Their test setup removes visual shortcuts and forces real identity binding from plain language queries. Their model samples video at 25 frames per second and squeezes motion into six tokens per frame. It can lip-read and locate speakers with no cropping, they report.
Catalog record
| CREATOR | Detao Bai, Shimin Yao, Weixuan Chen, Xihan Wei, Zhiheng Ma |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38083 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21664