HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration
Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao ‡ \ddagger, Su Linxiang ‡ \ddagger, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li † \dagger · 2026
Read HAT-4D: Lifting Monocular Video for 4D Multi-Object Interactions via Human-Agent Collaboration by Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao ‡ \ddagger, Su Linxiang ‡ \ddagger, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li † \dagger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28215.txt.
Description
Jiaxin Li and thirteen colleagues, with Yong-Lu Li as senior author, want three-dimensional reconstructions of several objects interacting over time, built from a single ordinary video. Recovering such 4D scenes would cheaply supply training data for embodied artificial intelligence, but existing methods handle lone objects and break under heavy occlusion.
Li's team built HAT-4D, which pairs vision-language models with a person who gives feedback at several stages. Human guidance resolves depth ambiguities and untangles objects hiding each other during generation and tracking. The system produced a new open benchmark of multi-object interaction scenes, with scoring that stresses physical plausibility and steadiness over time. It reached state-of-the-art results on most measures, and its data improved other models when used for fine-tuning.
Catalog record
| CREATOR | Jiaxin Li, Yuxiang Wu, Zhenkai Zhang, Xinrui Shi, Haoyuan Wang, Yichen Zhao ‡ \ddagger, Su Linxiang ‡ \ddagger, Chenyang Yu, Mingyu Zhang, Yifan Ding, Boran Wen, Li Zhang, Ruiyang Liu, Yong-Lu Li † \dagger |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28215