Action Images: End-to-End Policy Learning via Multiview Video Generation
Haoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan · 2026
Read Action Images: End-to-End Policy Learning via Multiview Video Generation by Haoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06168.txt.
Description
Zhen and nine colleagues turn robot policy learning into multiview video generation. Instead of coding control as compact tokens, they translate seven-degree-of-freedom robot actions into action images: multi-view videos grounded in pixels that track arm motion directly. That grounding lets the video backbone itself act as a zero-shot policy, with no separate policy head or action module. The same model also handles joint video-action generation and action labeling. On RLBench and real-world tests it posts the strongest zero-shot success rates yet.
Catalog record
| CREATOR | Haoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36283 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.06168