Action Images: End-to-End Policy Learning via Multiview Video Generation

Haoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan · 2026

Read Action Images: End-to-End Policy Learning via Multiview Video Generation by Haoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-06168.txt.

Description

Zhen and nine colleagues turn robot policy learning into multiview video generation. Instead of coding control as compact tokens, they translate seven-degree-of-freedom robot actions into action images: multi-view videos grounded in pixels that track arm motion directly. That grounding lets the video backbone itself act as a zero-shot policy, with no separate policy head or action module. The same model also handles joint video-action generation and action labeling. On RLBench and real-world tests it posts the strongest zero-shot success rates yet.

Catalog record

CREATORHaoyu Zhen ∗, Zixian Gao ∗ †, Qiao Sun †, Yilin Zhao, [3pt] Yuncong Yang, Yilun Du, Pengsheng Guo, [3pt] Tsun-Hsuan Wang, Yi-Ling Qiao, Chuang Gan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36283
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.06168

Browse the full library · About the Tower of Babel