CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models

Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu, Zhuoyang Zhang Yecheng Wu Zhaoshuo Li Qianli Ma Song Han Chelsea Finn, Ankur Handa Ming-Yu Liu Donglai Xiang Gordon Wetzstein Tsung-Yi Lin · 2025

Read CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models by Qingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu, Zhuoyang Zhang Yecheng Wu Zhaoshuo Li Qianli Ma Song Han Chelsea Finn, Ankur Handa Ming-Yu Liu Donglai Xiang Gordon Wetzstein Tsung-Yi Lin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22020.txt.

Description

Qingqing Zhao and co-authors add visual foresight to robot control models in a 2025 paper. Vision-language-action models learn control from large robot and non-robot data. Most map inputs straight to outputs with no planning step. Complex manipulation suffers from that missing foresight.

Their model, CoT-VLA, first predicts future image frames as visual goals. It then generates a short action sequence to reach those goals. The seven-billion-parameter model handles visual and action tokens together. Real-world manipulation tests beat the prior best model by a wide margin. Simulation benchmarks also improved over the leader. Autoregressive frame prediction supplies the chain of thought. Temporal planning thus enters the control loop explicitly. The authors release a project site with examples. Generalizable robot skills are the long-term aim of the line.

Catalog record

CREATORQingqing Zhao Yao Lu Moo Jin Kim Zipeng Fu, Zhuoyang Zhang Yecheng Wu Zhaoshuo Li Qianli Ma Song Han Chelsea Finn, Ankur Handa Ming-Yu Liu Donglai Xiang Gordon Wetzstein Tsung-Yi Lin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22020

Browse the full library · About the Tower of Babel