Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning

Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang · 2026

Read Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning by Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11219.txt.

Description

Yuehao Song and coauthors fix a consistency gap in self-driving stacks pairing a vision-language model with an end-to-end driving policy. Without alignment, the planned trajectory can drift from the model's high-level decision. Their Senna-2 system trains in three stages: pre-training decisions and planning with an adapter passing the model's decisions as embeddings, then aligning the two systems in open loop. A final closed-loop stage uses hierarchical reinforcement learning in simulated 3D worlds. Experiments show stronger decision-planning consistency and better safety.

Catalog record

CREATORYuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44315
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.11219

Browse the full library · About the Tower of Babel