Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning
Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang · 2026
Read Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making and Planning by Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11219.txt.
Description
Yuehao Song and coauthors fix a consistency gap in self-driving stacks pairing a vision-language model with an end-to-end driving policy. Without alignment, the planned trajectory can drift from the model's high-level decision. Their Senna-2 system trains in three stages: pre-training decisions and planning with an adapter passing the model's decisions as embeddings, then aligning the two systems in open loop. A final closed-loop stage uses hierarchical reinforcement learning in simulated 3D worlds. Experiments show stronger decision-planning consistency and better safety.
Catalog record
| CREATOR | Yuehao Song, Shaoyu Chen, Hao Gao, Yifan Zhu, Weixiang Yue, Jialv Zou, Bo Jiang, Zihao Lu, Yu Wang, Qian Zhang, Xinggang Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44315 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11219