Enhancing Vision-Language Models for Autonomous Driving through Task-Specific Prompting and Spatial Reasoning

Aodi Wu, Xubo Luo · 2025

Read Enhancing Vision-Language Models for Autonomous Driving through Task-Specific Prompting and Spatial Reasoning by Aodi Wu, Xubo Luo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24152.txt.

Description

Aodi Wu and Xubo Luo describe their entry to the RoboSense Challenge at IROS 2025, which tests vision language models on driving scenes. The tasks cover seeing, predicting, planning, and spotting corrupted input. One general prompt for all tasks causes interference, since each task needs different reasoning and detail.

Their system routes each question to a task specific prompt through a mixture of prompts classifier. Each prompt carries its own coordinate frame, spatial rules, and examples, with step by step reasoning where needed. A visual module also assembles multi view images, object crops, colored markers, and past frames as the question requires. The report details the full setup for the challenge.

Catalog record

CREATORAodi Wu, Xubo Luo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-24152

Browse the full library · About the Tower of Babel