NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE

Xiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay · 2026

Read NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Xiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13293.txt.

Description

Vision-language models now help cars perceive the road, but they can be fooled by small physical patches or by invisible tweaks spread across a whole image. NutVLM defends the full perception-to-decision chain with a self-adaptive design. A sentinel called NutNet++ first sorts inputs three ways, into benign samples, local patches, and global perturbations, then local threats are cleaned by grayscale masking while global ones trigger expert-guided prompt tuning.

That tuning builds corrective driving prompts through gradient-based optimization, refocusing the model's attention without retraining the whole model. On the Dolphins driving benchmark, the framework improved overall metrics by 4.89 percent, and its code is public. Xiaoxu Peng leads the work with five co-authors.

Catalog record

CREATORXiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13293

Browse the full library · About the Tower of Babel