NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE
Xiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay · 2026
Read NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving PubID: pubid: 0000–0000/00$00.00 © 2021 IEEE by Xiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13293.txt.
Description
Vision-language models now help cars perceive the road, but they can be fooled by small physical patches or by invisible tweaks spread across a whole image. NutVLM defends the full perception-to-decision chain with a self-adaptive design. A sentinel called NutNet++ first sorts inputs three ways, into benign samples, local patches, and global perturbations, then local threats are cleaned by grayscale masking while global ones trigger expert-guided prompt tuning.
That tuning builds corrective driving prompts through gradient-based optimization, refocusing the model's attention without retraining the whole model. On the Dolphins driving benchmark, the framework improved overall metrics by 4.89 percent, and its code is public. Xiaoxu Peng leads the work with five co-authors.
Catalog record
| CREATOR | Xiaoxu Peng, Dong Zhou, Jianwen Zhang, Guanghui Sun, Anh Tu Ngo, Anupam Chattopadhyay |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13293