VisualLeakBench : Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering
Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao · 2026
Read VisualLeakBench : Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering by Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13385.txt.
Description
Youting Wang and co-authors introduce VisualLeakBench to audit vision-language models for privacy leaks. Alignment tests stress explicit harmful content, yet privacy-critical multimodal cases stay under-tested. Their suite uses 1,000 synthetic adversarial images with 8 PII types, plus 50 real screenshots. Four frontier systems were tested with OCR injection and contextual PII leakage attacks. One model had the lowest OCR attack rate but the highest PII leak rate, with disclosure before any warning. A defensive prompt cut leaks sharply for some models but not for others.
Catalog record
| CREATOR | Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32757 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.13385