VisualLeakBench : Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering

Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao · 2026

Read VisualLeakBench : Auditing the Fragility of Large Vision-Language Models against PII Leakage and Social Engineering by Youting Wang, Yuan Tang, Yitian Qian, Chen Zhao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13385.txt.

Description

Youting Wang and co-authors introduce VisualLeakBench to audit vision-language models for privacy leaks. Alignment tests stress explicit harmful content, yet privacy-critical multimodal cases stay under-tested. Their suite uses 1,000 synthetic adversarial images with 8 PII types, plus 50 real screenshots. Four frontier systems were tested with OCR injection and contextual PII leakage attacks. One model had the lowest OCR attack rate but the highest PII leak rate, with disclosure before any warning. A defensive prompt cut leaks sharply for some models but not for others.

Catalog record

CREATORYouting Wang, Yuan Tang, Yitian Qian, Chen Zhao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32757
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13385

Browse the full library · About the Tower of Babel