Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing
Jinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University · 2025
Read Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing by Jinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01952.txt.
Description
Jinhua Yin and seven colleagues devise the first purely black-box membership inference attack on large vision-language models. Such attacks test whether a specific item was in a model's training data, a live privacy question for models trained on vast scraped corpora. Earlier attacks needed internal likelihoods, which mainstream models do not expose; only generated text comes out.
Their method probes the model's memory of private semantic details embedded in a suspect image, details unlikely to be inferable from general world knowledge, and calibrates the probe with prior knowledge. Across four vision-language models and three datasets, the attack identified training data effectively using outputs alone, performing comparably to gray-box and white-box methods that peek inside. Further analysis probes its robustness against adversarial manipulation by a defensive model owner.
Catalog record
| CREATOR | Jinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01952