Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing

Jinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University · 2025

Read Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing by Jinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01952.txt.

Description

Jinhua Yin and seven colleagues devise the first purely black-box membership inference attack on large vision-language models. Such attacks test whether a specific item was in a model's training data, a live privacy question for models trained on vast scraped corpora. Earlier attacks needed internal likelihoods, which mainstream models do not expose; only generated text comes out.

Their method probes the model's memory of private semantic details embedded in a suspect image, details unlikely to be inferable from general world knowledge, and calibrates the probe with prior knowledge. Across four vision-language models and three datasets, the attack identified training data effectively using outputs alone, performing comparably to gray-box and white-box methods that peek inside. Further analysis probes its robustness against adversarial manipulation by a defensive model owner.

Catalog record

CREATORJinhua Yin, Peiru Yang , Chen Yang, Huili Wang, Zhiyang Hu, Shangguang Wang, Yongfeng Huang, Tao Qi, [10pt] Department of Electronic Engineering, Tsinghua University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-01952

Browse the full library · About the Tower of Babel