Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models

Zeyuan Chen Yihan Ma Xinyue Shen Michael Backes Yang Zhang · 2026

Read Pop Quiz Attack: Black-box Membership Inference Attacks Against Large Language Models by Zeyuan Chen Yihan Ma Xinyue Shen Michael Backes Yang Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06423.txt.

Description

Zeyuan Chen and four co-authors introduce Pop Quiz Attack, a black-box way to test whether a large language model memorized a given training example. Membership inference matters because models can reveal private training data. Their idea turns target data into quiz-style multiple-choice questions and reads membership from the model's answers.

Across six widely used models and four datasets, the attack reaches an average ROC-AUC score well above prior methods. The authors also study what drives success, including question complexity and data type. Tested defenses, from filters to differential privacy, reduce the attack but do not remove the privacy risk.

Catalog record

CREATORZeyuan Chen Yihan Ma Xinyue Shen Michael Backes Yang Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06423

Browse the full library · About the Tower of Babel