Exploring the limits of strong membership inference attacks on large language models

Jamie Hayes Ilia Shumailov Christopher A. Choquette-Choo, Matthew Jagielski Georgios Kaissis Milad Nasr Sahra Ghalebikesabi, Meenatchi Sundaram Muthu Selva Annamalai Niloofar Mireshghallah Igor Shilov, Matthieu Meeus Yves-Alexandre de Montjoye, Katherine Lee, Franziska Boenisch Adam Dziedzic A. Fede · 2025

Read Exploring the limits of strong membership inference attacks on large language models by Jamie Hayes Ilia Shumailov Christopher A. Choquette-Choo, Matthew Jagielski Georgios Kaissis Milad Nasr Sahra Ghalebikesabi, Meenatchi Sundaram Muthu Selva Annamalai Niloofar Mireshghallah Igor Shilov, Matthieu Meeus Yves-Alexandre de Montjoye, Katherine Lee, Franziska Boenisch Adam Dziedzic A. Fede (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18773.txt.

Introduction

Jamie Hayes and co-authors test how well strong attacks can tell if a text was in a language model's training data. Their 2025 paper deals with membership inference, a privacy risk for models trained on huge text sets. Strong attacks were thought too costly to run on large models.

The team scaled a leading attack to GPT-2 models of varied size, training many reference models on a large public corpus. Strong attacks could work on pre-trained models, the authors find, but their success stayed limited in practice. Scores stayed below a modest ceiling that the paper reports. The result matters for privacy claims: risk is real, yet weaker tests in past work may have misled readers about how large the risk is.

Catalog record

CREATORJamie Hayes Ilia Shumailov Christopher A. Choquette-Choo, Matthew Jagielski Georgios Kaissis Milad Nasr Sahra Ghalebikesabi, Meenatchi Sundaram Muthu Selva Annamalai Niloofar Mireshghallah Igor Shilov, Matthieu Meeus Yves-Alexandre de Montjoye, Katherine Lee, Franziska Boenisch Adam Dziedzic A. Fede
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE196 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-18773

Browse the full library · About the Tower of Babel