On Reliability of Efficient Membership Inference Vulnerability Evaluation

Joonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela · 2026

Read On Reliability of Efficient Membership Inference Vulnerability Evaluation by Joonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25819.txt.

Description

Jälkö, Pradhan, Räisä and Honkela examine whether fast membership inference vulnerability review can be trusted. Vulnerability is usually judged by a model guessing whether a sample was in training data, using shadow models on random splits. Aggregates hide sample-level problems. Two weaknesses emerge: sampling shadow datasets from a fixed superset inflates estimates, and concatenating scores across individuals skews true hit rates at very low false alarm rates. For both they propose fixes, the simplest adding no computation, with costlier options improving reliability further.

Catalog record

CREATORJoonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39005
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25819

Browse the full library · About the Tower of Babel