On Reliability of Efficient Membership Inference Vulnerability Evaluation
Joonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela · 2026
Read On Reliability of Efficient Membership Inference Vulnerability Evaluation by Joonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25819.txt.
Description
Jälkö, Pradhan, Räisä and Honkela examine whether fast membership inference vulnerability review can be trusted. Vulnerability is usually judged by a model guessing whether a sample was in training data, using shadow models on random splits. Aggregates hide sample-level problems. Two weaknesses emerge: sampling shadow datasets from a fixed superset inflates estimates, and concatenating scores across individuals skews true hit rates at very low false alarm rates. For both they propose fixes, the simplest adding no computation, with costlier options improving reliability further.
Catalog record
| CREATOR | Joonas Jälkö, Gauri Pradhan, Ossi Räisä, Antti Honkela |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39005 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.25819