Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge
Gustavo de Carvalho Bertoli · 2026
Read Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge by Gustavo de Carvalho Bertoli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12737.txt.
Introduction
Gustavo de Carvalho Bertoli tests differential privacy as a shield for federated learning models. Federated models hide raw data yet leak membership through their outputs. The study uses the 2025 NIST genomics red team event as test bed. A stacking attack ensembles seven black-box estimators into a meta-model. It trains on prediction scores and losses. Three tiers are tested: no protection, low privacy, and high privacy. The attack beats all baselines without protection and at low privacy. Leakage persists even at that tier where a single-signal baseline collapses.
Catalog record
| CREATOR | Gustavo de Carvalho Bertoli |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45499 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12737
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge Gustavo de Carvalho Bertoli Abstract While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs). This paper presents an empirical evaluation of Differential Privacy (DP) as a defense mechanism against MIAs in FL, leveraging the environment of the 2025 NIST Genomics Privacy-Preserving Federated Learning (PPFL) Red Teaming Event. To improve inference accuracy, we propose a stacking attack strategy that ensembles seven black-box estimators to train a meta-classifier on prediction probabilities and cross-entropy losses. We evaluate this methodology against target models under three privacy configurations: an unprotected convolutional neural network (CNN, ϵ = ∞ \epsilon=\infty ), a low-privacy DP model ( ϵ = 200 \epsilon=200 ), and a high-privacy DP model ( ϵ = 10 \epsilon=10 ). The attack outperforms all baselines in the No DP and Low Privacy settings and, critically, maintains measurable membership leakage at ϵ = 200 \epsilon=200 where a single-signal LiRA baseline collapses. Evaluated on an independent third-party benchmark, these results provide an empirical characterisation of how stacking-based inference degrades across calibrated DP tiers in FL. 1 Introduction Federated Learning (FL) is a privacy-enhancing technology that enables machine learning (ML) tasks to be performed without exchanging raw data [ 11 ] . In addition, FL enables collaboration across privacy-sensitive domains such as healthcare, security, and banking, where data sharing directly is legally or ethically constrained [ 2 , 17 , 4 , 20 ] . Despite these benefits, FL introduces its own security and privacy risks, including malicious servers, poisoning attacks, backdoors, and membership inference attacks (MIAs) [ 13 ] . MIAs are risky in sensitive domains. At the individual level, a successful MIA confirms whether a specific record was used for training — raising concerns for data subjects who did not consent to inclusion or who wish to verify compliance with data protection regulations. At the adversarial level, knowledge of training membership serves as reconnaissance: an attacker who knows which records a model was trained on can craft more targeted evasion or extraction attacks [ 5 ] . More broadly, MIAs serve as an auditing tool — they provide an empirical means of verifying whether a privacy-preserving mechanism, such as differential privacy, bounds membership leakage in practice. To address privacy concerns, Privacy-Preserving Federated Learning (PPFL) extends FL with additional mechanisms to reduce what can be learned across all the FL participants or the final central model derived from them. A common approach is for FL clients to add calibrated noise to their local model updates before sharing them with the central server. This paper uses the NIST Genomics PPFL Red Teaming Event as a controlled environment to empirically assess how well Differential Privacy (DP) protects FL models against MIAs. The attack exploits the tendency of machine learning models to exhibit different behaviour on training versus non-training samples, particularly in terms of prediction confidence and loss. This effect is amplified in scenarios where models are over-fitted or trained on limited or heterogeneous data distributions. The methodology achieved the highest aggregate performance in the challenge’s membership inference track, ranking first in the No DP ( \epsilon=\infty ) and Low Privacy ( \epsilon=200 ) tiers. The contributions of this paper are threefold: (1) we propose a black-box, stacking-based MIA that replaces the canonical shadow-model requirement with auxiliary-record-driven meta-learning, making it compatible with FL threat models where private training data is inaccessible; (2) we provide an empirical characterisation of how this multi-signal attack degrades across three calibrated DP tiers, showing that residual leakage persists at \epsilon=200 even when single-signal baselines collapse; and (3) we evaluate on an independent, third-party genomic benchmark, establishing a reproducible baseline for future MIA research on this publicly available dataset. The remainder of this paper is organised as follows. Section 2 describes the problem setting. Section 3 presents the attack methodology. Section 4 reports the empirical results and further discussions including the evaluation metric and the trade-off between privacy and utility, followed by a discussion of limitations in Section 5 . Section 6 discusses related work. Section 7 concludes. 2 Problem Setting This work builds on the NIST Genomics Red Team Challenge 1 1 1 2025 Red Teaming Event: https://pages.nist.gov/genomics_ppfl/ , an initiative that evaluated the effectiveness of DP as a defence against two attack types in FL: membership inference and reconstruction attacks. For genomic data, the challenge presents a primary scenario on FL for predicting seed coat colour in soybean plants, with the attributes of the records being gene variants. A second scenario uses dog breed prediction as a proxy for human genomic data. This work focuses on the soybean dataset and membership inference. The NIST red teaming datasets are available in a public repository 2 2 2 NIST PPFL Dataset: https://github.com/usnistgov/genomics_ppfl . For this challenge, the soybean seed coat colour dataset features were reduced to the gene variants most relevant to the task, and the learning task is to classify records into one of four seed coat colour categories. The soybean seed coat dataset is split into five parts, four out of these five splits are used to train each of the four clients. The FL task in the challenge is restricted to 4 clients, the remaining subset represents records that are not used by any of the clients during its training. This data split as well as the attacker available data are presented by Figure 1 . Each client’s model is evaluated across three privacy configurations: No DP (standard CNN, \epsilon=\infty ), Low DP ( \epsilon=200 ), and High DP ( \epsilon=10 ). From the attacker’s perspective, three datasets are available in addition to the trained models: the relevant dataset represents complete records (with the correct target attribute) that might or might not be part of any of clients subsets, the external dataset represents complete records (with the correct target attribute) that is guaranteed not to be part of any of clients subsets, and the challenge dataset represents the blind evaluation set, for which the attacker must predict client membership. Figure 1 : Problem setting. Clients train local models on private genomic data and share them with a central aggregator. The adversary queries client models and uses auxiliary data (relevant, external, challenge) to execute MIA. 2.1 Threat Model The red-teaming challenge proposes the untrusted central aggregator threat model. In this setting, federated learning (FL) clients collaboratively train local models on private genomic data and periodically share their models with the central server responsible for aggregation. The adversary is the central server, which is assumed to be malicious. It has access to trained client models via black-box queries, i.e., the ability to submit arbitrary inputs and observe output predictions. Furthermore, as the central coordinator of the federated learning process, the adversary possesses full knowledge of the global training hyper-parameters, including the optimizer, learning rate, local epochs, and applied differential privacy constraints. The adversary also has auxiliary datasets, provided as part of the challenge, as presented by Figure 1 . This capability is realistic: similar auxiliary data could be purchased from data brokers, consistent with the challenge design. The attack is black-box, relying only on the trained models without