Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis
Ruijun Deng, Zhihui Lu, Qiang Duan · 2025
Read Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis by Ruijun Deng, Zhihui Lu, Qiang Duan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-10016.txt.
Introduction
Ruijun Deng and co-authors measure privacy leaks in split AI systems in a 2025 paper. Split inference cuts a neural network across two machines. A phone or clinic runs the first part, a server runs the rest. Attackers can use the shared middle data to rebuild private inputs.
The team builds a math frame that defines leakage in terms of attacker certainty. From it they derive error bounds for average and worst cases. A new metric uses Fisher information to make the leak easy to compute. A defense built on that metric guards the shared data. Tests across data sets show the metric tracks real risk well. The guarded system kept accuracy high while cutting leaks hard.
Catalog record
| CREATOR | Ruijun Deng, Zhihui Lu, Qiang Duan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-10016
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis Ruijun Deng, Zhihui Lu, Qiang Duan Abstract Split inference (SI) partitions deep neural networks into distributed sub-models, enabling privacy-preserving collaborative learning. Nevertheless, it remains vulnerable to Data Reconstruction Attacks (DRAs), wherein adversaries exploit exposed smashed data to reconstruct raw inputs. Despite extensive research on adversarial attack-defense games, a shortfall remains in the fundamental analysis of privacy risks. This paper establishes a theoretical framework for privacy leakage quantification using information theory, defining it as the adversary’s certainty and deriving both average-case and worst-case error bounds. We introduce Fisher-approximated Shannon information (FSInfo), a novel privacy metric utilizing Fisher Information (FI) for operational privacy leakage computation. We empirically show that our privacy metric correlates well with empirical attacks and investigate some of the factors that affect privacy leakage, namely the data distribution, model size, and overfitting. 1 INTRODUCTION Machine learning as a service (MLaaS) allows users to access machine learning (ML) services by simply uploading the raw data to be inferred. To enable privacy-preserving MLaaS, model partition, a mainstream research line of distributed machine learning, has been widely employed to segment deep neural networks (DNNs) into multiple sub-models and deploy them on different hosts (e.g., mobile phones and edge servers). These hosts conduct collaborative learning by communicating the data derived from the split layer (smashed data), thus hopefully preserving raw data privacy. This learning paradigm has been adopted in numerous model-distributed co-training and co-inference; for example, split/collaborative inference (SI) ( Zhang et al. 2021 ; Li et al. 2021 ) , split learning (SL) ( Deng et al. 2023 ; Yu et al. 2024 ; Oh et al. 2022 ) , vertical federated learning (VFL) ( Chen et al. 2023 ) , pipeline parallelism ( Yuan et al. 2023 ; Ye et al. 2024 ) , and their combinations ( Duan et al. 2022 ) . W.l.o.g, we focus on a two-party split inference system (see Figure 1 ) where the inputs are first fed into the bottom model on the client, and then the intermediate smashed data are transmitted to the top model on the server for further prediction, following Maeng et. al. ( Maeng et al. 2023 ) and Singh et. al. ( Singh et al. 2024 ) . It is worth noting that this system represents each client-server pair in a multi-party distributed framework (e.g., VFL) and also corresponds to the forward computation in co-training (e.g., SL). Figure 1: The information flow in a split inference system and the data reconstruction attack. Despite transmitting only smashed data rather than raw inputs, the SI paradigm remains susceptible to privacy inference attacks. Adversaries can execute Membership Inference Attacks (MIAs) to identify training data participation ( Song & Mittal 2021 ; Wu et al. 2024 ) , Attribute Inference Attacks (AIAs) to deduce sensitive attributes (e.g., demographic information) ( Liu et al. 2022 ) , and Data Reconstruction Attacks (DRAs) to recover original user data ( He et al. 2019 ; Yang et al. 2022 ; Pasquini et al. 2021 ) . Our research focuses on DRAs due to their heightened severity: successful execution compromises entire datasets rather than isolated attributes, potentially causing substantial damage. As illustrated in Figure 1 , the adversary (malicious server) can reconstruct input images after seeing the smashed data, demonstrating the critical nature of this vulnerability. Privacy leakage quantification (or privacy quantification for short), is an independent concept that transcends the dualistic framework of privacy attack and defense. This critical but less investigated aspect of privacy issues on ML is to develop an appropriate measure (i.e., a privacy metric) of private information leakage from the exposed data (e.g., model parameters, gradients, and smashed data) about the raw input data. In this paper, privacy quantification means assessing the private information exposure from smashed data under DRAs in the SI system of Figure 1 . The challenges for this can be presented through the lens of the twofold requirements: i) the privacy metric should be theoretically principled and possess a semantic interpretation aligned with privacy and security settings; and ii) the privacy metric should be quantitative and operationally computable, not just theoretical guidance. Privacy quantification methods primarily fall into two categories: attack-based and theory-based approaches. Attack-based methods ( Wu et al. 2024 ; Hu et al. 2023 ; Deng et al. 2023 ) measure privacy leakage through inference attacks, using success rates as empirical metrics. While operationally feasible, these methods rely heavily on adversary capabilities and provide only theoretical lower bounds without guarantees. Theory-based approaches employ various frameworks, including differential privacy (DP) ( Singh et al. 2023 ) , information theory (mutual information; MI) ( Arevalo et al. 2024 ; Noorbakhsh et al. 2024 ) , and hypothesis testing (Fisher information; FI) ( Maeng et al. 2023 ) . However, DP faces compatibility issues with nonlinear transformations in split inference, while MI often proves computationally intensive. Although FI offers theoretical rigor and computational efficiency, it focuses solely on mean squared error (MSE), limiting its privacy assessment scope. Consequently, existing methods exhibit inherent limitations in addressing these dual challenges simultaneously. Contributions. In this paper, we tackle the two challenges by answering the following three questions: i) how to formulate privacy leakage theoretically; ii) how to easily compute the privacy leakage; and iii) what factors may potentially impact privacy leakage. For the first question, we formulate the privacy leakage as the certainty/confidence of the adversary with conditional entropy. Then, for the second question, we employ FI as a tool to directly calculate an approximation of defined privacy leakage, offering a quantitative and operational privacy metric called Fisher-approximated Shannon information (FSInfo) that also considers the adversary’s ability to exploit information. Finally, with FSInfo, we experimentally analyze the determinants of privacy leakage, including data distribution, model width and depth, and overfitting. 2 RELATED WORK Table 1: Comparisons of the privacy metrics. Metrics Bound Comp. Adv. Fea. Adv. Err. Attack ✓ ✓ ✓ Dcor ✓ MI ✓ ✓ FI ✓ ✓ ✓ FSInfo ✓ ✓ ✓ ✓ The privacy metrics ( Wagner & Eckhoff 2018 ) or privacy quantification, although having been studied decades ago in the areas of smart meter ( Kalogridis et al. 2010 ) and database ( Agrawal & Aggarwal 2001 ) , does not receive much attention in the field of machine learning until recently ( Deng et al. 2024 ) . Current privacy metrics primarily emerge as byproducts of privacy attacks, demonstrating that attacks developed for breaching privacy can also be applied to quantify it ( Wu et al. 2024 ; Hu et al. 2023 ; Deng et al. 2023 ; Abuadbba et al. 2020 ; Pasquini et al. 2021 ; Liu et al. 2022 ) . For instance, Liu et al. ( Liu et al. 2022 ) developed the ML-Doctor framework, employing four distinct privacy inference attacks to assess private data and model parameter vulnerabilities. However, these empirical approaches, while intuitive, lack theoretical foundations and exhibit adversary-dependent limitations, potentially yielding overly optimistic privacy assessments with weaker adversaries ( Deng et al. 2024 ) . Theoretical approaches offer deeper privacy analysis through advanced mathematical tools. Pan et al. ( Pan et al. 2022 ) quantify privacy leakage via activated