On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction

Fatima Ezzeddine, Rinad Akel, Ihab Sbeity, Silvia Giordano, Marc Langheinrich, Omran Ayoub · 2025

Read On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction by Fatima Ezzeddine, Rinad Akel, Ihab Sbeity, Silvia Giordano, Marc Langheinrich, Omran Ayoub (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08847.txt.

Introduction

Fatima Ezzeddine and co-authors weigh a three-way trade-off in cloud machine learning in two thousand twenty-five. Firms rent ready models as a service, which saves cost but opens a theft risk: rivals can copy the model by probing it. Added explanations, meant to build trust, can leak clues that ease the theft, above all counterfactual ones.

The team tests differential privacy, a noise-adding shield, as a counter. Two spots for the noise are compared: inside model training, and at the explainer when it crafts the counterfactuals. Scores track how prediction quality, privacy, and clarity of the explanations fare under each choice. The paper maps where the shield helps most.

Catalog record

CREATORFatima Ezzeddine, Rinad Akel, Ihab Sbeity, Silvia Giordano, Marc Langheinrich, Omran Ayoub
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08847

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the interplay of Explainability, Privacy and Predictive Performance with Explanation-assisted Model Extraction Fatima Ezzeddine, Rinad Akel, Ihab Sbeity, Silvia Giordano, Marc Langheinrich, Omran Ayoub Abstract Machine Learning as a Service (MLaaS) has gained important attraction as a means for deploying powerful predictive models, offering ease of use that enables organizations to leverage advanced analytics without substantial investments in specialized infrastructure or expertise. However, MLaaS platforms must be safeguarded against security and privacy attacks, such as model extraction (MEA) attacks. The increasing integration of explainable AI (XAI) within MLaaS has introduced an additional privacy challenge, as attackers can exploit model explanations—particularly counterfactual explanations (CFs) to facilitate MEA. In this paper, we investigate the trade-offs among model performance, privacy, and explainability when employing Differential Privacy (DP), a promising technique for mitigating CF-facilitated MEA. We evaluate two distinct DP strategies: implemented during the classification model training and at the explainer during CF generation. *1 1 1 keywords Counterfactual Explanations ,Model Extraction Attack ,Differential Privacy † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: Late-breaking work, Demos and Doctoral Consortium, colocated with the 3rd World Conference on eXplainable Artificial Intelligence: July 09–11, 2025, Istanbul, Turkey † † email: [email protected] † † address: Università della Svizzera italiana, Lugano, Switzerland † † address: University of Applied Sciences and Arts of Southern Switzerland, Lugano, Switzerland † † address: Lebanese University, Beirut, Lebanon † † corresp: Corresponding author. † † These authors contributed equally. 1 Introduction Machine Learning (ML) as a Service (MLaaS) is becoming increasingly popular for deploying powerful predictive models as it facilitates access to ML training and deployment tools, while eliminating the need for extensive computational resources Tramèr et al. (2016) . The adoption of MLaaS, however, introduces important security and privacy risks. For instance, adversaries can query the deployed ML models through application programming interfaces (APIs) to perform various types of attacks, such as membership inference (MIA) Shokri et al. (2017) and model extraction (MEA) Tramèr et al. (2016) . These attacks, if successful, pose serious threats to data privacy and intellectual property. For instance, MIA can reveal whether specific data points were used in training, MEA, instead, enables adversaries to replicate proprietary models, leading to financial losses and competitive disadvantages and facilitates other data privacy attacks by having access to a copy of the model. To defend against these attacks, data privacy-enhancing technologies such as Differential Privacy (DP) Dwork (2006) exist. DP has shown effectiveness in defending against such attacks and is therefore widely adopted in use cases that require data and model sharing and deployments Abadi et al. (2016) . DP enables privacy-preserving training of deep neural networks (DNN) to effectively mitigate inferential attacks by adding a controlled amount of noise to either raw data or model weights and ensures that individual data points have minimal influence on the model’s response, which limits the amount of sensitive information leaked when an attacker queries the model. Recently, with the increasing demand for transparency in automated decision-making, MLaaS platforms are starting to incorporate Explainable Artificial Intelligence (XAI) Guidotti et al. (2018) techniques into their workflows to provide explanations of the model’s decisions 1 1 1 https://aws.amazon.com/sagemaker/clarify/, https://cloud.google.com/explainable-ai Ezzeddine (2024) ; Shokri et al. (2021) . These platforms now provide not only the final decisions of ML models but also explanations of the underlying processes. The increased transparency provided by XAI introduces new challenges for preserving privacy and safeguarding MLaaS platforms from adversarial threats, as model’s explanations can inadvertently reveal information about model’s decision boundaries Spartalis et al. (2023) . Specifically, counterfactual explanations (CFs) Wachter et al. (2017) , which aim to identify the smallest changes to input data that would alter an ML model’s prediction to a desired outcome, can reveal the factors most influential in the model’s decision-making. Recent research has indeed explored how explanations can be leveraged to enhance the effectiveness of such attacks Spartalis et al. (2023) ; Shokri et al. (2021) ; Ezzeddine et al. (2024) ; Ezzeddine (2024) ; Aïvodji et al. (2020) . Complementing this, DP can also be applied at the explanation level, where it masks explanations to limit their utility to adversaries while balancing interpretability and privacy Shokri et al. (2021) ; Ezzeddine et al. (2024) . As DP can have impact on predictive performance and explanation quality, and can be applied on both levels, there is a growing research to highlight the importance of DP in developing mitigation strategies that specifically address risks introduced by explanations, emphasizing the need to adapt, utilize, or extend existing defense methods to the exploitation of explainability. In this work, we focus on analyzing the mitigation framework that integrates DP at the model and at the explainer, and investigate the interplay between i) model’s accuracy , as DP is expected to influence model’s inference capability, ii) privacy , as employing DP provides resilience against attacks, and iii) explainability , as noise added to model or explainer may impact quality of explanations Ezzeddine (2024) ; Abbasi et al. (2024) . We aim to quantify this interplay and extract insights on the choice of where to employ DP (at model or at explainer, or at both) and the degree of noise level to be employed to balance predictive performance, explainability and privacy. To perform the attack, we employ a recently proposed MEA technique based on Knowledge Distillation (KD) due to its proven performance and practicality Ezzeddine et al. (2024) . In terms of mitigation strategies, we employ DP at the ML model using Differential Private-SGD (DP-SGD) and at the explainer using a DP-based Generative Adversarial Network (GAN) Ezzeddine et al. (2024) with varying noise levels. To this end, we investigate the following research questions (RQs): • RQ1: To what extent does applying DP at the model or at the explainer, or both, effectively mitigates MEA facilitated by CFs? • RQ2: How does noise level in DP influence the effectiveness of MEAs that leverage CFs? • RQ3: In what ways does the quality of CF explanations differ when DP is applied at the model compared to the explainer? 2 Related Work Several studies have explores leveraging XAI techniques and exploiting model explanations to perform privacy attacks. In Oksuz et al. (2024) , authors explore the vulnerabilities of Local Interpretable Model-agnostic Explanations and show that an adversary can generate new data samples near the decision boundary and, consequently, perform MEA by crafting adaptive queries. In Shokri et al. (2021) , authors show that by leveraging gradient-based explanations, adversaries can enhance the effectiveness of MIA. In Yan et al. (2022) , the authors propose a methodology that performs MEA by jointly minimizing classification and explanation loss, thereby improving its fidelity. Other works explore the use of CFs to enhance the effectiveness of MEA. For instance, Aïvodji et al. (2020) introduces a methodology that relies on model predictions and CFs to train a substitute model. Similarly, Wang et al. (2022) presents a novel strategy where CF pairs, including the CF of the CF, serve as training samples to MEA. More recently, Ezzeddine et al. (2024) proposes a methodology based on KD techniques that exploit CFs

Read the complete text