DeepLeak : Privacy Enhancing Hardening of Model Explanations Against Membership Leakage Firas Ben Hmida 1 , Zain Sbeih 2 , Philemon Hailemariam 3 , Birhanu Eshete 4 Abstract Machine learning (ML) explainability is central to algorithmic transparency in high-stakes settings such as predictive diagnostics and loan approval. However, these same domains require rigorous privacy guaranties, creating tension between interpretability and privacy. Although prior work has shown that explanation methods can leak membership information, practitioners still lack systematic guidance on selecting or deploying explanation techniques that balance transparency with privacy. We present DeepLeak , a system to audit and mitigate privacy risks in post-hoc explanation methods. DeepLeak advances the state-of-the-art in three ways: (1) comprehensive leakage profiling: we develop a stronger explanation-aware membership inference attack (MIA) to quantify how much representative explanation methods leak membership information under default configurations; (2) lightweight hardening strategies: we introduce practical, model-agnostic mitigations, including sensitivity-calibrated noise, attribution clipping, and masking, that substantially reduce membership leakage while preserving explanation utility; and (3) root-cause analysis: through controlled experiments, we pinpoint algorithmic properties (e.g., attribution sparsity and sensitivity) that drive leakage. Evaluating 15 explanation techniques across four families on image benchmarks, DeepLeak shows that default settings can leak up to 74.9% more membership information than previously reported. Our mitigations cut leakage by up to 95% (minimum 46.5%) with only ≤ \leq 3.3% utility loss on average. DeepLeak offers a systematic, reproducible path to safer explainability in privacy-sensitive ML. I Introduction Machine learning (ML) models are increasingly being deployed in high‑stakes settings, ranging from medical diagnostics [ 45 ] to credit scoring [ 4 ] and cybersecurity [ 11 ] , where decisions must be accurate and explainable to a human. Post hoc explanation methods, which generate human interpretable attributions of model inference, have thus become crucial in debugging a model, algorithmic recourse, detecting bias, satisfying regulatory requirements, and enabling expert oversight [ 28 , 21 ] . At the same time, these domains often handle sensitive personal data, introducing stringent privacy mandates (e.g., HIPAA, GDPR). However, recent work has shown that explanations can leak private training examples via membership inference attacks, compromising the privacy guarantees that practitioners seek to uphold [ 33 , 18 , 53 , 9 , 22 ] . Despite compelling evidence of membership information leakage from explanations, there is no unified methodology to (i) establish membership leakage benchmarks across diverse explanation methods, (ii) systematically reduce membership leakage without compromising explainability, or (iii) analyze why certain methods leak more than others. As a result, ML practitioners and domain experts lack clear guidance on which explanation methods to use and under what safeguards to balance explainability and privacy. In the absence of such guidance, privacy leaks may go undetected or explainability may be unnecessarily sacrificed, outcomes that are undesirable in mission-critical domains such as predictive diagnostics, financial forecasting, and autonomous driving. In a recent work, Liu et al. [ 18 ] demonstrated a novel explanation-guided membership inference attack based on perturbation of important features, which resulted in the highest membership leakage from model explanations. While their attack demonstrates that explanations can exacerbate membership leakage, even in black‑box settings, it has several key limitations. First, it focuses exclusively on attribution‑based explanation methods and evaluates only seven techniques (e.g., SmoothGrad, SHAP, LIME). Second, beyond a perturbation‑guided attack, the study does not propose mitigations or guidance on how to reduce membership leakage without sacrificing explainability. Third, although the paper correlates overfitting and explanation quality with membership leakage, it lacks analysis of the underlying causes of leakage across explanation families. Fourth, all experiments are conducted using default explanation configurations, without exploring the parameter settings practitioners often adjust (e.g., number of perturbation samples in LIME [ 28 ] or kernel width in SHAP [ 21 ] ), missing an opportunity to understand how tuning impacts the privacy–explainability trade‑off. Motivated by these gaps, in this paper we present DeepLeak , a practical system for profiling, diagnosing, and hardening post‑hoc explanation methods against membership leakage. DeepLeak advances the state-of-the-art [ 18 ] on three fronts: (i) systematic membership leakage profiling : We develop a more effective membership inference attack against a broad spectrum of explanation methods to establish a membership leakage benchmark under default configurations, which advances the state-of-the-art [ 18 ] by a large margin; (ii) privacy enhancing hardening : We design lightweight hardening, such as calibrated noise injection and gradient clipping, which are easily integrated into existing explanation pipelines; and (iii) root-cause analysis : Through controlled ablation studies, we correlate key algorithmic parameters with observed membership leakage rates, revealing the mechanisms by which explanations expose membership information. We validate DeepLeak on 15 explanation methods that span four families on models trained on image classification datasets: CIFAR‑10 [ 15 ] , CIFAR-100 [ 15 ] , and GTSRB [ 42 ] . Our evaluation provides a comprehensive picture of privacy risks in ML explainability. First, we show that widely used explanation methods, under default configurations, can leak up to 74.9% more membership information than previously reported [ 18 ] . Second, our improved explanation-only membership inference attack achieves state-of-the-art accuracy without relying on predicted labels or confidence scores, underscoring the severity of explanation-driven leakage. Third, we demonstrate that lightweight, model-agnostic defenses such as attribution clipping, sensitivity-calibrated noise injection, and masking achieve 46.5–95% membership leakage reduction with a negligible \leq 3.3% drop in explanation utility, and combining these defenses yields the strongest privacy–utility trade-offs. Finally, our root-cause analysis attributes leakage to factors such as attribution sparsity, gradient outliers, and sensitivity to input perturbations, offering actionable guidance for privacy enhancing deployment of explanation methods. This paper makes the following key contributions: • DeepLeak approach : A unified toolkit for membership leakage auditing and hardening of post‑hoc explanations. • Comprehensive benchmark : An extensive membership leakage profile of representative explanation methods across four families, establishing a new baseline for membership leakage of ML explanation methods. • Practical mitigations : Simple, effective mitigation techniques that reduce privacy risks by orders of magnitude while retaining explanation utility. • Mechanistic insights : A root‑cause analysis that pinpoints which algorithmic features drive membership leakage, informing principled defense design. • Reproducible open-source code : To foster future research at the intersection of ML privacy and explainability, we release reproducible DeepLeak code at: https://https://github.com/um-dsp/DeepLeak . II Background We review ML explanation methods, whose outputs serve as the attack surface, and membership inference attacks, which we use as proxy for auditing membership leakage. II-A ML Explanation Methods Broadly, explanation methods fall into two complementary categories: inherent (ante-hoc) and post-hoc approaches. An inherently interpretable model satisfies transparency of structure with a fairly simple decision function (e.g., linear, rule-based) and feature-level interpretability whereby the influence of each feature on the output is explicitly accessible. Although inherently interpretable models are favored in regulated or safety-critical settings, they may sacrifice accuracy or expressiveness on complex tasks where high-capacity models such as deep neural networks (DNNs) excel. Post-hoc explanation methods generate explanations by quantifying the contributions of individual input features to the output of a model [ 44 , 32 ] . In this work, we consider post‑hoc explanation methods E(f,x) that produce attributions for f(x) . Let f:\mathbb{R}^{d}\rightarrow\mathbb{R}^{C} be a model that maps input x\in\mathbb{R}^{d} to a vector of class scores C , f_{c}(x) denote the scalar output (e.g., logit or probability) for class c , \frac{\partial f_{c}(x)}{\partial x} denote the gradient of the output with respect to the input, x^{\prime} be a baseline input (e.g., zero vector), and \odot denote the element-wise product. In this work, we categorize them into four families and provide formal definitions. Gradient-based explanation methods compute gradients of the model’s output with respect to input features to determine feature importance. Saliency Map score [ 37 ] is computed as \text{Saliency}(x)=\left|\frac{\partial f_{c}(x)}{\partial x}\right| , where \frac{\partial f_{c}(x)}{\partial x} measures sensitivity of the model output to changes in each input feature and |.| captures element-wise absolute value to measure feature importance. Deconvolution score [ 23 ] computed as \text{Deconv}(x)=\left.\frac{\partial f_{c}(x)}{\partial x}\right|_{\text{deconv rules}} is similar to Saliency map but modifies the backward pass through ReLU layers and passes gradients only for activations that were positive in the forward pass. Input×Gradient [ 36 ] computed as \text{InputGrad}(x)=x\odot\frac{\partial f_{c}(x)}{\partial x} highlights input dimensions that are large and whose gradients strongly influence the output. SmoothGrad [ 38 ] computed as \text{SmoothGrad}(x)=\frac{1}{n}\sum_{i=1}^{n}\left|\frac{\partial f_{c}(x+\mathcal{N}_{i})}{\partial x}\right| adds Gaussian noise \mathcal{N}_{i}\sim\mathcal{N}(0,\sigma^{2}) to the input and averages the resulting gradients. It reduces visual noise and sharpens the saliency map. VarGrad [ 2 ] is a variance-based extension of SmoothGrad that measures the variability of explanation scores under noise perturbations. It is defined as E_{\text{vg}}(x)=\mathcal{V}\big(E(x+g_{i})\big) . The noise vectors \mathbf{g}_{i} are sampled independently from a normal distribution \mathcal{N}(0,\sigma^{2}) and the variance \mathcal{V} quantifies how sensitive the explanations are to these perturbations. Integrated Gradients [ 44 ] computed as \text{IG}(x)=(x-x^{\prime})\odot\int_{\alpha=0}^{1}\frac{\partial f_{c}(x^{\prime}+\alpha(x-x^{\prime}))}{\partial x}\,d\alpha attribute prediction differences between x and a baseline x^{\prime} along a straight path and it does so by aggregating gradients along the path from x^{\prime} to x . DeepLIFT [ 35 ] calculated as \text{DeepLIFT}(x)=\Delta x\odot\frac{\Delta f_{c}}{\Delta x} uses differences from a reference input x^{\prime} instead of gradients, \Delta x=x-x^{\prime} , and \Delta f_{c} is the difference in output. DeepLIFT avoids gradient saturation by assigning contribution scores. Guided Backpropagation [ 41 ] computed as \text{GuidedBP}(x)=\left.\frac{\partial f_{c}(x)}{\partial x}\right|_{\text{guided rules}} combines saliency and deconvolution. Backpropagation through ReLU passes gradient only if both the forward activation and the backward gradient are positive. Perturbation-based explanation methods modify input features and observe the corresponding changes in model predictions to assess feature relevance. Let x_{i} be the input where the i^{th} feature is occluded (e.g., set to zero or blurred) and S\subseteq\{1,\dots,d\} be a subset of input features. Occlusion Sensitivity [ 46 ] calculated as \text{Occlusion}_{i}(x)=f_{c}(x)-f_{c}(x_{i}) measures the change in prediction when feature i is occluded where x_{i} is input with feature i removed or replaced (e.g., with baseline value). A large drop in f_{c} implies feature i is important. SHAP (SHapley Additive exPlanations) [ 21 ] uses game-theoretic perturbations to compute feature importance as \text{SHAP}_{i}(x)=\sum_{S\subseteq\{1,\dots,d\}\setminus\{i\}}\frac{|S|!(d-|S|-1)!}{d!}\left[f_{c}(x_{S\cup\{i\}})-f_{c}(x_{S})\right] , where x_{S} is input where only features in S are present (others set to a baseline). It fairly distributes the total prediction among input features using Shapley values from cooperative game theory and it also accounts for feature interactions by averaging marginal contributions over all possible subsets. Anchors [ 29 ] is based on the idea of an Anchor , a set of input conditions (feature values) that, when fixed, are sufficient to keep the model’s prediction stable. It is computed as \text{Anchor}(x)=\text{Minimal feature set }A\subseteq\{1,\dots,d\}\text{ s.t. }P(f(x^{\prime})=f(x)\mid x^{\prime}_{A}=x_{A})\geq\tau , where x^{\prime}_{A}=x_{A} is perturbed inputs that preserve the anchor conditions, \tau is a precision threshold. It provides human-interpretable if-then rules explaining why the model predicted class c . Representation-guided explanation methods use intermediate feature representations in DNNs to provide saliency maps. Gradient-weighted Class Activation Mapping (Grad-CAM) [ 32 ] produces coarse localization maps that highlight important regions in a convolutional layer for a given class. Let A^{k}\in\mathbb{R}^{H\times W} denote the k^{th} feature map (activation) in a convolutional layer for input x and s_{y}(x)=\log f_{y}(x) be the score (e.g., logit) for class y , Grad-CAM computes the importance weight \alpha_{k}^{y} for each channel k as the spatially averaged gradient of the class score with respect to the feature map: \alpha_{k}^{y}=\frac{1}{H\times W}\sum_{i=1}^{H}\sum_{j=1}^{W}\frac{\partial s_{y}(x)}{\partial A_{i,j}^{k}}. The final class-discriminative localization map is computed as: \phi^{\text{Grad-CAM}}(x)=\mathrm{ReLU}\left(\sum_{k}\alpha_{k}^{y}A^{k}\right), where ReLU ensures that only the features positively influencing the class are visualized. Grad-CAM++ [ 6 ] extends Grad-CAM by providing better localization and handling multiple object instances. It improves the weighting scheme by incorporating higher-order partial derivatives to compute more accurate weights for each feature in the feature maps. Approximation-based explanation methods fit interpretable surrogate models (e.g., decision trees, linear regression models) to approximate local decision boundaries. LIME [ 28 ] explanation is obtained by solving the optimization problem: \phi^{\text{LIME}}(x)=\arg\min_{g\in\mathcal{G}}\mathcal{L}(f,g,\pi_{x})+\Omega(g), where: \mathcal{L}(f,g,\pi_{x}) is a loss function that measures the fidelity of g in approximating f in the local neighborhood of x , weighted by a locality kernel \pi_{x} , \pi_{x}(z) assigns higher weights to perturbed samples z that are closer to x , and \Omega(g) is a regularization term that penalizes the complexity of the interpretable model g . In practice, LIME perturbs the input instance x to generate a dataset \{z_{i}\} , obtains predictions f(z_{i}) , and fits g to minimize: \mathcal{L}(f,g,\pi_{x})=\sum_{i}\pi_{x}(z_{i})(f(z_{i})-g(z_{i}))^{2} . The explanation \phi^{\text{LIME}}(x) is then the feature attribution vector from the fitted surrogate g . SHAP [ 21 ] also approximates the Shapley values by averaging marginal contributions over all possible subsets of features. ProtoDash [ 12 ] is a prototype-based method that selects a small set of representative examples (prototypes) from a dataset \mathcal{Z}=\{z_{1},z_{2},\ldots,z_{n}\} to best summarize a target distribution (e.g., instances with the same prediction as x ). The explanation is a sparse set of prototypes with associated nonnegative weights indicating their relevance to the input. Let \mu_{p} be the mean embedding of a target sample distribution p (e.g., all instances predicted the same as x ) in a reproducing kernel Hilbert space (RKHS), and let \mu_{q}(S,\mathbf{w})=\sum_{i\in S}w_{i}\phi(z_{i}) be the weighted empirical kernel mean embedding over selected prototypes z_{i} from \mathcal{Z} , ProtoDash selects a subset S\subseteq\{1,\ldots,n\} and weights \mathbf{w}\in\mathbb{R}^{|S|}_{\geq 0} that maximize the similarity (inner product) between \mu_{p} and \mu_{q} : \phi^{\text{ProtoDash}}(x)=\arg\max_{S,\mathbf{w}\geq 0}\langle\mu_{p},\mu_{q}(S,\mathbf{w})\rangle=\arg\max_{S,\mathbf{w}\geq 0}\sum_{i\in S}w_{i}\langle\mu_{p},\phi(z_{i})\rangle , subject to constraints: |S|\leq m,\quad w_{i}\geq 0\;\forall i, where m is the maximum number of prototypes. A greedy algorithm is used to select the prototypes that best align with the target distribution in kernel space. The final explanation consists of the selected prototypes \{z_{i}\}_{i\in S} and their weights \{w_{i}\} , which highlight representative examples that influence f(x) . II-B Membership Inference Attacks Membership inference attacks (MIAs) pose a significant privacy risk in ML models trained on sensitive data, allowing the leakage of private information [ 34 , 20 , 30 , 40 , 5 , 25 , 52 , 19 , 7 , 8 , 51 , 16 ] . Given a model f trained on a training set D , a sample x , and auxiliary information \mathcal{I} available to the adversary, an attack model \mathcal{A} aims to infer membership as: \mathcal{A}:(x,f,\mathcal{I})\rightarrow\{0,1\}, (1) where \mathcal{A} outputs 1 if x\in D (member) and 0 otherwise (non-member) [ 34 ] . In practice, most MIAs operate in a black-box setting, where the adversary has access only to the model’s output probabilities but lacks direct knowledge of its parameters [ 5 , 39 ] . A common strategy for such attacks is to train shadow models that mimic the behavior of f on auxiliary training data drawn from the same distribution as D . The shadow models are then used to generate attack data to train \mathcal{A} [ 34 , 30 ] . Other approaches rely on statistical properties such as loss entropy [ 50 ] or loss trajectories during model training [ 17 ] . Recent work has introduced a more advanced MIA called the Likelihood Ratio Attack (LiRA), which assesses per-sample vulnerability by training multiple (hundreds) shadow models per instance and analyzing confidence distributions [ 5 ] . When explanations are exposed, the adversary’s auxiliary information \mathcal{I} expands to include explanation outputs and \mathcal{A} is denoted as: \mathcal{A}:(x,f,\mathcal{I},\mathcal{E})\rightarrow\{0,1\}, (2) where \mathcal{E} denotes a post‑hoc explanation function. For example, \mathcal{E} may output an attribution map \phi\in\mathbb{R}^{d} such that \phi_{i} scores feature i . Recent work [ 18 ] shows that supplementing f(x) with \phi can increase the effectiveness of MIA. III Related Work Shokri et al. [ 33 ] were the first to demonstrate membership leakage of model explanations. They use a threshold-based attack that leverages explanation variance to differentiate members vs. nonmembers. Their intuition is that when a model is confident about a prediction, small perturbations will not change the model’s output; therefore, the feature attributions are low, leading to lower explanation variance. They showed that gradient-based explanations, due to their high variance, are the most susceptible to MIAs. They explored the trade-off between privacy and explainability by analyzing perturbation-based explanations which are more resistant to such attacks but come at the cost of lower-quality explanations. Liu et al. [ 18 ] built the most effective MIAs on attribution-based explanations, through a model-based attack that uses explanations to perturb inputs. The intuition is that members should have lower prediction probability scores under perturbations compared to non-members. They attribute differences in explanations to the generalization gap between training and testing data. They also noted that there is a difference in the degree of membership leakage between different explanation methods, inferring that explanation methods with greater accuracy potentially pose a higher risk of membership leakage. Counterfactual explanations, which are hypothetical data samples that provide insights into decision boundaries, also pose privacy risks. Pawelczyk et al. [ 27 ] show that in algorithmic recourse, counterfactual explanations intended to help users reverse a bad decision of the model can be abused to launch MIAs. Furthermore, Naretto et al. [ 24 ] show that membership leakage risks extend to global explanation methods. DeepLeak performs a comprehensive analysis to examine the root causes of membership leakage, but also includes lightweight hardening strategies to reduce membership leakage while maintaining explanation utility. This additional contribution makes DeepLeak the first framework for diagnosing and minimizing privacy risks across explanation methods. Beyond MIAs, additional privacy risks of model explanations have also been explored. Zhao et al. [ 53 ] extended the privacy risks of model explanations to model-inversion attacks: reconstructing sensitive information (e.g., faces) from model explanations. They found that activation-based (saliency map) explanations leak more privacy than sensitivity-based (gradient) explanations. Duddu et al. [ 9 ] focus on the privacy risks of attribute inference attacks on model explanations, where an adversary can infer sensitive attributes (e.g., race and gender) of individual data records from their corresponding model explanations. Luo et al. [ 22 ] analyze the privacy risks of Shapley‐values-based model explanations by introducing two feature‐inference attacks, reconstructing private model inputs from their shapley explanation values, and validating their effectiveness across leading ML-as-a-service platforms. Wang et al. [ 47 ] address the issue of security and privacy of model extraction attacks with counterfactual explanations. Differential privacy (DP) has been utilized to mitigate the privacy risks of model explanations. Patel et al. [ 26 ] developed DP for computing model explanations, via an adaptive DP-SGD algorithm that uses a minimal privacy budget but provides accurate explanations. However, they also applied DP-SGD on the model, which resulted in utility loss. Yang et al. [ 48 ] proposed a DP algorithm to derive counterfactual explanations with DP guarantee. IV DeepLeak Approach Fig. 1: DeepLeak system overview. Overview. Figure 1 shows an overview of the DeepLeak approach. It comprises two main steps: (I) Membership Leakage Profiling and (II) Privacy-Enhancing Hardening Against Membership Leakage . In (I), we develop a more effective explanation-guided MIA than prior work [ 18 ] . This stage establishes a membership leakage benchmark for a broad spectrum of explanation methods deployed under default configurations. In § V-B , we show that our attack is more powerful than state-of-the-art explanation-guided MIA [ 18 ] , which proves the existence of more powerful attacks against explanation APIs, and hence the need for effective mitigation strategies. Using membership leakage measurements, we correlate key algorithmic parameters of explanation methods with observed leakage rates to reveal mechanisms by which explanations expose membership signals. Then in (II) we develop lightweight hardening methods tailored to the nature of each explanation family. We design our hardening methods in a manner that significantly reduces membership leakage with no/minimal reduction on explanation utility. In § V-C , we show the effectiveness of our hardening strategies across families of explanation methods. In § V-D , we draw insights as to the root causes of membership leakage in model explanations across families of explanation methods. Our analysis suggests new insights compared to previous work [ 18 ] . IV-A Membership Leakage Profiling The default configurations of post-hoc explanation methods are primarily aimed at offering high utility explanations. Establishing privacy risk implications of such default configurations/parameters of explanation methods offers a benchmark of leakage under default configurations and informs our pursuit for privacy-enhancing hardening of explanation methods against leakage without compromising explanation utility. Let f_{t} be a model trained on dataset D=(x_{1},y_{1})...(x_{n},y_{n}) where each x_{i} is of dimension d . Let \mathcal{E} be an explanation method (function) that, given a model f_{t} , a test input x , and a prediction f_{t}(x) , generates an attribution score vector \phi=[\phi_{1},...,\phi_{d}] where each \phi_{i} scores feature i of x . To realize membership leakage profiling, we need to launch a MIA against f_{t} by exploiting attribution scores of inputs generated by \mathcal{E} as potential sources of membership signal. This process is repeated for as many \mathcal{E} ’s and MIAs as needed. Threat Model. The adversary’s goal is to use feature attribution scores from \mathcal{E}(f_{t}(x)) to determine whether or not x was used to train f_{t} . With regards to adversary’s knowledge and capabilities , we assume that the adversary has black-box API access to f_{t} with access only to f_{t}(x) , and the corresponding attribution map \phi with no visibility into the details of the explanation method and its configurations. We also assume that the adversary has access to an auxiliary dataset sampled from the same distribution as D . This will give the attacker the possibility to collect another auxiliary data that is out-of-distribution compared to D . The adversary also knows the target model architecture. These assumptions are realistic and are in line with previous threat models for MIAs [ 20 , 5 , 16 , 30 , 34 ] . We will relax these assumptions in the ablation study by using disjoint datasets and different model architectures between the target model and the shadow model. Attack Details. The rationale of our MIA is that defenders can audit leakage using publicly known attacks by simulating a worst-case attack. With this rationale, our MIA is inspired by [ 33 ] , which demonstrated explanation-guided MIA. Based on our threat model, we partition each dataset into 4 subsets: D^{\mathit{train}}_{\mathit{target}} , D^{\mathit{train}}_{\mathit{shadow}} , D^{\mathit{test}}_{\mathit{target}} , and D^{\mathit{test}}_{\mathit{shadow}} . D^{\mathit{train}}_{\mathit{target}} is used to train the target model f_{t} and all samples in it are members . D^{\mathit{test}}_{\mathit{target}} consists of samples that are treated as non-members with respect to the target model. D^{\mathit{train}}_{\mathit{shadow}}\subset D^{\mathit{train}}_{\mathit{target}} is used to train a shadow model f_{s} that mimics f_{t} . D^{\mathit{test}}_{\mathit{shadow}} is designated as non-members with the same size as D^{\mathit{train}}_{\mathit{shadow}} , and is disjoint with D^{\mathit{test}}_{\mathit{target}} so as to test the generalization of our MIA against f_{t} on members and non-members. We craft a shadow-model MIA based on Shokri et al. [ 33 ] , but with two key modifications motivated by our baseline results. In the canonical shadow attack the shadow models are trained and evaluated on data drawn from the same distribution as f_{t} . When we followed that recipe the attack achieved poor performance (Table I ). We hypothesize that the attack’s weakness stems from a mismatch between (a) the distributional relationship between members and non-members that the adversary assumes during shadow training and (b) the actual relationship produced by the target model where the training and test data could come from different distributions. To better match realistic conditions and give the attack model a stronger leverage on the membership signal, we train and test each shadow model using shadow training set D^{\mathit{train}}_{\mathit{shadow}} and shadow test set D^{\mathit{test}}_{\mathit{shadow}} , respectively, such that members are drawn from the same distribution as training set of f_{t} and non-members are drawn from a held-out out-of-distribution data. More precisely, for each shadow model we (i) sample D^{\mathit{train}}_{\mathit{shadow}} and D^{\mathit{test}}_{\mathit{shadow}} , (ii) train a shadow model f_{s} on D^{\mathit{train}}_{\mathit{shadow}} , and (iii) compute attribution vectors \phi(x)=\mathcal{E}(f_{s},x) for samples in D^{\mathit{train}}_{\mathit{shadow}} (labeled ( y=1 = member) and D^{\mathit{test}}_{\mathit{shadow}} (labeled y=0 = non-member). The labeled attribution dataset \{(\phi(x),y)\} is used to train the attack model \mathcal{A} that predicts membership from explanation outputs. Because adversaries in practice explore multiple design choices (e.g., random seeds, shadow model architectures), we repeat the entire shadow-model-training \rightarrow attribution \rightarrow attack-model-training pipeline across multiple random seeds and configurations, and report the attack variant that performs best on a held-out validation set. TABLE I: Effectiveness of baseline MIA . Measured via TPR@0.1%FPR for different explanation methods using same distribution for train and test of shadow model on CIFAR-10. Explanation Method TPR@0.1%FPR SmoothGrad 0.07% VarGrad 0.1% IG 0.52% GradCAM 0.11% GradCAM++ 0.52% SHAP 0.41% LIME 0.37% Membership Leakage Score. The effectiveness of MIAs is realistically measured by the true-positive rate (TPR) an adversary achieves at a low false-positive rate (FPR) [ 5 , 3 ] . Accordingly, we define the Membership Leakage Score (MLS) of an explanation method \mathcal{E} on a target model f_{t} as the TPR at a (low) target FPR level (e.g., 0.1\% ) when the attack has access to only explanation vectors \phi(x)=\mathcal{E}(f_{t},x) . This isolates leakage that originates from explanations, unlike previous works (e.g., [ 18 ] ) which combine explanations with additional model outputs (e.g., predicted labels and probability vectors) as the attacker’s information \mathcal{I} . Formally, the MLS score is calculated against the target model f_{t} as: MLS=TPR(\mathcal{A}(\mathcal{E}(\mathbf{X_{f_{t}}}))\mid FPR=\epsilon) (3) where, \mathcal{E}(x) is the explanation vector, \text{TPR}(\mathcal{A}(\mathcal{E}(x))) is the true positive rate of the attack, \mathbf{X_{f_{t}}}\subset(D^{\mathit{train}}_{\mathit{target}}\cup D^{\mathit{test}}_{\mathit{target}}) , and \epsilon is the FPR target. \text{FPR}\leq\epsilon ensures low false positives. Explanation Utility. We consider the utility of an explanation method as its ability to retain informative explanations under privacy-preserving transformations. In this context, higher sensitivity indicates a greater change in explanations, which implies a reduction in utility. Conversely, lower sensitivity corresponds to higher utility, as the explanations remain closer to their original form. We compared (in)fidelity and sensitivity from [ 49 ] , and we found that the (in)fidelity metric behaved inconsistently across different explanation methods (e.g., some produced values in the range [10,100] while others in the range [1,000,40,000]). In light of the inconsistency, we opted to use the more stable metric, explanation sensitivity , as a proxy to assess the stability of model explanations. Sensitivity measures the degree of explanation changes to subtle input perturbations, which impacts the reliability of model explanations. High sensitivity implies that small variations in the input can lead to significant shifts in the explanation, making models more susceptible to adversarial manipulation and membership leakage attacks [ 10 , 18 ] . In DeepLeak , we measure the sensitivity pre- and post-hardening to assess the trade-off between hardening against leakage and the utility of the explanation. The lower sensitivity of an explanation to a membership leakage hardening countermeasure implies that the explanation method reduces membership leakage without reducing the utility of the explanation. We define the sensitivity of an explanation method \mathcal{E} denoted as \mathbb{S}(\mathcal{E},f_{t},x,r) as the maximum change between \mathcal{E}(f_{t},x) and \mathcal{E}(f_{t},x+\delta) with respect to a small perturbation \delta around input x , constrained within an ||L||_{p} (e.g., L_{\infty} ) ball of radius r . It is computed as follows: \mathbb{S}(\mathcal{E},f_{t},x,r)=\max_{\|\delta-x\|\leq r}\|\mathcal{E}(f_{t},x+\delta)-\mathcal{E}(f_{t},x)\| (4) High sensitivity entails vulnerability to adversarial perturbations, where small, imperceptible changes in input significantly alter feature attributions. By minimizing \mathbb{S} , the stability of the explanation is improved, implying robust interpretability. IV-B Hardening Against Membership Leakage Explanation-guided MIA is a statistical test that aims to distinguish two distributions: the explanation vectors (maps) for members versus non-members. An ideal hardening mitigation works by reducing the statistical separability (divergence) between these two distributions without compromising the utility of the explanation. To achieve this objective, previous work used differential privacy (DP) [ 1 ] . However, Patel and Shokri [ 26 ] show that DP negatively affects the utility of the model by reducing the fidelity of the explanation. Given the well-documented cost of DP on model and explanation utility, in the hardening strategies we describe next, we aim to develop privacy enhancing hardening with no impact on model utility and acceptably minimal impact on explanation utility. Before developing our hardening methods, we conducted an in-depth analysis of 15 explanation methods. Based on our analysis, we categorize the explanation methods into two main classes: parameterized (those with tunable parameters) and non-parameterized (those without tunable parameters). Algorithm 1 Hardening Parameterized Methods 1: X (data), f_{t} (target model), \mathcal{E} (explanation method), \mathcal{A} (attack model), \mathcal{\phi} (attribution scores) , \mathcal{T} (Number of Trials) 2: \theta^{*} (Optimized explanation parameters) 3: function Objective ( \mathcal{E},f_{t},X ) 4: Sample \theta_{p} 5: \phi_{\theta_{p}}\leftarrow\mathcal{E}(f_{t},X;\theta_{p}) 6: U\leftarrow\mathbb{S}(\phi_{\theta_{p}}) \triangleright Evaluate explanation utility 7: MLS\leftarrow\mathcal{A}(\phi_{\theta_{p}}) \triangleright MIA attack 8: return (U_{\theta_{p}},MLS_{\theta_{p}}) 9: end function 10: function Optimizeexplanation ( \mathcal{E},f_{t},X ) 11: \theta^{*}\leftarrow\arg\min_{\theta}MLS,\arg\max_{\theta}U(\text{Objective($\mathcal{E},f_{t},X$)},n=\mathcal{T}) 12: return \theta^{*} 13: end function Algorithm 2 Hardening Non-Parameterized Methods 1: X (data), f_{t} (target model), \phi (raw attributions), \theta_{p}=(\sigma,c_{min},c_{max},\tau) (privacy params), \mathcal{T} (Number of Trials) 2: \theta^{*} (privacy-enhancing parameters) 3: function PrivacyHardening ( \mathcal{E},f_{t},X ) 4: \phi\leftarrow\mathcal{E}(f_{t},X) 5: Sample \theta_{p} 6: \phi\leftarrow\text{clip}(\phi,c_{\min},c_{\max}) \triangleright Clamp Values 7: \phi[\phi<\tau]\leftarrow 0 \triangleright Mask Weak Attributions 8: \phi\leftarrow\phi+\mathcal{N}(0,\sigma^{2}) \triangleright Add Noise 9: U\leftarrow\mathbb{S}(\phi_{\theta_{p}}) \triangleright Evaluate explanation utility 10: MLS\leftarrow\mathcal{A}(\phi_{\theta_{p}}) \triangleright MIA attack 11: return (U_{\theta_{p}},MLS_{\theta_{p}}) 12: end function 13: function OptimizePrivacy ( \mathcal{E},f_{t},X ) 14: \theta^{*}\leftarrow\arg\min_{\theta}MLS,\arg\max_{\theta}U(\text{PrivacyHardening($\mathcal{E},f_{t},X$)},n=\mathcal{T}) 15: return \theta^{*} 16: end function Hardening Parameterized Explanation Methods. In this category ( [ 38 , 2 , 44 , 44 , 32 , 6 , 21 , 28 , 46 , 12 , 29 ] ), each explanation method relies on a set of parameters to generate explanations. These parameters were originally introduced by the researchers who developed the methods. In their respective works, they explore different configurations and report varying performance outcomes, ultimately recommending what we refer to as default configuration per an explanation method. Table VIII in the Appendix shows, for each explanation method, the complete set of parameters along with their respective search space or options available for attribution generation. Using the default settings in the original works, we implemented all these explanation methods and observed that they resulted in significant membership leakage. As detailed in Algorithm 1 , the hardening of a parameterized explanation method is an optimization problem with two objectives: (a) minimizing membership leakage score MLS and (b) maximizing explanation utility U . Towards achieving these objectives, DeepLeak systematically explores different combinations of parameter values to reach an ideal point : a MLS score of zero and the highest achievable score for U . This is achieved through an iterative search process in the search space of tunable parameters of the explanation method at hand. Initially (e.g., the first 5 iterations), Algorithm 1 identifies promising trajectories for tuning, i.e., whether a given parameter it to be increased or decreased based on observed performance. For example, if a parameter is numeric with defined upper and lower bounds, early iterations help infer the direction in which adjustments are likely to lead to achieving the optimization goals. Over time, this adaptive sampling leads to convergence to an optimal or near-optimal configuration for each explanation method, balancing privacy and utility. Algorithm 1 at its core is an objective function that evaluates a candidate set of parameters \theta for a given explanation method \mathcal{E} (line 2). It then computes the explanation vector \phi_{\theta_{p}} by directly applying \mathcal{E} with the sampled parameters (line 3). It then evaluates sensitivity (line 4). Next, membership leakage is measured by applying the attack model \mathcal{A} on \mathcal{E} and computing the MLS (line 5). Our hardening method iteratively samples configurations and uses this dual-objective feedback to refine its search (lines 8–11). The hardening continues for a fixed number of trials (e.g., 20), and ultimately returns the parameter setting \theta^{*} that minimizes MLS while maximizing U . This iterative approach allows DeepLeak to automatically discover the most privacy-preserving yet interpretable configurations of explanation methods. The initial iterations help map the performance landscape, guiding subsequent trials toward optimal regions in the parameter space. Hardening Non-Parameterized Explanation Methods: For non-parameterized explanation methods [ 36 , 37 , 23 , 35 , 41 ] , our approach to hardening against leakage is based on transformations applied on attribution scores. Algorithm 2 shows the details of our hardening method, with three transformation functions applied to attribution scores: clipping gradient values, masking low signal attributions, and calibrated noise injection. Next, we describe these three transformations. Clipping Attribution Values: Bounding attribution values (line 4) within a fixed range prevents extreme feature contributions from standing out. Since attributions values have negative and positive values, we allocate the lowest boundary for clipping negative values and highest boundary for the positive ones. Doing so ensures that training samples (members) do not have significantly different attribution (e.g., gradient) magnitudes compared to non-training samples (non-members), making it harder for the adversary to identify members. Masking Low Signal Attributions: Zeroing out low signal attributions below a set threshold removes weak, non-informative attribution values that could be used to infer hidden model behavior (line 5). This reduces the risk of attackers detecting tiny but consistent feature contributions unique to members/non-members. Calibrated Noise Injection: Adding Gaussian noise (line 6) to attributions ensures that attackers cannot distinguish between members and non-members based on small variations in gradient sensitivity. It introduces randomness, making MIA less effective by preventing exact reconstruction of model responses as is done in privacy-preserving training schemes such as DP-SGD [ 1 ] . Unlike DP-SGD, where the sensitivity of gradients must be carefully computed to determine a fixed Gaussian noise that satisfies DP guarantees, DeepLeak directly parameterizes the Gaussian noise distribution and integrates this parameter into an optimization schema. This schema functions as a noise-level selector, dynamically balancing the privacy–utility trade-off: it enforces an upper bound on the noise parameter to satisfy privacy requirements while simultaneously ensuring that the utility of explanations is preserved. In doing so, sensitivity considerations are implicitly handled through the bounded parameter space, allowing us to achieve privacy protection without compromising the interpretability of the explanation method. As shown in Algorithm 2 , we parameterize the attributions using \theta_{p}=(\sigma,c_{\min},c_{\max},\tau) (line 3) and compute their utility U (line 7) and privacy leakage MLS (line 8). These metrics are returned jointly for evaluation (line 9). Finally, the optimization procedure (lines 11–14) selects the parameters \theta^{*} that minimize leakage while maximizing utility. The order and combination of these transformations dictate the membership leakage vs. explanation utility trade-off. Our empirical analysis on how to order the three transformations is in § V-E . We observe that applying only clipping and masking (lines 4–5) led to a reduction in membership leakage. However, the utility of explanation dropped. Therefore, we add calibrated Gaussian noise (line 6), as the clip \rightarrow mask \rightarrow add_noise sequence proved effective in significantly reducing leakage with negligible reduction on explanation utility. Empirical hardening vs. formal privacy: Although DeepLeak provides lightweight and model-agnostic mechanisms to reduce membership leakage from explanation outputs, these defenses are empirical in nature and do not offer formal guarantees of the likes of differential privacy (DP). Instead, our goal is to characterize and optimize the empirical privacy/utility tradeoffs that arise across a wide range of explanation methods. As we show in § V-C , carefully chosen transformations can substantially reduce leakage while preserving explanation utility. We position DeepLeak as a practical framework for auditing and mitigating explanation-induced leakage, complementary to formal DP-based approaches. V Evaluation Our validation of DeepLeak is guided by the following research questions: • RQ1 : How leaky are explanation methods when deployed with default parameters/configurations? • RQ2 : How effective are explanation method hardening strategies in reducing membership leakage without compromising explanation utility? • RQ3 : What are the root causes of leakage in explanation methods? Do the root causes vary among classes of explanation methods? V-A Experimental Setup Datasets : We use 3 benchmark datasets: CIFAR-10 [ 15 ] , CIFAR-100 [ 15 ] , and GTSRB [ 42 ] that are common in MIA studies. Following our attack setup in § IV-A , Table IX in the Appendix §