EnsembleSHAP : Faithful and Certifiably Robust Attribution for Random Subspace Method

Yanting Wang & Jinyuan Jia · 2026

Read EnsembleSHAP : Faithful and Certifiably Robust Attribution for Random Subspace Method by Yanting Wang & Jinyuan Jia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-30034.txt.

Introduction

Wang and Jia present EnsembleSHAP, a way to explain random subspace ensembles. Such ensembles back certified defenses against attacks and jailbreaks. Stock tools like Shapley values and LIME cost too much here. They also give no security promises. EnsembleSHAP reuses byproducts of the ensemble's own math. It stays faithful and keeps local accuracy. It also provably resists attacks that aim at the explanations. Tests cover backdoor, adversarial, and jailbreak attacks.

Catalog record

CREATORYanting Wang & Jinyuan Jia
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91890
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.30034

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EnsembleSHAP : Faithful and Certifiably Robust Attribution for Random Subspace Method Yanting Wang & Jinyuan Jia Abstract Random subspace method has wide security applications such as providing certified defenses against adversarial and backdoor attacks, and building robustly aligned LLM against jailbreaking attacks. However, the explanation of random subspace method lacks sufficient exploration. Existing state-of-the-art feature attribution methods, such as Shapley value and LIME, are computationally impractical and lacks security guarantee when applied to random subspace method. In this work, we propose EnsembleSHAP , an intrinsically faithful and secure feature attribution for random subspace method that reuses its computational byproducts. Specifically, our feature attribution method is 1) computationally efficient, 2) maintains essential properties of effective feature attribution (such as local accuracy), and 3) offers guaranteed protection against privacy-preserving attacks on feature attribution methods. To the best of our knowledge, this is the first work to establish provable robustness against explanation-preserving attacks. We also perform comprehensive evaluations for our explanation’s effectiveness when faced with different empirical attacks, including backdoor attacks, adversarial attacks, and jailbreak attacks. The code is at https://github.com/Wang-Yanting/EnsembleSHAP . WARNING: This document may include content that could be considered harmful. 1 Introduction Random subspace method ( Ho 1998 ) , also referred to as attribute bagging ( Bryll et al. 2003 ) , is an ensemble learning method that combines the prediction results on random subsets of features to obtain the final prediction. While it was initially proposed to enhance decision trees ( Ho 1998 ) , this method gained widespread adaptation recently in security applications ( Jia et al. 2021 ; Levine & Feizi 2020b ; Robey et al. 2023 ; Zhang et al. 2023 ; Wang et al. 2021 ; Zeng et al. 2023 ; Cao et al. 2023 ) , such as providing certified defenses ( Levine & Feizi 2020b ; Zeng et al. 2023 ; Zhang et al. 2023 ; Wang et al. 2021 ; Jia et al. 2020 ; Zhang et al. 2021 ; Cao et al. 2021 ; Jia et al. 2022 ) against adversarial attacks, and enhancing the robustness of large language models against jailbreaking attacks ( Cao et al. 2023 ; Robey et al. 2023 ) . This method begins by generating predictions for multiple sub-sampled versions of a given input sample using a base model. It then creates an ensemble model that aggregates these predictions using a majority vote to determine the final prediction. As this approach only requires black-box access to the base model, it can be applied across different base model architectures ( Levine & Feizi 2020b ; Zeng et al. 2023 ; Zhang et al. 2023 ; Wang et al. 2021 ; Robey et al. 2023 ) . Understanding the output of the random subspace method is crucial. For instance, in defending against jailbreaking attacks, it’s essential for users to pinpoint the specific elements of the input prompt that lead to its classification as ‘harmful’ (or ‘non-harmful’). Additionally, when a certified defense is compromised by strong empirical attacks, it becomes important for users to determine the specific adversarial words that caused the misclassification. However, existing leading feature attribution methods for black-box models ( Lundberg & Lee 2017a ; Chen et al. 2023b ; Ribeiro et al. 2016 ; Enouen et al. 2023 ; Paes et al. 2024 ; Amara et al. 2024 ; Mosca et al. 2022 ; Lopardo et al. 2023 ; Wang et al. 2025 ) such as Shapley values ( Lundberg & Lee 2017a ; Chen et al. 2023b ) and LIME ( Ribeiro et al. 2016 ) have following disadvantages when applied to random subspace method. Firstly, they incur prohibitively high computational costs. Specifically, these methods involve randomly perturbing the input sample numerous times, denoted by M . The explained model then generates a prediction for each perturbed version of the input. In the case of the random subspace method, the objective is to explain the behavior of the ensemble model. Therefore, for each perturbed input version, N sub-sampled variants of it are created and each is evaluated to generate the ensemble model’s prediction, resulting in M\times N total queries to the base model for just one input sample. In practical applications, both N and M can exceed 1,000 ( Enouen et al. 2023 ; Zeng et al. 2023 ; Levine & Feizi 2020b ) , which significantly increases the computational cost. Secondly, current feature attribution methods do not provide security guarantees against the recently proposed explanation-preserving attack ( Nadeem et al. 2023 ; Noppel & Wressnegger 2023 ) . In this type of attack, an adversary can perturb certain features of the input sample to cause misclassifications, and at the same time conceal these changes by preserving the original explanation. Our contributions. In this work, we propose a computationally efficient feature attribution method for random subspace method that is inherently faithful and secure. We conduct a theoretical analysis to show that our method maintains key properties of Shapley value and is provably robust against explanation-perserving attacks on feature attribution methods. We note that this is the first work to establish provable robustness against explanation-preserving attacks. We carry out empirical evaluations to assess the effectiveness of our explanations across various security applications. 2 Background and Related Work 2.1 Random Subspace Method Random subspace-based method ( Ho 1998 ) is a versatile approach that is agnostic to model architecture and scalable to large neural networks. This method employs a base model to generate predictions for random feature subsets, which are then aggregated to produce the ensemble prediction. Next, we summarize a general framework for the random subspace method and introduce its security applications. Building an ensemble model. Suppose we have a testing input \mathbf{x}=\{x_{1},x_{2},\cdots,x_{d}\} that consists of d elements, where each element represents a feature of the input. For instance, when \mathbf{x} is a text, each x_{i} represents a word. We use f:X\rightarrow[C] to represent the base model (e.g., an off-the-shelf text classifier), where X is the space of model input and [C] represents the set of unique labels \{1,2,...,C\} that base model can output. For example, in a binary classification problem, [C]=\{1,2\} . For the simplicity of notation, we define a simplified base model h that can take subsets of \mathbf{x} (denoted by \mathbf{z} ) as input. Specifically, we define h:\mathcal{P}(\mathbf{x})\rightarrow[C] as h(\mathbf{z})=f(\text{ABLATE}(\mathbf{x},\mathbf{z})) , where \mathcal{P}(\mathbf{x}) is the power set of \mathbf{x} , \mathbf{z}\in\mathcal{P}(\mathbf{x}) is a subset of \mathbf{x} and ABLATE replaces all features of \mathbf{x} not in \mathbf{z} by a special value (e.g., the ‘[MASK]’ token). That is, x_{i}=x_{i}\text{ if }x_{i}\in\mathbf{z} , and x_{i}=SpecialValue\text{ if }x_{i}\notin\mathbf{z} . The random subspace method first uses the simplified base model h to make predictions for random subsets of \mathbf{x} . Formally, we define the probability that a label c\in[C] is predicted by the base model as: \displaystyle p_{c}(\mathbf{x},h,k)=\mathbb{E}_{\mathbf{z}\sim\mathcal{U}(\mathbf{x},k)}[\mathbb{I}(h(\mathbf{z})=c)], (1) where \mathbb{I} is an indicator function whose output is 1 if the condition is satisfied and 0 otherwise, and \mathbf{z}\sim\mathcal{U}(\mathbf{x},k) is a subset of \mathbf{x} with size k that is randomly sampled from the uniform distribution. i.e., \Pr(\mathbf{z}=\mathbf{z}^{\prime}\mid\mathbf{z}\sim\mathcal{U}(\mathbf{x},k))=\frac{1}{\binom{d}{k}} for any \mathbf{z}^{\prime}\subseteq\mathbf{x} satisfying |\mathbf{z}^{\prime}|=k . Then the label with the largest probability is viewed as the predicted label of the ensemble classifier H for the testing input \mathbf{x}

Read the complete text