Improving Stability Estimates in Adversarial Explainable AI through Alternate Search Methods

Christopher Burger, Charles Walter · 2025

Read Improving Stability Estimates in Adversarial Explainable AI through Alternate Search Methods by Christopher Burger, Charles Walter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-09006.txt.

Introduction

Christopher Burger and Charles Walter probe how easily the explanations of a black-box model can be fooled, in a 2025 paper. Local surrogate methods explain a complex model by fitting a simple stand-in near one case. Attackers have shown they can nudge the input so the explanation changes a lot while the model's answer stays much the same. Prior work proved such attacks exist. This paper asks how big the nudge must be.

The authors search for the smallest set of changes that drives an explanation to a set level of difference. A method that falls after few changes is weaker than one that holds out for many. That yardstick turns a yes-or-no weakness into a graded score, so explanation methods can be ranked by toughness. The paper speaks to the growing field that audits AI explanations for trust.

Catalog record

CREATORChristopher Burger, Charles Walter
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-09006

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving Stability Estimates in Adversarial Explainable AI through Alternate Search Methods Christopher Burger, Charles Walter Abstract Advances in the effectiveness of machine learning models have come at the cost of enormous complexity resulting in a poor understanding of how they function. Local surrogate methods have been used to approximate the workings of these complex models, but recent work has revealed their vulnerability to adversarial attacks where the explanation produced is appreciably different while the meaning and structure of the complex model’s output remains similar. This prior work has focused on the existence of these weaknesses but not on their magnitude. Here we explore using an alternate search method with the goal of finding minimum viable perturbations, the fewest perturbations necessary to achieve a fixed similarity value between the original and altered text’s explanation. Intuitively, a method that requires fewer perturbations to expose a given level of instability is inferior to one which requires more. This nuance allows for superior comparisons of the stability of explainability methods. Keywords: Explainability, Interpretability, XAI, Stability, Robustness 1 Introduction Recent progress in machine learning and AI has enabled the widespread use of powerful but highly complex models in many aspects of our lives. These advanced models are applied across various fields but in areas with high consequences for failure, such as medicine or finance, there has been slower adoption due to the potential cost of error. All models come with some degree of error, but assigning blame for failures can be challenging, depending on the type of failure. Error within a generative model creating an email template may only cause confusion or embarrassment but failure in a model designed for producing a medical diagnosis is far more serious. When any model fails, its stakeholders need to understand the reasons behind the cause of the failure. However, as models grow increasingly complex, directly interpreting their actions becomes impractical. In response, the field of explanatory AI (XAI) aims to develop tools that assist both developers and users to comprehend the workings of a model. One common XAI approach is to create an interpretable approximation of the complex model, known as a surrogate model. These surrogate models help explain why a model made certain predictions and, in practice, focus primarily on explaining individual predictions rather than attempting to explain the entire model. As surrogate models are models themselves, they are subject to the same weaknesses as the original models they are designed to explain. One necessary property these surrogate models need to satisfy is stability, also known as robustness. Intuitively, a stable model possesses the attribute that when an insignificant change is applied to the input, the corresponding output is subject to at most an insignificant change ( Alvarez-Melis & Jaakkola 2018a ) . A lack of stability is leveraged in adversarial attacks where, given an unstable model, we see changes that should not be meaningful to an ideal model produce appreciably erroneous results. Many common local surrogate models have been shown to lack stability across the major data modalities: image, tabular, and text ( Slack et al. 2020a ; Ivankay et al. 2022a ) . For XAI specifically, this instability is with respect the explanation provided by the surrogate model. Generally, these explanations are a list of features ordered by importance and, when subject to a successful adversarial attack, the explanation produced is different while the input to the model being explained remains similar. Evidence of instability means the explanation, and the explanatory model itself, cannot be trusted. The original complex model then remains a black-box and may be prevented from use due to lack of understanding surrounding its function due to legal or social conventions despite gains in effectiveness over other models. Prior work in adversarial XAI has focused on establishing the existence of instability and has yet to explore how much instability may be present. Previous work determined the presence of instability using a fixed maximum amount of change applied to the input. Notably, this contrasts to standard adversarial attacks on classification models. Standard attacks often also possess some perturbation or query limit, but the process itself has an inherent end goal which is the change in predicted class. Adversarial attacks on XAI methods lack this innate measure of success and so have relied on search exhaustion or perturbation maximums instead ( Burger et al. 2023a ) . As the existence of instability has already been demonstrated, we instead seek to refine our knowledge of it by asking: What is the minimum number of perturbations that will sufficiently alter an explanation. To do so, we need to establish some metric of success, just as that which exists in classical adversarial perturbations. However, unlike a standard adversarial attack in which the alteration of the predicted class is the obvious metric of success, exactly what determines a suitable difference between explanations is more subjective. With a defined threshold, we will be able to answer the question of stability beyond its existence and instead answer how substantial is it, especially compared to other methods. A method that consistently lacks stability when exposed to only a small number of perturbations is clearly worse than one that that requires a consistently large subset of the text to be changed. In Table 1 we see the first document (26.5% similarity) requires substantially more perturbations to achieve a comparable level of similarity to that of the second attack (28.9% similarity). The presence of a single perturbation that provides a significant decrease in similarity implies a much more serious instability when compared to a method that requires multiple perturbations. We restrict our focus to XAI models for text-based data. Our reasons are as follows: (1) Text-based models are ubiquitous as natural language is a primary communication mechanism. (2) As natural language is woven into the fabric of our daily experiences we as humans have an inherently robust sense for understanding the meaning of a document and so can judge the quality of a text document’s explanation effectively without any formal procedure or training. This innate ability to judge the quality of an explanation allows us to choose reasonable thresholds, reducing some of what is an inherently subjective decision. (3) Our depth of investigation can be increased significantly. Testing XAI stability is a computationally intensive process, so focusing on one data type allows a much broader range of thresholds and similarity measures to be tested given a fixed amount of time and computational resources. To locate these sets of minimal perturbations we test an alternative search process, a genetic algorithm, to allow a wider exploration of the search space. Existing search processes are highly greedy due a combination of being developed primarily to show the existence of instability and due to extensive computational requirements. These processes, while effective, are not optimal for the purposes of investigating just how little perturbation is necessary to achieve a given threshold of instability. The use of the genetic algorithm will then allow more accurate estimates of XAI stability. Table 1: Perturbed explanations with close similarity values (calculated with respect to similarity measure RBO 0.5 ) despite substantial differences in perturbation rate. Original 26.5% Sim. 28.9% Sim. Feature Weight Feature Weight Feature Weight 1 dogs 2.60 tennis 3.16 tennis

Read the complete text