XMutant : XAI-based Fuzzing for Deep Learning Systems

Xingcheng Chen 1,2, Matteo Biagiola 3, Vincenzo Riccio 4, Marcelo d’Amorim 5, Andrea Stocco 1,2 · 2025

Read XMutant : XAI-based Fuzzing for Deep Learning Systems by Xingcheng Chen 1,2, Matteo Biagiola 3, Vincenzo Riccio 4, Marcelo d’Amorim 5, Andrea Stocco 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07222.txt.

Introduction

Xingcheng Chen and co-authors present a software testing paper, published in 2025, on hunting failures in deep learning systems more cleverly. Standard test generators mutate inputs at random until a model breaks, which can waste much time. Their tool, XMutant, instead reads an explanation of why the model answered as it did and aims mutations at the parts of the input that mattered most.

Trials covered sentiment analysis, digit reading, and a driving assistance system. Guided mutation produced far more failure revealing inputs than a random baseline and found them much faster. Human and automated checks confirmed most generated inputs were still valid examples. The authors argue explanations, built to help people trust models, can also help break them on purpose.

Catalog record

CREATORXingcheng Chen 1,2, Matteo Biagiola 3, Vincenzo Riccio 4, Marcelo d’Amorim 5, Andrea Stocco 1,2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07222

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

XMutant : XAI-based Fuzzing for Deep Learning Systems Xingcheng Chen 1,2, Matteo Biagiola 3, Vincenzo Riccio 4, Marcelo d’Amorim 5, Andrea Stocco 1,2 Abstract Semantic-based test generators are widely used to produce failure-inducing inputs for Deep Learning (DL) systems. They typically generate challenging test inputs by applying random perturbations to input semantic concepts until a failure is found or a timeout is reached. However, such randomness may hinder them from efficiently achieving their goal. This paper proposes XMutant , a technique that leverages explainable artificial intelligence (XAI) techniques to generate challenging test inputs. XMutant uses the local explanation of the input to inform the fuzz testing process and effectively guide it toward failures of the DL system under test. We evaluated different configurations of XMutant in triggering failures for different DL systems both for model-level (sentiment analysis, digit recognition) and system-level testing (advanced driving assistance). Our studies showed that XMutant enables more effective and efficient test generation by focusing on the most impactful parts of the input. XMutant generates up to 125 % 125\% more failure-inducing inputs compared to an existing baseline, up to 7 × \times faster. We also assessed the validity of these inputs, maintaining a validation rate above 89 % 89\% , according to automated and human validators. ∎ Journal: Empirical Software Engineering Keywords. Software testing, testing deep learning systems, explainable AI 1 Introduction Deep Learning (DL) systems play a crucial role in software engineering Schütze et al. (2008) due to their ability to solve complex tasks by learning from a corpus of data. However, DL systems pose additional challenges to testing. Unlike traditional software, where behavior can be anticipated through source code analysis, the actions of DL systems are less predictable Riccio et al. (2020) . Therefore, test generation for DL systems is critical for ensuring their reliability and correctness. This process involves creating a diverse set of input data to uncover potential weaknesses or biases in the model, such as incorrect predictions or failures to generalize from data samples that are different from those available during training. Testing is particularly important in applications where DL is used for decision-making in critical scenarios, such as autonomous driving Stocco et al. (2020) ; Ben Abdessalem et al. (2018) ; Gambi et al. (2019) ; Tang et al. (2022) . Exhaustive testing of DL systems is impractical because of the size of the input space. Thus, generating effective test cases for these systems is an important problem, which requires a comprehensive understanding of the DL model’s architecture, its training data, and the application domain. Prior work proposed solutions to generate test cases automatically for DL systems Riccio and Tonella (2020) ; Abdessalem et al. (2018) ; Ben Abdessalem et al. (2016) ; Ben Abdessalem et al. (2018) ; Lu et al. (2023) ; Lou et al. (2021) ; Mullins et al. (2018) ; Gambi et al. (2019) ; Kim et al. (2022) ; Zhong et al. (2021) ; Li et al. (2020) ; Jha et al. (2019) ; Cheng et al. (2023) . Some approaches such as DeepXplore Pei et al. (2017) , DLFuzz Guo et al. (2018) and DeepTest Tian et al. (2018) target DL image classification systems and they involve raw input manipulation techniques that modify/corrupt pixel values of an input. These techniques do not generate new functional inputs as they produce minimal, often imperceptible changes to the original inputs, and are therefore suitable to test robustness and security deficiencies of the DL system Riccio and Tonella (2023) ; Maryam et al. (2025) . In contrast, functional test generation focuses on creating new inputs that deviate significantly from the original training distribution. These inputs target the long-tail problem of DL testing Zhang et al. (2024a) , testing the DNN’s ability to generalize to novel, unseen scenarios. Instances of functional test generators are the model-based approaches (also called semantic-based approaches) like DeepJanus Riccio and Tonella (2020) , DeepHyperion Zohdinasab et al. (2021) and DeepMetis Riccio et al. (2021) or latent space manipulation techniques like SINVAD Kang et al. (2020) ; Kang et al. (2024) and CIT4DNN Dola et al. (2024) . They generate new inputs with mutations that are randomly applied to a semantic representation of the inputs Riccio and Tonella (2020) , or to a latent vector Kang et al. (2020) . While random mutations can eventually produce inputs that expose failures, these solutions are extraneous to the internal state of the DL system. Indeed, finding failures with black-box approaches is time-consuming, especially in computationally demanding contexts, such as simulation-based testing of self-driving cars. This paper investigates the development of more effective and efficient semantic-based test generators for DL systems, leveraging eXplainable Artificial Intelligence (XAI) techniques. While methods similar to XAI, leveraging neuron activation values and gradient information, have been explored for raw input manipulation Yuan et al. (2021) ; Ma et al. (2018) ; Xie et al. (2019) to derive adversarial attacks for robustness testing, their application to semantic-based functional test generation remains unexplored. Therefore, we focus on semantic-based approaches that utilize an input model, as they have been successfully applied to various DL systems (e.g., classification, regression) and input types (e.g., text, images, logical driving scenarios). In this work, we leverage the post-hoc local explanations of DL systems for individual predictions. Local explanations reveal the contributions of features in the input with respect to the model prediction Du et al. (2019) . Depending on the model’s input, the explanations can take the form of feature contributions (e.g., for textual inputs), or visual heatmaps (e.g., for image inputs) Jetley et al. (2018) ; Samek et al. (2019) ; Samek et al. (2017) . Previous work highlights the valuable insights provided by XAI, particularly for understanding and debugging DL systems Tjoa et al. (2022) ; Stocco et al. (2022) ; Fahmy et al. (2023) ; Fahmy et al. (2020) . In our work, we leverage their information for guiding test generation by introducing XMutant , a DL testing technique that leverages XAI’s explanations to derive challenging inputs. More specifically, XMutant leverages a novel mutation operator that uses the local explanation on a given input to identify the area of its semantic representation that has higher contributions to the decision-making process of DL systems. Our work shows that fuzzing such attention areas with targeted minor modifications accelerates fault exposure and preserves validity (i.e., the realism of the inputs) and label (i.e., the oracle). XMutant uses the local explanations in two ways. First, it uses them to rank and select the candidate semantic concepts from the semantic representation of input for mutation. Second, it uses these concepts to direct the mutation in the area of most attention of the DL system, enabling the approach to perform targeted input modifications. We have evaluated XMutant on three DL systems, representative of diverse DL tasks, namely sentiment analysis, digit recognition, and advanced driving assistance. These case studies differ in the forms of inputs (text, images, logical scenarios) and testing levels (model-level and system-level), resulting in different semantic-based input representations and XAI explanations. In our experiments, accounting for more than 2 k test cases, XMutant shows superior effectiveness and efficiency compared to DeepJanus , a state-of-the-art semantic-based test generator that has been

Read the complete text