A Grey-box Text Attack Framework using Explainable AI
Esther Chiramal*, Kelvin Soh Boon Kai* · 2025
Read A Grey-box Text Attack Framework using Explainable AI by Esther Chiramal*, Kelvin Soh Boon Kai* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08226.txt.
Introduction
Esther Chiramal and Kelvin Soh Boon Kai wrote this security paper in 2025 on attacking text classifiers. Tools that explain a model's call, showing which words swayed it, build trust. The same maps can guide an attacker to the words worth swapping. Classic attacks need inside access or make swaps a human can spot, like flipping a word's meaning outright.
Their grey-box method needs no view of the target. It attacks stand-in BERT models with explainable maps, then carries the crafted text to other BERT models. The shared attention design of transformers makes the trick travel. Swapping as little as one word, in a way readers miss, still fooled the targets. The authors warn that openness about model logic cuts both ways.
Catalog record
| CREATOR | Esther Chiramal*, Kelvin Soh Boon Kai* |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08226
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Grey-box Text Attack Framework using Explainable AI Thanks: ∗ * Equal contribution. Listing order is alphabetical order. E. Chiramal and K. Soh Boon Kai, work are done during our stint at Amaris.AI Pte Ltd a AI-Cybersecurity Company based in Singapore Esther Chiramal*, Kelvin Soh Boon Kai* Abstract Explainable AI is a strong strategy implemented to understand complex black-box model predictions in a human interpretable language. It provides the evidence required to execute the use of trustworthy and reliable AI systems. On the other hand, however, it also opens the door to locating possible vulnerabilities in an AI model. Traditional adversarial text attack uses word substitution, data augmentation techniques and gradient-based attacks on powerful pre-trained Bidirectional Encoder Representations from Transformers (BERT) variants to generate adversarial sentences. These attacks are generally white-box in nature and not practical as they can be easily detected by humans E.g. Changing the word from “Poor” to “Rich”. We proposed a simple yet effective Grey-box cum Black-box approach that does not require the knowledge of the model while using a set of surrogate Transformer/BERT models to perform the attack using Explainable AI techniques. As Transformers are the current state-of-the-art models for almost all Natural Language Processing (NLP) tasks, an attack generated from BERT1 is transferable to BERT2. This transferability is made possible due to the attention mechanism in the transformer that allows the model to capture long-range dependencies in a sequence. Using the power of BERT generalisation via attention, we attempt to exploit how transformers learn by attacking a few surrogate transformer variants which are all based on a different architecture. We demonstrate that this approach is highly effective to generate semantically good sentences by changing as little as one word that is not detectable by humans while still fooling other BERT models. 1 Introduction Adversarial attacks are an important and serious issue in the AI-Cybersecurity landscape. It opens doors for attackers to exploit the loopholes of AI models by misleading the AI model’s decision process. Due to the complexity of AI models it is difficult to determine if an adversarial attack has taken place leading to misclassified results. Adversarial attacks can fool facial recognition systems to approve authorised documents or modify images used in autonomous driving systems to cause accidents [ 1 ] . Such misuses of AI can violate human values and cause detrimental impacts when used in industries having a direct connection to an individual’s lifestyle. The black-box nature of AI models pushes forth the requirement of having valid explanations for decisions made by AI systems. This is achieved through the concept of Explainable AI (XAI) that focuses on providing reasonings in a human interpretable form [ 2 , 3 ] . Various explainable AI strategies that provide user-friendly visualisations have been implemented, mediating the issue of lack of transparency. The reasoning provided includes highlighting areas of a given data that contribute greatly to the final prediction made by AI systems. It is a powerful strategy used to instil trust in individual predictions and at the same time detect anomalies within the system. Text classification, a machine learning technique, is one of the fundamental tasks in Natural Language Processing (NLP) and has various applications ranging from sentiment analysis to spam detection. The experiments in this paper are conducted on sentiment classifications incorporated with explainable AI strategies. As mentioned, Explainable AI is a powerful tool required for any decision making AI system. However, by opening the ‘black-box’ nature of AI systems and providing explanations, it reveals existing vulnerabilities within the system to all users. In this paper, we propose an algorithm to exploit the advantages of existing Explainable AI frameworks and construct an adversarial attack on AI models. The main contributions of the paper are summarised as follows. • Producing sentence variations by replacing words with synonyms of a given text based on information received through XAI strategies. • Robustness of model and attack transferability between transformers. 2 Related Work 2-A TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP TextAttack is a framework to craft adversarial attacks in NLP. The framework is catered to work with various NLP models and Transformers that can be implemented through the use of traditional neural network and HuggingFace models [ 5 ] . TextAttack contains various algorithms to perform adversarial attack on classification tasks such as DeepWordBug, HotFlip, PWWS, TextBugger, and TextFooler. Most of the algorithms follow the idea of iterating through all possible combinations of potential transformations to find a sequence of successful alterations that can form adversarial samples. White-box and black-box transformations can be applied using TextAttack where white-box indicates that access to the model is available and black-box is conducted without any knowledge. White-box permutations allow attackers to determine words to change based on the model gradients; however, this is not possible for black-box transformations. Figure 1: TextAttack framework for adversarial attacks, data augmentation, and model training in NLP To conduct a fair comparison, we tested the TextAttack framework using a transformer model from HuggingFace pre-trained on the ‘IMDB’ dataset. Probability Weighted Word Saliency (PWWS) attack was implemented on the test samples since it generates sentences less likely to be detected by humans. PWWS is word substitution order introduced in 2019 [ 6 ] which aims to generate adversarial examples that maintain lexical correctness, grammatical correctness and semantic similarity. It is a technique where words are swapped with their synonyms based on their saliency score and maximum swap effectiveness. Original Text Adversarial Text throws in enough clever and unexpected twists to make the formula feel fresh throws in enough clever and unexpected twists to crap the formula feel fresh possibility of bankruptcy. lack of assurance. Poor stability. theory of bankruptcy. want of sureness . Poor stability. the story gives ample opportunity for large scale action and suspense, which director shekhar kapur supplies with tremendous skill the story gives ample opportunity for large scale action and suspense, which director shekhar kapur supplies with terrible skill Table 1: Comparison of original samples with their adversarial substitutions using PWWS. As shown in Table 1 , successful adversarial samples are generated after swapping one or more words in the sentence. The saliency score tells us the contribution of the original word to the final prediction. The word is replaced if its contribution changes depending on its word saliency. 2-B Grey-Box Adversarial Attack and Defence for Sentiment Classification Grey-box text attack is not a well studied area, as performing an attack with limited to no knowledge of the model can be challenging. A paper was released by researchers at IBM that proposes a framework to produce high quality adversarial samples while simultaneously training a classifier for adversarial defence [ 7 ] . The architecture makes use of a generator which can be implemented as an autoencoder or a paraphrase generator. This generator is fed into two copies of a pre-trained classifier where one copy is updated during the training process with adversarial samples while the second copy is used to predict labels of the adversarial texts. The framework however, requires details regarding the model architecture and weights apriori and hence cannot be directly applied to other classifier models. As shown in Figure 2 , the