Imitation Game for Adversarial Disillusion with Multimodal Generative Chain-of-Thought Role-Play

Ching-Chun Chang, Fan-Yun Chen, Shih-Hong Gu, Kai Gao, Hanrui Wang, Isao Echizen · 2025

Read Imitation Game for Adversarial Disillusion with Multimodal Generative Chain-of-Thought Role-Play by Ching-Chun Chang, Fan-Yun Chen, Shih-Hong Gu, Kai Gao, Hanrui Wang, Isao Echizen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19143.txt.

Introduction

Ching-Chun Chang and five co-authors guard AI perception with an imitation game against fake inputs. Fakes come in two kinds: crafted inputs that fool a set model, and trap doors planted in training that fire on a cue. Their 2025 shield sends each input to a generative agent that reads it, takes in its sense, and builds it back clean. The rebuild keeps the sense, not the trap.

The agent works in words and images at once, and thinks in steps as it reads. Simulations ran the shield on white-box and black-box fakes of both kinds. In each case the shield washed the fake out and the model read true. The team frames sense-first rebuilds as one guard for both fake kinds, where past guards split by kind.

Catalog record

CREATORChing-Chun Chang, Fan-Yun Chen, Shih-Hong Gu, Kai Gao, Hanrui Wang, Isao Echizen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-19143

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Imitation Game for Adversarial Disillusion with Multimodal Generative Chain-of-Thought Role-Play Ching-Chun Chang, Fan-Yun Chen, Shih-Hong Gu, Kai Gao, Hanrui Wang, Isao Echizen Abstract As the cornerstone of artificial intelligence, machine perception confronts a fundamental threat posed by adversarial illusions. These adversarial attacks manifest in two primary forms: deductive illusion, where specific stimuli are crafted based on the victim model’s general decision logic, and inductive illusion, where the victim model’s general decision logic is shaped by specific stimuli. The former exploits the model’s decision boundaries to create a stimulus that, when applied, interferes with its decision-making process. The latter reinforces a conditioned reflex in the model, embedding a backdoor during its learning phase that, when triggered by a stimulus, causes aberrant behaviours. The multifaceted nature of adversarial illusions calls for a unified defence framework, addressing vulnerabilities across various forms of attack. In this study, we propose a disillusion paradigm based on the concept of an imitation game. At the heart of the imitation game lies a multimodal generative agent, steered by chain-of-thought reasoning, which observes, internalises and reconstructs the semantic essence of a sample, liberated from the classic pursuit of reversing the sample to its original state. As a proof of concept, we conduct experimental simulations using a multimodal generative dialogue agent and evaluates the methodology under a variety of attack scenarios. I Introduction Machine perception refers to the capability of a computational system to process data in a way that mimics how humans use their senses to interpret and interact with the world around them [ 1 , 2 , 3 ] . Adversarial illusions pose an insidious threat to the safety of machine perception with wide-ranging catastrophic consequences in economy, politics and society [ 4 , 5 , 6 , 7 ] . These attacks typically manifest in two distinct forms. One is the exploitation of the vulnerabilities in a model’s decision boundaries to craft an illusory stimulus that appears imperceptible to humans yet elicits deviant decisions from the model [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 ] . Another is the poisoning of learning data to implant a backdoor within a model, conditioning it towards erroneous behaviours when triggered by an illusory stimulus [ 18 , 19 , 20 , 21 , 22 , 23 , 24 ] . We refer to the former as deductive illusion , reflecting that specific stimuli are derived from the victim model’s general decision logic. We refer to the latter as inductive illusion , reflecting that the victim model’s general decision logic is shaped by specific stimuli. Together, these adversarial illusions represent a fundamental attack paradigm that jeopardises the trustworthiness of artificial intelligence. Adversarial disillusion is the process of neutralising illusory stimuli, thereby minimising the risks of diverged behaviours [ 25 , 26 , 27 , 28 , 29 , 30 ] . In essence, illusory stimuli can be regarded as additive noise, making it natural to relate their solutions to denoising methods [ 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ] . However, with the virtually limitless forms that illusory stimuli can take, it is impractical to account for every possible variation through tailored denoising methods. Addressing such multifaceted adversarial illusions calls for a unified defence framework that transcends the boundaries of individual attack forms. In this paper, we propose a disillusion paradigm motivated by the concept of an imitation game for a universal defence against both deductive and inductive illusions. At its core lies multimodal generative agent [ 40 , 41 , 42 , 43 , 44 , 45 , 46 ] . Through the interpretative power of chain-of-thought reasoning, the multimodal generative agent is guided to observe, internalise and reconstruct the essential features of a given sample [ 47 , 48 , 49 ] . We formulate this process as an imitation game, in which the multimodal generative agent plays the role of an imitator [ 50 , 51 , 52 , 53 , 54 ] . While previous disillusion methods typically aim to restore the sample to its original state, prioritising perceptual similarity between the neutralised and original samples, we argue that adherence to fidelity constraint is not always indispensable and may hinder broader exploration of potential solutions. Rethinking the core principles of disillusion, we conceptualise an imitation game as an outside-the-box solution. As a proof of concept, we validate this solution with ChatGPT, a multimodal generative chatbot developed by OpenAI, through experimental simulations in the presence of various adversarial attacks. The remainder of this paper is organised as follows. Section II provides a review of representative attack and defence methods, which serve as benchmarks for this study. Section III formulates the problem and presents the proposed methodology. Section IV evaluates the performance and discusses the experimental results. Finally, Section V concludes the paper with a summary of the research findings and potential directions for future research. II Preliminaries In this section, we outline the foundational concepts that define the landscape of adversarial attacks and defences. We present a taxonomy that systematically categorises adversarial illusions based on their nature. Following this, we briefly review representative methods from both the attack and defence domains, which serve as key benchmarks in this study. II-A Inference-Time Deductive Illusion Deductive reasoning involves forming specific conclusions from general premises. Analogously, deductive illusion entails crafting illusory stimuli derived from the victim model’s decision boundary in the inference time. In this context, a malicious sample, denoted as x^{\prime} , is generated by adding a deductive stimulus \delta to a benign sample x , expressed as: x^{\prime}=x+\delta. (1) These stimuli are optimised to mislead the model’s decision process while remaining imperceptible to human observers. Imperceptibility is governed by two fundamental constraints: the magnitude of distortion and the multitude of distorted components. The stimulus is generated by maximising the loss function or the distance between the model’s prediction on the malicious sample and the ground truth, formulated as: \delta=\arg\max_{\delta}\mathcal{L}(f(x+\delta),y), (2) subject to a magnitude constraint on the maximum allowable distortion magnitude \epsilon_{\text{mag}} in the \ell_{\infty} -norm: \|\delta\|_{\infty}\leq\epsilon_{\text{mag}}, (3) or a multitude constraint on the maximum allowable number of distorted components \epsilon_{\text{mul}} : \|\delta\|_{0}\leq\epsilon_{\text{mul}}. (4) Under the magnitude constraint, a foundational algorithm is the fast gradient sign method (FGSM) [ 9 ] , which exploits the gradient of a loss function \mathcal{L} with respect to the input sample. The stimulus is computed as: \delta=\epsilon_{\text{mag}}\cdot\operatorname{sign}\left(\nabla_{x}\mathcal{L}(f(x),y)\right), (5) where \operatorname{sign} represents the signum function, taking the value +1 , -1 or 0 , depending on whether the gradient is positive, negative or zero. An iterative extension of this approach, known as projected gradient descent (PGD) [ 13 ] , updates the stimulus at each iteration as: \delta=\operatorname{clip}_{\epsilon_{\text{mag}}}\left(\alpha\cdot\operatorname{sign}\left(\nabla_{x^{(t)}}\mathcal{L}(f(x^{(t)}),y)\right)\right), (6) where \operatorname{clip}_{\epsilon} denotes the clipping operator, ensuring the stimulus remains within the allowable \ell_{p} -norm ball of radius \epsilon , \alpha represents the step size, and x^{(t)} represents the input at the t -th iteration. Several variations of this method have been proposed. One notable strategy, termed the diverse inputs iterative fast gradient sign method (DI {}^{\text{2}} -FGSM) [ 14 ] , enhances robustness against defensive measures by applying random

Read the complete text