Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks

Bin Duan, Matthew B. Dwyer, Guowei Yang · 2025

Read Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks by Bin Duan, Matthew B. Dwyer, Guowei Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17630.txt.

Introduction

Bin Duan, Matthew Dwyer, and Guowei Yang build ARGUS, a way to craft test inputs that trip up neural networks while still looking real. Older methods nudge pixels and hope; the results often look fake and miss faults. ARGUS works in a compressed latent space instead.

It blends a sample's latent code with codes from other classes, then decodes the mix back into an image. Two checker networks, one in latent space and one on pixels, keep the outputs plausible. In trials, ARGUS produced varied, realistic samples and perturbed every starting sample. It found faults up to four times more often than rivals.

Catalog record

CREATORBin Duan, Matthew B. Dwyer, Guowei Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17630

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Generating Realistic, Diverse, and Fault-Revealing Inputs with Latent Space Interpolation for Testing Deep Neural Networks Bin Duan, Matthew B. Dwyer, Guowei Yang Abstract Deep Neural Networks (DNNs) have been widely employed across various domains, including safety-critical systems, necessitating comprehensive testing to ensure their reliability. Although numerous DNN model testing methods have been proposed to generate adversarial samples that are capable of revealing faults, existing methods typically perturb samples in the input space and then mutate these based on feedback from the DNN model. These methods often result in test samples that are not realistic and with low-probability reveal faults. To address these limitations, we propose a black-box DNN test input generation method, Argus , to generate realistic, diverse, and fault-revealing test inputs. Argus first compresses samples into a continuous latent space and then perturbs the original samples by interpolating these with samples of different classes. Subsequently, we employ a vector quantizer and decoder to reconstruct adversarial samples back into the input space. Additionally, we employ discriminators both in the latent space and in the input space to ensure the realism of the generated samples. Evaluation of Argus in comparison with state-of-the-art black-box testing and white-box testing methods, shows that Argus excels in generating realistic and diverse adversarial samples relative to the target dataset, and Argus successfully perturbs all original samples and achieves up to 4 times higher error rate than the best baseline method. Furthermore, using these adversarial samples for model retraining can improve model classification accuracy. August 24, 2026 K eywords Deep Neural Networks (DNNs), Adversarial Sample Generation, Black-Box Testing 1 Introduction Deep Neural Network [ 29 ] (DNNs) have been widely applied in security-sensitive fields [ 26 ] and safety-critical systems [ 11 ] such as autonomous driving [ 25 ] , facial recognition [ 3 ] , and malware detection [ 12 ] . Like traditional software applications, these require extensive testing to ensure safe and reliable deployment [ 32 ] . However, the mechanism of DNNs is significantly different from that of traditional software. Traditional software testing primarily focuses on identifying erroneous outcomes caused by code defects [ 39 ] . In contrast, during training, a DNN model learns to compute feature embeddings that allow it to classify data associated with different labels [ 7 ] . Due to the diversity of data classes, the features of different classes vary, making DNNs predict the class with the highest likelihood of matching the input features [ 21 ] . When the input features are not distinct to those of other classes, DNNs may incorrectly predict another label [ 1 ] . Such errors are not due to internal flaws in the DNN model, but arise because the training process may not include such error-prone inputs that are similar to other classes. In this way, the mechanism of DNN training limits the effectiveness of traditional software testing methods [ 35 ] . To ensure the predictive accuracy of DNNs, various testing methods [ 15 , 27 , 24 , 37 , 28 , 10 , 41 , 17 ] have been proposed to identify model defects. When testing DNN models, the primary task is to generate samples that can reveal the erroneous behaviors of the model. Current DNN model testing methods can be divided into white-box and black-box testing. In white-box testing, existing methods [ 24 , 28 , 15 , 40 ] leverage neuron selection strategies to achieve high neuron coverage and identify samples that may lead to errors. They typically require internal knowledge of the target model, including its structure and parameters. Yet, the internal model information may not always be accessible in practice, especially in privacy-sensitive testing scenarios [ 18 , 2 ] . In contrast, black-box testing methods [ 27 , 37 , 17 , 10 ] do not require access to the internal details of the model. They generally identify model defects or inconsistencies by iteratively querying model outputs and updating test samples based on feedback. However, many of these methods employ fuzzing [ 27 , 15 , 37 ] to generate adversarial samples, which tend to produce a large number of invalid samples that do not cause misclassifications, reducing the success rate of generating adversarial samples [ 9 ] . Moreover, even though perturbation has been shown to be effective in generating adversarial samples, most methods directly conduct perturbations in the input space [ 27 , 15 , 28 , 24 , 41 , 37 , 17 ] and thus may generate samples whose features do not correspond to those represented by the original class. We contend that for black-box DNN test input generation methods to be usable in practice, they must generate samples that are: realistic and diverse relative to a target dataset, i.e., the dataset on which the model under test was trained; and that with high-probability reveal faults in models trained to state-of-the-art accuracy. Moreover, whereas white-box test generation approaches must generate new tests for each version of the model under test across its development cycle, for black-box testing the cost of test generation is less of a concern since new tests need only be generated when the underlying dataset shifts. These properties of black-box test generation approaches help to (a) ensure a low false positive rate – since realistic fault-revealing inputs are one’s developers will be concerned about; (b) better utilize testing efforts – since diverse fault-revealing inputs allow testers to minimize the triage of redundant tests and ensure a test run covers a breadth of behavior; and (c) increase the probability of a test sample causing an error – since a high-probability of failure means fewer tests need to be generated in order to reveal model faults. In this work, we propose Argus , that aims to generate realistic, diverse, and fault-revealing inputs to test DNN models. These adversarial samples are not only closely similar to the original samples but also effectively mislead the model’s predictions, causing classification errors across multiple classes.. Specifically, Argus utilizes, adapts, and extends a VQ-VAE [ 36 ] , a generative model where an encoder compresses complex input samples, images, into a low-dimensional latent space, allowing the model to learn the features of the training data and obtain a suitable representation for reconstruction. After obtaining the latent representation of the sample, we introduce perturbations by interpolating with the latent representations of samples from other classes, using a perturbation factor \lambda to control the degree of the perturbation. The perturbed latent representation is then reconstructed back to a sample in the input space. Direct perturbations in the input space often lead to reduced realism and a large number of samples that fail to reveal model faults [ 5 , 9 , 30 ] . Unlike previous DNN model testing methods that directly perturb images in the input space, Argus perturbs in the latent space, ensuring that the generated samples are realistic. Since these perturbations occur in the latent space, they are highly inconspicuous, and the generated adversarial samples exhibit slight differences from the original samples. Moreover, Argus incorporates discriminators, which are trained to force the perturbation process to remain close to the data distribution, which significantly improves the realism of generated samples. Additionally,

Read the complete text