Feature-Aware Test Generation for Deep Learning Models DOI: XXXXXXX.XXXXXXX

Xingcheng Chen, Oliver Weißl, Andrea Stocco · 2026

Read Feature-Aware Test Generation for Deep Learning Models DOI: XXXXXXX.XXXXXXX by Xingcheng Chen, Oliver Weißl, Andrea Stocco (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-14081.txt.

Introduction

Xingcheng Chen, Oliver Weißl, and Andrea Stocco build tests that explain why deep learning models fail in 2026. Test generation plays a crucial role in assessing model quality before deployment. The most advanced generators rely on generative AI to synthesize inputs. They remain limited in giving semantic insight into misbehavior causes.

They also lack fine-grained semantic controllability over generated inputs. The team's feature-aware test generation targets those gaps. Tests tied to clear features show which trait trips a model. Developers can then fix the flaw, not just count its effects.

Catalog record

CREATORXingcheng Chen, Oliver Weißl, Andrea Stocco
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-14081

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature-Aware Test Generation for Deep Learning Models DOI: XXXXXXX.XXXXXXX Xingcheng Chen, Oliver Weißl, Andrea Stocco Abstract . As deep learning models are widely used in software systems, test generation plays a crucial role in assessing the quality of such models before deployment. To date, the most advanced test generators rely on generative AI to synthesize inputs; however, these approaches remain limited in providing semantic insight into the causes of misbehaviours and in offering fine-grained semantic controllability over the generated inputs. In this paper, we introduce Detect , a feature-aware test generation framework for vision-based deep learning (DL) models that systematically generates inputs by perturbing disentangled semantic attributes within the latent space. Detect perturbs individual latent features in a controlled way and observes how these changes affect the model’s output. Through this process, it identifies which features lead to behavior shifts and uses a vision-language model for semantic attribution. By distinguishing between task-relevant and irrelevant features, Detect applies feature-aware perturbations targeted for both generalization and robustness. Empirical results across image classification and detection tasks show that Detect generates high-quality test cases with fine-grained control, reveals distinct shortcut behaviors across model architectures (convolutional and transformer-based), and bugs that are not captured by accuracy metrics. Specifically, Detect outperforms a state-of-the-art test generator in decision boundary discovery and a leading spurious feature localization method in identifying robustness failures. Our findings show that fully fine-tuned convolutional models are prone to overfitting on localized cues, such as co-occurring visual traits, while weakly supervised transformers tend to rely on global features, such as environmental variances. These findings highlight the value of interpretable and feature-aware testing in improving DL model reliability. 1. Introduction Deep learning models such as deep neural networks (DNNs) and transformers have achieved impressive performance on many computer vision benchmarks ( 9 ) , and they are widely deployed in many software systems nowadays. However, this success often fails to translate to real-world deployments, where models encounter inputs that differ, sometimes subtly, from those seen during training ( 37 ; 53 ) . These failures arise from two main challenges, namely limited generalization to unseen but valid inputs, and lack of robustness to spurious correlations that the model mistakenly relies on. Conceptually, these problems are related to the empirical risk minimization paradigm ( 47 ) , which encourages DL models to optimize average performance over the training distribution, regardless of whether the features they exploit are semantically meaningful. As a result, models often learn shortcut solutions —spurious correlations tied to background, lighting, pose, or co-occurring objects—that work on the training set but fail when exposed to real-world data distributions ( 12 ; 15 ) . While deriving from the same root problem, in literature, these two challenges have been investigated separately. While test generation techniques aim to evaluate model generalization by synthesizing new inputs that may uncover erroneous behaviors ( 35 ; 54 ) , they often lack explainability and perturbed multiple features in an entangled manner (e.g., by injecting global Gaussian noise or simultaneously modifying lighting, texture, and object semantics). Therefore, generated tests uncover failures , but they do not clarify which feature changes caused the failure, or whether those changes are semantically relevant to the task. Conversely, spurious feature analysis focuses on robustness . They investigate whether models rely on invariant, task-irrelevant features, but typically through reactive post-hoc explanation tools such as saliency maps ( 41 ; 5 ) . This separation between input synthesis and feature diagnosis limits effective test generation and DL model improvements . To fill this gap, in this paper, we propose Detect , an explainable test generation framework that links test generation and spurious feature identification by systematically testing DNN behaviors under semantically controlled perturbations . Our approach is based on the insight that both generalization and robustness concerns can be addressed through feature-aware changes in the disentangled latent space: by precisely manipulating individual semantic features, either relevant or irrelevant to the task , Detect assesses not only if the DNN model generalizes to novel inputs by varying the relevant features, but also whether it consistently misbehaves to irrelevant ones. To achieve this, Detect leverages a style-based generative model trained on the same distribution as the system under test (SUT), which provides access to a disentangled latent space of semantic features. By perturbing latent dimensions, we generate realistic input variants with fine-grained changes in specific attributes. This enables feature-aware test generation , by perturbing task-relevant features to explore decision boundaries to uncover untested regions or modifying task-irrelevant features to assess prediction invariance and identify spurious dependencies. Detect incorporates a feature-aware test oracle , which quantifies model response to controlled perturbations using a logit-based criterion. If the model’s prediction significantly shifts after modifying a semantically irrelevant feature, it is flagged as a robustness failure due to spurious correlation. Conversely, changes in response to relevant features are used to test generalizability. In our empirical evaluation on image classification and detection tasks, Detect outperforms existing test generation and spurious feature localization methods both in revealing feature-sensitive failures, as well as robustness failures due to task-irrelevant features. Detect reveals spurious correlations in high-accuracy models that standard accuracy metrics overlook, and distinguishes between different shortcut behaviors learned by weakly and fully supervised models. On fine-grained tasks, Detect also serves as an effective general test generator, offering flexible applicability across model types and data domains. Our contributions are as follows: 1) We propose Detect , a feature-aware testing framework that perturbs semantically meaningful latent features to evaluate both generalization and robustness; 2) We introduce a feature-aware test oracle that dually checks invariance for irrelevant features (robustness) and explores decision boundaries for relevant features (generalization); 3) We show that test generation and spurious feature analysis, though distinct in goal, can be combined into a unified methodology for probing, explaining, and repairing DNN defects. 2. Background Figure 1 . System Architecture of Detect : It comprises two main components, a StyleGAN generator and the SUT, and operates across three domains: the latent \mathcal{S} -space, the image space, and the output space. It contains 1) a forward generation pass from an initial seed z mapped into a set of style vectors s , which synthesize an image input passed to the SUT to obtain output logits; and 2) a backpropagation through SUT and synthesis network back to s . In the context of deep learning testing, generative AI techniques are used due to their ability to model complex, feature-rich data distributions and synthesize new functional inputs through latent space manipulation ( 40 ; 29 ; 10 ; 49 ) . Among these, Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and diffusion models are widely used for image generation ( 29 ) . However, each comes with limitations for test generation purposes. VAEs, while structurally interpretable, tend to produce low-fidelity and therefore low quality input images. Conventional GANs suffer from entangled latent spaces, making semantic control over generated features difficult. Diffusion models can generate high-quality outputs but are computationally expensive, and also lack disentanglement in their latent representations. Among these, recent approaches based on style-based GANs (StyleGAN) ( 21 ; 22 ; 20 ) , offer high controllability due to disentangled and interpretable latent spaces. These models enable

Read the complete text