A Contrastive Teacher-Student Framework for Novelty Detection under Style Shifts

Hossein Mirzaei, Mojtaba Nafez, Moein Madadi *, Arad Maleki *, Mahdi Hajialilue, Zeinab Sadat Taghavi, Sepehr Rezaee, Ali Ansari, Bahar Dibaei Nia, Kian Shamsaie, Mohammadreza Salehi, Mackenzie W. Mathis, Mahdieh Soleymani Baghshah, Mohammad Sabokrou, Mohammad Hossein Rohban · 2025

Read A Contrastive Teacher-Student Framework for Novelty Detection under Style Shifts by Hossein Mirzaei, Mojtaba Nafez, Moein Madadi *, Arad Maleki *, Mahdi Hajialilue, Zeinab Sadat Taghavi, Sepehr Rezaee, Ali Ansari, Bahar Dibaei Nia, Kian Shamsaie, Mohammadreza Salehi, Mackenzie W. Mathis, Mahdieh Soleymani Baghshah, Mohammad Sabokrou, Mohammad Hossein Rohban (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17289.txt.

Introduction

Hossein Mirzaei and a large team of co-authors present a 2025 method for spotting novel inputs to AI systems even when the style of the data shifts. Detectors of this kind learn only from normal data, so they often latch onto style traits, like lighting or texture, instead of the core content. A small style shift then breaks them, which is risky in self-driving cars and medical scans.

The team builds a fake set of odd samples that share the style of the normal set but differ in core content. A teacher-student training scheme then teaches the detector to judge by core traits, not style. Tests on made-up and real data sets, against nine rival methods, backed the design. The authors argue that core-based judging is the path to detectors that hold up in the wild.

Catalog record

CREATORHossein Mirzaei, Mojtaba Nafez, Moein Madadi *, Arad Maleki *, Mahdi Hajialilue, Zeinab Sadat Taghavi, Sepehr Rezaee, Ali Ansari, Bahar Dibaei Nia, Kian Shamsaie, Mohammadreza Salehi, Mackenzie W. Mathis, Mahdieh Soleymani Baghshah, Mohammad Sabokrou, Mohammad Hossein Rohban
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE96 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17289

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Contrastive Teacher-Student Framework for Novelty Detection under Style Shifts Hossein Mirzaei, Mojtaba Nafez, Moein Madadi *, Arad Maleki *, Mahdi Hajialilue, Zeinab Sadat Taghavi, Sepehr Rezaee, Ali Ansari, Bahar Dibaei Nia, Kian Shamsaie, Mohammadreza Salehi, Mackenzie W. Mathis, Mahdieh Soleymani Baghshah, Mohammad Sabokrou, Mohammad Hossein Rohban Abstract There have been several efforts to improve Novelty Detection (ND) performance. However, ND methods often suffer significant performance drops under minor distribution shifts caused by changes in the environment, known as style shifts. This challenge arises from the ND setup, where the absence of out-of-distribution (OOD) samples during training causes the detector to be biased toward the dominant style features in the in-distribution (ID) data. As a result, the model mistakenly learns to correlate style with core features, using this shortcut for detection. Robust ND is crucial for real-world applications like autonomous driving and medical imaging, where test samples may have different styles than the training data. Motivated by this, we propose a robust ND method that crafts an auxiliary OOD set with style features similar to the ID set but with different core features. Then, a task-based knowledge distillation strategy is utilized to distinguish core features from style features and help our model rely on core features for discriminating crafted OOD and ID sets. We verified the effectiveness of our method through extensive experimental evaluations on several datasets, including synthetic and real-world benchmarks, against nine different ND methods. The code repository is available at: https://github.com/rohban-lab/CTS . 1 Introduction Novelty detection (ND) has emerged as a critical component in developing reliable real-world machine learning models. The primary task of ND is to distinguish Out-of-distribution (OOD) samples from the in-distribution (ID) samples during inference, using only unlabeled ID samples for training ( 5 ; 75 ; 84 ; 110 ) . This task is essential across various computer vision applications, including industrial defect detection, medical disease screening, and video surveillance ( 46 ; 82 ; 108 ; 84 ) . However, these methods often experience significant performance drops when confronted with test data exhibiting minor distribution shifts in their style , such as changes in the test sets due to environmental variations (See Figure 1 ) ( 13 ; 11 ; 95 ; 96 ; 20 ) . A robust detector should be invariant to changes in the style features, as variations in these features do not change a sample’s label (ID or OOD). Instead, it should be expected to learn the core features which determine the label ( 3 ; 61 ; 13 ) . Robustness against style shifts is a crucial aspect of ND methods since variations in style are common in real-world applications. For instance, an ND method for autonomous driving tasks trained on images from Germany streets ( 21 ) should also perform effectively on the streets of Los Angeles ( 81 ) , despite variations in style features caused by different lighting and atmospheric conditions. A similar challenge exists in medical imaging, where shifts can occur due to different imaging equipment, patient positioning, and variations in tissue properties ( 109 ) . The vulnerability of existing ND methods stems from their implicit assumption that the training data should strongly mirror the test data, even in stylistic features. This leads to the misprediction of an ID test sample with a different style feature as OOD. Furthermore, training data in the ND setup is limited to ID samples. By relying solely on ID samples, the detector learns a correlation between the dominant style features present in ID samples and the label. Consequently, the detector mistakenly uses these style features for discrimination instead of focusing on core features. As a result, the detector incorrectly predicts an ID test sample with a different style as OOD and an OOD sample with a similar style as ID ( 13 ) . Notably, current domain generalization and domain adaptation methods cannot be applied to develop robust ND methods against distribution shifts, as they require access to labeled training data or extra data from different environments, which are not available in the ND setup ( 42 ; 76 ; 113 ; 117 ; 71 ; 123 ; 119 ) . Furthermore, our study distinguishes itself from recent works such as RedPanda ( 20 ) and PCIR ( 13 ) , which leverage different environmental annotations as additional information to improve the ND robustness. In many real-world scenarios, ID training samples are collected from unknown environments, and hence such metadata is often missing ( 11 ; 87 ) . Motivated by these challenges, we propose crafting an auxiliary OOD set by identifying the core features of the ID samples and distorting them. To identify the core features, we employ a feature attribution method (Grad-CAM ( 90 ) ) applied on the output of a pre-trained network when fed with the ID samples. We apply light augmentations (e.g., color jitter ( 16 ; 37 ; 38 ) ) to the input, and compute saliency maps for both the original and augmented versions. By taking the element-wise product of these saliency maps, we derive a final saliency map where higher values correspond to the core features of the assumed ID sample. These light augmentations facilitate producing a final saliency map agnostic to style shifts. Subsequently, hard transformations ( 55 ; 100 ; 100 ; 97 ; 73 ; 22 ) (e.g., elastic transformation) are applied to regions of the assumed ID sample with higher saliency values, ensuring robustness against style shifts. Given the crafted OOD set and ID set, we apply light augmentation to each set while maintaining the labels to provide various style shifts to each set. To effectively leverage information from the created sets and develop a robust ND pipeline, we introduce a task-based knowledge distillation strategy ( 32 ) . Specifically, we use a pre-trained encoder concatenated with a trainable binary classification layer as the teacher and a model trained from scratch as the student. We train the teacher to classify the created ID and OOD sets while only updating the binary layer. Then, using a novel objective function, we force the student to align its output with the teacher when the input is an ID sample and to diverge from the teacher when the input is an OOD sample. The discrepancy between the student and teacher outputs will be utilized as the OOD score at inference time. Our approach is inspired by knowledge distillation, which has proven effective for ND tasks compared to other strategies ( 19 ; 88 ; 23 ; 34 ; 107 ; 11 ) . Notably, our method achieves superior performance compared to both previous knowledge distillation-based and other ND methods, underscoring the effectiveness of our pipeline. Contributions: In this study, we propose a novel data-centric approach along with a new pipeline to achieve a robust and meta-data free ND method. Our strategy, by providing augmented samples obtained through applying style shifts while retaining labels, achieves a more robust representation of distribution shifts. Moreover, through intervening ID samples by identifying and distorting their core regions, we reach synthesized OOD samples. Such samples are then leveraged to make our model more sensitive to the core features. From a causal viewpoint (Refer

Read the complete text