Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection

Jianhong Han, Yupei Wang *, Liang Chen · 2025

Read Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection by Jianhong Han, Yupei Wang *, Liang Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20498.txt.

Introduction

Jianhong Han, Yupei Wang, and Liang Chen teach an object detector to dress for weather it has never seen in a 2025 paper. The training budget holds one source domain only, yet the detector must work in fog, rain, night, and glare it never trained on. Older fixes augment data and align features, which helps only when fake weather resembles the real thing.

Their Style-Adaptive Detection Transformer works at test time. A style adapter keeps a memory bank of style prototypes that organizes itself as new scenes arrive, projecting each unseen style back toward the source look. An object-aware contrastive module, fenced by gating masks in space and meaning, pulls out features that ignore style altogether. Across five weather scenarios, the detector beat prior methods on accuracy and generalization alike.

Catalog record

CREATORJianhong Han, Yupei Wang *, Liang Chen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-20498

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Style-Adaptive Detection Transformer for Single-Source Domain Generalized Object Detection Jianhong Han, Yupei Wang *, Liang Chen Abstract Single-source Domain Generalization (SDG) in object detection aims to develop a detector using only data from source domain that can exhibit strong generalization capability when applied to other unseen target domains. Existing methods are built upon CNN-based detectors and primarily improve their robustness by employing carefully designed data augmentation strategies integrated with feature alignment techniques. However, the data augmentation methods have inherent drawbacks that are only effective when the augmented sample distribution approximates or covers the unseen scenarios, thus failing to enhance the detector’s generalization capability across all unseen scenarios. Furthermore, while the recent DEtection TRansformer (DETR) has demonstrated superior generalization capability in domain adaptation tasks due to its efficient global information extraction, its potential for SDG tasks remains unexplored. To this end, we introduce a strong DETR-based detector named the Style-Adaptive DEtection TRansformer (SA-DETR) for SDG in object detection. Specifically, we present an online domain style adapter to project the style representation of the unseen target domain into the training domain. This adapter maintains a dynamic memory bank that self-organizes onto diverse style bases and continuously absorbs information from unseen domains within a test-time adaptation framework, ultimately delivering precise style adaptation. Then, the object-aware contrastive learning module is proposed to force the detector to extract domain-invariant features of objects through contrastive learning. By employing carefully designed object-aware gating masks to constrain the scope of feature aggregation in both spatial position and semantic category, this module effectively achieves cross-domain contrast of instance-level features, thereby further enhancing the detector’s generalization capability. Extensive experimental results demonstrate the superior performance and generalization capability of our proposed SA-DETR across five different weather scenarios. Code is released at https://github.com/h751410234/SA-DETR . I Introduction The task of Single-source Domain Generalization (SDG) is designed to mitigate domain gaps that arise from distribution differences between the training data (source domain) and real-world application data (target domain). Unlike traditional unsupervised domain adaptation tasks [ 1 , 2 , 3 ] , which allow the use of images from the target domain, SDG enhances the robustness of networks to various unseen target scenarios by using only data from a single-source domain. Although this task is more difficult, it has been receiving increasing attention because it better aligns with practical deployment requirements, such as autonomous driving. Recently, numerous single-source domain generalization (SDG) methods [ 4 , 5 , 6 ] for object detection have been proposed, employing carefully designed generalization strategies to enhance detector robustness, yet they still suffer from two major limitations. First, to overcome the limited sample space of single-domain data, existing methods typically employ innovative data augmentation strategies (at the image or feature level) to obtain more diverse samples for training, thereby enhancing the generalization capability of detectors for unseen scenarios. These methods have inherent drawbacks that are only effective when the augmented sample distribution approximates or covers the unseen scenarios. Due to the unpredictable deployment environments, which are likely to have significant differences from the augmented samples, the improvement in detector performance is minimal in such cases. Second, another line of research tackles this issue through style modulation, suppressing domain‑specific styles via instance normalization [ 7 ] and feature whitening [ 8 ] , or rectifying input images with simply constructed style bases [ 9 , 10 ] . Such a static treatment overlooks the high diversity and dynamic nature of styles in the wild. Consequently, it remains insufficiently adaptive when confronted with large domain gaps. Moreover, researchers have primarily explored and demonstrated effectiveness within the Faster R-CNN [ 11 ] , while transformer-based detectors have been less studied. Recent works [ 12 , 13 , 14 ] increasingly shows that detectors based on the DEtection TRansformer (DETR) [ 15 ] exhibit superior performance in handling domain gaps compared to those based on Convolutional Neural Networks (CNNs). Unlike purely convolutional designs, DETR innovatively integrates a CNN backbone with transformer architecture (i.e., encoder and decoder), leveraging the attention mechanism to refine features for more detailed representations [ 16 ] . Additionally, the transformer architecture has been proven to be highly effective in capturing global structural information [ 17 , 18 , 19 ] , potentially offering better generalization capability for unseen scenarios. Therefore, it is highly meaningful to explore the effectiveness of DETR-based detectors in addressing the SDG task. In this paper, we introduce a strong DETR-based detector named the Style-Adaptive DEtection TRansformer (SA-DETR) for SDG in object detection. To better adapt to unseen scenarios, we propose the Online Domain Style Adapter (ODS-Adapter), which addresses the domain gap through statistical matching rather than data augmentation. During training, the adapter employs a dynamic style memory bank to cache the channel-wise statistics (mean and variance) of backbone features and treats them as style bases, consistent with the classic style-transfer paradigm [ 20 ] . As iterations progress, similar bases are integrated through momentum-based updates, whereas redundant ones are discarded, allowing the memory bank to self-organize into multiple style bases that faithfully capture the diversity of the training data. During inference, the method first employs the Wasserstein distance [ 21 ] to quantify distribution discrepancies between the statistics of the testing image and the stored style bases. Next, we convert the distances into weights, which are injected into an AdaIN-like modulation to achieve fast and precise feature projection. Considering that the target domain may exhibit entirely novel styles, the adapter writes the newly observed statistics back to the memory bank during testing, thereby continuously absorbing information from unseen domains and preventing under-adaptation caused by large style gaps. To further enhance the generalization capability, we have introduced an object-aware contrastive learning module that forces the detector to extract domain-invariant features of objects through contrastive learning. Differing from the proposed adapter that addresses the SDG challenge at the image level, this module employs carefully designed object-aware gating masks to constrain the scope of feature aggregation in both spatial position and semantic category, thereby effectively achieving cross-domain contrast (i.e., between the source domain and the augmented domain) of instance-level features. Specifically, we insert multiple class queries into the transformer encoder, with each query responsible for capturing features of a specific category. Utilizing object-aware gating masks created from annotations, these queries adaptively aggregate specific positional and categorical features from the feature tokens by leveraging a cross-attention mechanism. Then, we introduce a contrastive learning to minimize the differences among these aggregated queries, which can be considered instance-level prototypes for each category. By attracting prototypes of the same category and repelling those of different categories across domains, the detector is forced to extract domain-invariant object features, thereby enhancing its generalization capability. The main contributions of this paper are as follows: • We show that DETR-based detectors exhibit better generalization in unseen scenarios, and propose a style-adaptive detection transformer. To the best of our knowledge, this is the first work that explores the benefit of a DETR-based detector for the single-source domain generalization task. • We introduce an Online Domain Style Adapter (ODS-Adapter) that projects the style representation of unseen target domains onto the training domain. By maintaining a dynamic memory bank that self-organizes into diverse style bases and continuously absorbs statistics

Read the complete text