Contrastive-SDXL: Annotation-Preserving Night-Time Augmentation for Pedestrian Detection

Franky George, Muhammad Khalid, Adil Khan · 2026

Read Contrastive-SDXL: Annotation-Preserving Night-Time Augmentation for Pedestrian Detection by Franky George, Muhammad Khalid, Adil Khan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16406.txt.

Introduction

George and three others present Contrastive-SDXL, a day-to-night framework for night-time pedestrian detection built on SDXL-Turbo. Night-time detection suffers from limited night data and illumination differences that make daytime-only detectors unreliable. To preserve correspondence, the method uses a patch-wise contrastive loss guided by a DINOv2 encoder plus an object consistency loss preserving pedestrians. Contrastive-SDXL images score FID 22.5, and detectors trained on them cut miss rate 6-7 percent versus a daytime-only baseline.

Catalog record

CREATORFranky George, Muhammad Khalid, Adil Khan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49313
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16406

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Contrastive-SDXL: Annotation-Preserving Night-Time Augmentation for Pedestrian Detection Franky George, Muhammad Khalid, Adil Khan Abstract Night-time pedestrian detection remains challenging because labelled night-time data are limited and large illumination differences make daytime-only trained detectors unreliable. Latent diffusion models (LDMs) provide a powerful basis for image-to-image translation and cross-domain augmentation, but their effectiveness in safety-critical perception depends on whether detector-relevant objects and local semantic structure are preserved when translating between source and target domains. In this work, we present Contrastive-SDXL, a day-to-night augmentation framework for night-time pedestrian detection built on SDXL-Turbo and fine-tuned using Low-Rank Adaptation (LoRA). To preserve semantic correspondence between daytime inputs and translated night-time images, we introduce a patch-wise semantic contrastive loss guided by a pretrained DINOv2 encoder rather than generator encoder features. Multi-level DINOv2 self-attention maps enforce both local and global semantic consistency, while an object consistency loss explicitly encourages pedestrian preservation. Contrastive-SDXL produces realistic night-time images, achieving a Fréchet Inception Distance (FID) of 22.5. Detectors trained with our synthetic images obtain a 6–7% reduction in miss rate compared with a daytime-only baseline, approaching the performance of detectors trained on real night-time data. These results demonstrate that consistency-driven diffusion augmentation can effectively support safety-critical night-time pedestrian detection. I Introduction Night-time pedestrian detection remains a critical challenge for autonomous vehicles and advanced driver-assistance systems (ADAS). Although pedestrian detectors have improved substantially under daytime and favourable lighting conditions, their performance often degrades in low-light environments due to reduced visibility, glare, noise, and large illumination differences. This is particularly problematic in safety-critical transportation scenarios, where missed pedestrians can have severe consequences. One direct solution is to collect and annotate large-scale night-time datasets, but this process is costly, time-consuming, and difficult to scale across different locations, weather conditions, and traffic scenes. Cross-domain augmentation offers an attractive alternative. Instead of collecting and annotating new night-time data, labelled daytime images can be translated into night-style images while reusing their original annotations. A promising direction is diffusion-based image-to-image translation. Diffusion models have recently emerged as powerful generative models, capable of producing high-quality, high-resolution images across diverse domains. With improvements in generation quality and computational efficiency, state-of-the-art latent diffusion models (LDMs) can now generate photorealistic images that are often difficult to distinguish from real samples. This raises an important question for safety-critical perception tasks: can diffusion-based image-to-image translation be used to convert images from a data-abundant source domain to a data-scarce target domain while preserving the object-level consistency required for detection? Works such as Palette and SDEdit [ 26 , 20 ] showed that diffusion models can generate target-style images while preserving source content. However, most existing studies focus on generic visual transformations and perceptual quality, rather than examining whether diffusion-generated images are suitable for safety-critical object detection. This limitation is especially important for day-to-night pedestrian augmentation. Unlike generic image synthesis or style transfer, in a pedestrian detection setting, the translated image must preserve small, occluded, and cluttered pedestrians with sufficient fidelity for the original bounding-box annotations to remain valid. If a pedestrian is distorted, removed, or shifted during translation, the inherited annotation may become unreliable and the downstream detector may learn incorrect cues. This work addresses this gap in two ways. First, we assess pretrained LDMs as cross-domain augmentation tools for night-time pedestrian detection, focusing on whether synthetic night-time pedestrian images generated from daytime inputs can improve robustness under low-light conditions. Since existing datasets are often biased toward clear lighting and favourable weather, generating night-time images that preserve pedestrian details and scene context could enhance detector training while reducing dependence on newly collected night-time data. Second, and as our main contribution, we propose Contrastive-SDXL, a framework tailored for safety-critical day-to-night augmentation. Built on SDXL-Turbo [ 27 ] , our method is fine-tuned using Low-Rank Adaptation (LoRA) [ 10 ] and guided by a patch-wise semantic contrastive loss [ 13 ] to preserve local and global semantic consistency. Unlike prior unpaired image-to-image approaches that rely on generator encoder features, Contrastive-SDXL uses a pretrained DINOv2 [ 22 ] encoder to guide contrastive learning across multiple spatial scales. We further introduce an object consistency loss to explicitly enforce pedestrian preservation during translation. Experimental results show that Contrastive-SDXL generates visually coherent night-time images while retaining pedestrian structure and scene context. Detectors trained with our synthetic images perform comparably to those trained on real night-time data, demonstrating the effectiveness of consistency-driven diffusion augmentation for night-time pedestrian perception. Our main contributions are: • We investigate pretrained LDMs as cross-domain augmentation tools for safety-critical night-time pedestrian detection. • We propose Contrastive-SDXL, an SDXL-Turbo-based day-to-night translation framework fine-tuned with LoRA and guided by DINOv2-based patch-wise semantic contrastive learning. • We introduce an object consistency loss to explicitly preserve pedestrians and maintain annotation validity during day-to-night translation. • We show that Contrastive-SDXL improves low-light detector robustness, achieving a 6–7% reduction in miss rate over a daytime-only baseline and approaching real night training performance. II Related Works II-A Night-time pedestrian augmentation Pedestrian detection is a core perception task for autonomous vehicles and ADAS, but its reliability remains sensitive to illumination, visibility, scale, and occlusion. Large-scale benchmarks such as Caltech, CityPersons [ 30 ] , and EuroCity Persons [ 3 ] have driven substantial progress with dense pedestrian annotations in urban traffic scenes. However, these datasets are still dominated by daytime or favourable lighting conditions. Although ECP includes night images, daytime samples remain much more common. This creates a clear domain shift problem, detectors trained mainly on daytime data often degrade at night due to reduced visibility, glare, noise, and large illumination differences. Night-time perception is commonly addressed through low-light enhancement or day-to-night translation. Enhancement methods improve visibility of images already captured at night. For example, LightDiff [ 15 ] uses diffusion-based enhancement with perception-aware guidance to favour outputs that improve detector performance. However, such methods enhance existing night-time images rather than generate new labelled night-time samples for augmentation. Day-to-night translation instead converts labelled daytime images into night-style images so that the original annotations can be reused. Early approaches used GAN-based pipelines [ 33 , 1 ] , later augmenting them with semantic, geometric, and depth-based constraints to better preserve scene layout under severe illumination changes. Bang et al. [ 2 ] , for instance, use semantic segmentation and depth cues in a structure-aware CycleGAN-style framework. More recent diffusion-based methods improve fidelity and practicality: CycleGAN-Turbo [ 25 ] enables efficient one-step unpaired translation, while Seed-to-Seed Translation [ 7 ] operates in the latent seed space of a pretrained diffusion model to improve structural correspondence in driving scenes. These works are closely related to our goal, but most focus on perceptual realism, reconstruction quality, or overall task performance, with limited explicit attention to pedestrian-level annotation consistency. In contrast, we treat day-to-night translation as a source-annotation-preserving augmentation strategy for pedestrian detection, rather than as night-style transfer alone. To this end, we introduce Contrastive-SDXL, a diffusion-based framework that combines target-domain realism with semantic- and object-level preservation through patch-wise correspondence in an external feature space and a novel object-level consistency loss. Beyond visual quality, we evaluate whether the generated images improve downstream night-time pedestrian detection compared with real night-time data and existing day-to-night translation methods, providing empirical evidence for the usefulness of Contrastive-SDXL. II-B Consistency-preserving unpaired translation Preserving spatial and

Read the complete text