Adapting Vehicle Detectors for Aerial Imagery to Unseen Domains with Weak Supervision

Xiao Fang, Minhyek Jeon, Zheyang Qin, Stanislav Panev, Celso de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre · 2025

Read Adapting Vehicle Detectors for Aerial Imagery to Unseen Domains with Weak Supervision by Xiao Fang, Minhyek Jeon, Zheyang Qin, Stanislav Panev, Celso de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20976.txt.

Introduction

Spotting cars from the air helps traffic watch and city planning, but a detector trained in one place often fails in another. Xiao Fang and co-authors tackle that drop in a 2025 paper. Light, layout, road style, and car types all shift between regions, and those shifts drag scores down. Their fix uses AI-made aerial images, with labels, to widen the training set.

A multi-stage transfer framework fine-tunes image-making models to close the gap between source and target areas. Tests across many aerial domains showed steady gains over several rival methods. Gains ranged from a few points to more than half again on one rival class. The team also adds two new test sets to the field. The work suits defense and civic uses alike.

Catalog record

CREATORXiao Fang, Minhyek Jeon, Zheyang Qin, Stanislav Panev, Celso de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-20976

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Adapting Vehicle Detectors for Aerial Imagery to Unseen Domains with Weak Supervision Xiao Fang, Minhyek Jeon, Zheyang Qin, Stanislav Panev, Celso de Melo, Shuowen Hu, Shayok Chakraborty, Fernando De la Torre Abstract Detecting vehicles in aerial imagery is a critical task with applications in traffic monitoring, urban planning, and defense intelligence. Deep learning methods have provided state-of-the-art (SOTA) results for this application. However, a significant challenge arises when models trained on data from one geographic region fail to generalize effectively to other areas. Variability in factors such as environmental conditions, urban layouts, road networks, vehicle types, and image acquisition parameters (e.g., resolution, lighting, and angle) leads to domain shifts that degrade model performance. This paper proposes a novel method that uses generative AI to synthesize high-quality aerial images and their labels, improving detector training through data augmentation. Our key contribution is the development of a multi-stage, multi-modal knowledge transfer framework utilizing fine-tuned latent diffusion models (LDMs) to mitigate the distribution gap between the source and target environments. Extensive experiments across diverse aerial imagery domains show consistent performance improvements in AP 50 \text{AP}_{50} over supervised learning on source domain data, weakly supervised adaptation methods, unsupervised domain adaptation methods, and open-set object detectors by 4-23%, 6-10%, 7-40%, and more than 50%, respectively. Furthermore, we introduce two newly annotated aerial datasets from New Zealand and Utah to support further research in this field. Project page is available at: https://humansensinglab.github.io/AGenDA Figure 1 : Overview. We propose a pipeline that generates high-quality aerial images along with their labels. Our method outperforms baseline detectors trained on source images and open-set detectors directly inferred on target images. 1 Introduction Recent advancements in Generative AI have led to the development of diffusion-based models [ 50 ] that can synthesize images with unprecedented realism, making them nearly indistinguishable from real-world data [ 12 , 52 , 36 ] . Additionally, integrating natural human language has become crucial to modern vision systems. This integration can take the form of conditioning the generation process (for example, in text-to-image generative models [ 46 , 41 ] utilizing cross-attention mechanism) or serving as an additional modality that shares a common latent space with vision (as seen in models like CLIP [ 42 ] ). One key reason for the success of these new models is that they have been trained on extensive datasets, such as LAION-5B [ 49 ] or WIT [ 42 ] , consisting of hundreds of millions or billions of text-image pairs. As a result, the scientific community often refers to them as Foundational Models . Upon their inception, researchers have explored how to use foundational models to generate training data for downstream tasks like classification, object detection, and semantic segmentation. Methods such as ALIA [ 8 ] , DatasetDM [ 60 ] , Dataset Diffusion [ 35 ] , DiffusionEngine [ 70 ] are particularly useful in scenarios with limited or no access to natural training data. Despite the promising results these methods have achieved augmenting general-purpose datasets like VOC [ 9 ] and COCO [ 28 ] , little to no effort has been made to generate diverse, high-resolution annotated synthetic datasets on aerial views. These datasets are important for training detectors on small objects like vehicles. The primary reason for this gap is that off-the-shelf generative models, struggle to generate these views due to insufficient aerial imagery representation in their training datasets. Hence, fine-tuning is unavoidable in this case. Recent efforts have primarily produced generative models constrained by geospatial resolution, such as DiffusionSat [ 17 ] and SatDiffMoE [ 51 ] , and therefore cannot be utilized for vehicle detection. The lack of sufficient large-scale aerial view training data impacts the performance of not only generative foundational models but also other types of Vision Large Language Models [ 67 ] (VLLMs), such as BLIP2 [ 22 ] , InternVL3 [ 75 ] , and Gemini [ 55 ] . Our analysis reveals that these models perform poorly in zero-shot settings for tasks such as car presence classification and center localization in aerial imagery. (see Appendix Section C ). This limitation also extends to open-set object detectors [ 23 , 72 , 34 ] , which, as shown in Table 1 , exhibit unsatisfactory results on aerial imagery. On the other hand, detection annotations should also be produced for each generated image object. However, text-to-image models do not naturally provide these. There are two potential options to address this challenge: 1) the annotations ( e.g . , semantic segmentation maps, bounding box layouts) to be created before the image generation and used as a spatial conditioning signal [ 69 , 26 ] , or 2) products from the generative process, such as cross- and self-attention maps extracted from the denoising U-Net [ 12 ] to be employed to image produce annotations [ 54 , 60 , 35 , 70 ] . In the first case, studies reported that the generative models often do not fully obey the imposed conditioning, especially for small objects such as vehicles in aerial view images [ 62 ] , which may result in generated annotations that do not match the corresponding synthetic images. Therefore, we studied the prospects of the second approach. To this end, we introduce a new method for annotated aerial view vehicle detection via synthetic image dataset generation, which employs stacked (multi-channel) cross-attention maps, learnable text prompt tokens, and multi-stage cross-environment (source to target) knowledge transfer. We consider the scenario where a fully annotated dataset with bounding box annotations from the source environment and a target environment dataset with weak binary annotations (whether or not a vehicle is present in a given image) are available, which are much easier to obtain than full bounding box annotations. Our extensive experiments show considerable improvements over baseline (model trained only on the source environment data) performance demonstrated by four popular object detectors. Our contributions can be summarized as follows: • We introduce a novel approach for generating aerial view annotated synthetic image datasets based on multi-channel cross-attention maps and multi-stage cross-environment knowledge transfer. • We conduct extensive experiments involving four popular state-of-the-art object detectors to assess the effectiveness of our approach. We compared the performance of our method with other published methods for open-set detection, unsupervised object detector adaptation, and weakly supervised model adaptation. The results corroborate the promise and potential of our framework. • We introduce two new real-world aerial view datasets captured in Selwyn (New Zealand), 2,078,077 images, and Utah (USA), 2,684,658 images, including car (small vehicle) detection task annotations. 2 Related Work 2.1 Diffusion Models for Perception Tasks Diffusion models [ 12 , 6 ] have undergone significant developments and have emerged as prominent generative models in modern research. These models work by progressively corrupting data with noise, and then learning to reverse this process to reconstruct the original data. Once trained, new data can be generated by applying the learned denoising process to randomly sampled noise. Latent diffusion models (LDM) [ 46 , 41 ] perform diffusion process in the latent image space, which reduces the computation cost towards high-resolution image synthesis. Customized image generation can be achieved by adding various types of control, such as texts [ 47

Read the complete text