A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery

Pengyu Chen a , Sicheng Wang a , Cuizhen Wang a , Senrong Wang b , Beiao Huang c , Lu Huang b , and Zhe Zang d · 2025

Read A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery by Pengyu Chen a , Sicheng Wang a , Cuizhen Wang a , Senrong Wang b , Beiao Huang c , Lu Huang b , and Zhe Zang d (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23200.txt.

Introduction

Pengyu Chen and co-authors build a tool that finds rooftops in old aerial photos. Such photos can show how towns grew over decades. But the black-and-white film is grainy, lacks color, and has decayed with age. Modern detectors trained on crisp color images cope poorly with them. The team first upgrades each photo in two stages before detection runs.

One stage adds color with a generative network, and the next sharpens detail. Three detectors were then trained on the upgraded images. The best one scored above eighty-five percent on a standard accuracy scale. That score beats plain photos by about forty percent and color-only upgrades by twenty percent. The code and data are shared for others to repeat the work.

Catalog record

CREATORPengyu Chen a , Sicheng Wang a , Cuizhen Wang a , Senrong Wang b , Beiao Huang c , Lu Huang b , and Zhe Zang d
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-23200

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A GAN-Enhanced Deep Learning Framework for Rooftop Detection from Historical Aerial Imagery Pengyu Chen a , Sicheng Wang a , Cuizhen Wang a , Senrong Wang b , Beiao Huang c , Lu Huang b , and Zhe Zang d Abstract Precise detection of rooftops from historical aerial imagery is essential for analyzing long-term urban development and human settlement patterns. Nonetheless, black-and-white analog photographs present considerable challenges for modern object detection frameworks due to their limited spatial resolution, absence of color information, and archival degradation. To address these challenges, this research introduces a two-stage image enhancement pipeline based on Generative Adversarial Networks (GANs): image colorization utilizing DeOldify, followed by super-resolution enhancement with Real-ESRGAN. The enhanced images were subsequently employed to train and evaluate rooftop detection models, including Faster R-CNN, DETReg, and YOLOv11n. The results demonstrate that the combination of colorization with super-resolution significantly enhances detection performance, with YOLOv11n achieving a mean Average Precision (mAP) exceeding 85%. This signifies an enhancement of approximately 40% over the original black-and-white images and 20% over images enhanced solely through colorization. The proposed method effectively bridges the gap between archival imagery and contemporary deep learning techniques, facilitating more reliable extraction of building footprints from historical aerial photographs. Code and resources for reproducing our results are publicly available at github.com/Pengyu-gis/Historical-Aerial-Photos . keywords Rooftop detection; historical aerial imagery; deep learning; generative adversarial networks (GAN); image enhancement; object detection † † articletype: RESEARCH ARTICLE † † affiliation: a Department of Geography, University of South Carolina, Columbia, SC, USA; b School of Computer Science and Engineering, Wuhan University of Technology, Wuhan, China; c School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China; d College of Marine Geosciences, Ocean University of China, Qingdao, China 1 Introduction Remote sensing has been a cornerstone for understanding and managing the built environment, enabling applications such as urban planning, infrastructure monitoring, and disaster response ( Wellmann et al., 2020 ; Tosti et al., 2021 ; Pi et al., 2020 ) . Among these, rooftop detection plays a vital role in extracting building footprints, assessing structural integrity, and informing policy decisions related to land use and hazard mitigation ( Yuan, 2025 ; Bauchet et al., 2021 ; Ding and Shi, 2013 ; Norman et al., 2019 ) . The United States started to collect aerial photographs since the early 20th century. These historical aerial images provide rich information about past built environments. For example, the United States Geological Survey (USGS) began to acquire aerial photographs in 1937 ( U.S. Geological Survey, 1973 ) . These archival images are invaluable for tracking the urban development history of many cities and regions, offering insights into growth patterns, spatial structure transformation, and long-term infrastructural changes ( Farella et al., 2022b ) . Recent advances and applications of aerial image enhancement techniques, such as colorization and super-resolution, have seen remarkable progress with deep learning, demonstrating promise for overcoming the limitations of historical imagery. Colorization methods, especially Generative Adversarial Networks (GANs) and Stable Diffusion, can significantly enhance visual interpretability by introducing plausible color channels to grayscale images ( Goodfellow et al., 2014 ; Rombach et al., 2022 ) . For instance, DeOldify, a widely adopted GAN-based approach, can effectively reduce artifacts while preserving structural details in archival images ( Antic, 2025 ) . Additionally, super-resolution techniques like Real-ESRGAN address low-resolution constraints by reconstructing high-resolution details, thereby improving object detection by refining edges and textures ( Wang et al., 2021 ; Haut et al., 2018 ; Dong et al., 2022 ; Zhang et al., 2024 ) . Various rooftop detection architectures have emerged and evolved, including Convolutional Neural Networks (CNNs) and Transformers, each tailored to specific detection needs. CNN-based models, such as Faster R-CNN, Mask R-CNN, and YOLO variants, have been demonstrated effective. For example, Faster R-CNN has accurately delineated rooftops in medium-resolution aerial images, whereas YOLO models have provided rapid, reliable detection suitable for large-scale applications ( Ren et al., 2017 ; He et al., 2018 ; Park et al., 2024 ; Avudaiamal et al., 2024 ; Khanam and Hussain, 2024 ) . Transformer-based models, including DETR and Vision Transformer (ViT), capture both local and global contexts, beneficial for complex urban rooftop detection scenarios ( Carion et al., 2020 ; Bar et al., 2023 ; Dosovitskiy et al., 2021 ; Yuan, 2025 ) . Although existing rooftop detection methodologies have shown effectiveness, a significant gap remains in the exploration of historical aerial imagery, which presents unique challenges due to limited quality and low resolution Fertel et al., 2023 . Furthermore, few models are optimized explicitly for grayscale images, which are prevalent in archival datasets. As illustrated in Figure 1 , two common issues are exemplified: the left image suffers from blurriness and low resolution, impairing object identification and feature extraction; the right image exhibits overexposed regions that obscure critical structural details. These limitations often result in discrepancies between detected and actual building outlines. In response to the lack of a standardized, scalable workflow for processing such images, this study introduces a two-stage enhancement pipeline. The proposed approach leverages GAN-based frameworks to sequentially apply image colorization and super-resolution, aiming to improve the fidelity of feature extraction from degraded historical imagery. Figure 1: Limitations of Historical Black-and-White Aerial Images. Source: USC Library, 2024 . The rationale behind this two-stage process lies in aligning historical imagery with the data standards expected by modern deep learning models, which are typically trained on high-resolution, RGB-colored imagery. Colorization serves as the first step to reconstruct the chromatic information lost in early aerial imagery, thus enabling the models to exploit color-related features such as material reflectance or shadow hue, which are critical for distinguishing rooftops from other urban surfaces ( Liu et al., 2018 ; Fu et al., 2024 ; Farella et al., 2022a ) . Super-resolution is applied after colorization because color information provides additional spatial priors that improve the quality of upscaling. Previous studies have shown that applying super-resolution directly to grayscale images often leads to artifacts or poor reconstruction of semantic features, whereas colored inputs yield sharper and more accurate high-resolution outputs ( Liang et al., 2021 ; Konovalov and Myasnikov, 2024 ; Gao et al., 2025 ) . This sequence thus maximizes the fidelity of restored features crucial for object detection tasks ( Feng et al., 2022 ) . This enhancement pipeline transforms black-and-white historical aerial images into rich, high-quality data, enabling more accurate rooftop detection and paving the way for long-term geospatial analysis in urban planning, disaster resilience, and architectural history. 2 Study Area and Data 2.1 Study area: Charleston, South Carolina Charleston, the largest city in South Carolina, is a historic port city founded in 1670 on the southeastern U.S. coast, crucial in colonial and post-colonial times. Its well-preserved architecture, including Colonial, Georgian, Federal, Greek Revival, Italianate, and Victorian styles, makes it ideal for studying urban transformations and rooftop detection from historical imagery ( Baco and Stiefel, 2009 ) . Charleston’s densification, expansion, and infrastructure evolution provide insights into urban morphology and spatial changes, serving as a benchmark for historical urban studies ( Baco and Stiefel, 2009 ; Rybczynski, 2019 ; Hanchett, 2020 ) . 2.2 Historical aerial imagery dataset in 1979 The primary dataset utilized in this study

Read the complete text