Using street view imagery and deep generative modeling for estimating the health of urban forests

Akshit Gupta, Remko Uijlenhoet · 2025

Read Using street view imagery and deep generative modeling for estimating the health of urban forests by Akshit Gupta, Remko Uijlenhoet (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14583.txt.

Introduction

City trees cool streets, catch rain, and clean air, but checking their health takes legwork. Akshit Gupta and Remko Uijlenhoet propose reading it from street view photos instead. Their method pairs those photos with tree inventory records and weather data. Image translation networks then estimate two health measures: a greenness index and canopy temperature depression. Both normally need special sensors.

A field campaign with handheld instruments will supply the ground truth for comparison. The authors argue street view archives already cover whole cities. That coverage could let officials monitor urban forests at scale, without sending crews to every street. The work targets cities with thin budgets.

Catalog record

CREATORAkshit Gupta, Remko Uijlenhoet
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE9 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14583

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Using street view imagery and deep generative modeling for estimating the health of urban forests Akshit Gupta, Remko Uijlenhoet Abstract Healthy urban forests comprising of diverse trees and shrubs play a crucial role in mitigating climate change. They provide several key advantages such as providing shade for energy conservation, and intercepting rainfall to reduce flood runoff and soil erosion. Traditional approaches for monitoring the health of urban forests require instrumented inspection techniques, often involving a high amount of human labor and subjective evaluations. As a result, they are not scalable for cities which lack extensive resources. Recent approaches involving multi-spectral imaging data based on terrestrial sensing and satellites, are constrained respectively with challenges related to dedicated deployments and limited spatial resolutions. In this work, we propose an alternative approach for monitoring the urban forests using simplified inputs: street view imagery, tree inventory data and meteorological conditions. We propose to use image-to-image translation networks to estimate two urban forest health parameters, namely, NDVI and CTD. Finally, we aim to compare the generated results with ground truth data using an onsite campaign utilizing handheld multi-spectral and thermal imaging sensors. With the advent and expansion of street view imagery platforms such as Google Street View and Mapillary, this approach should enable effective management of urban forests for the authorities in cities at scale. 1 Introduction Urban forests comprising of trees and shrubs are a fundamental asset in urban cities whihc provide multiple benefits for mitigating climate change. These benefits range from intercepting rainfall to prevent water from directly flowing to impervious surfaces, thus reducing flood events, to reducing energy consumption by providing shade, thus reducing urban heat island effect and local temperatures ( Gregory McPherson 1992 ; Hobbie and Grimm 2020 ). However, with the changing urban climate, the existing urban forests need increased maintenance and more freuquent monitoring ( Werbin et al. 2020 ). This heightened attention is essential because the capacity of urban forests to deliver their diverse and beneficial ecosystem services is significantly compromised when these green spaces are unhealthy ( Hand and Doick 2019 ). Currently, the health of urban forests is measured by arborists (or tree doctors) moving manually from one tree to next, which involves high human labor. This leads to infrequent health assessments with a temporal delay of two to four years. Recent approaches relying on technological developments (as shown in Figure 1 ) such as LEO (low earth orbit) remote sensing data, embedded sensing can supplement traditional manual approaches ( Gupta et al. 2024a ). However, these approaches are impeded respectively with challenges related to the spatial resolution of satellite imaging sensors and the scalability of embedded sensors. Recent vehicular mobile sensing approaches (shown as drive-by in Figure 1 ) involve deployment of multiple imaging sensors on dedicated or opportunistic fleets such as cars, bikes etc, leading to high operational costs. Figure 1: Current methods for monitoring health of urban forest and the scalability challenges associated with them due to spatial and temporal resolutions. Adapted from ( Gupta et al. 2024a ) Both the technological approaches of remote sensing and drive-by sensing utilize multi-spectral or hyper-spectral imaging sensors deployed on a moving entity which can be satellites, air balloons, cars or bikes. These multi-spectral sensors generate multi-spectral images with additional channels (or spectral bands) compared to the standard RGB (Red, Green and Blue) channels associated with optical imagery. These additional channels (mainly comprising of NIR (near-infrared) and Thermal (or long-infrared)) are in-turn processed for calculating simple and reliable vegetation indexes which indicate urban tree health. For instance, GreenScan ( Gupta et al. 2024b ) based on drive-by sensing utilizes two imaging sensors which concurrently capture Red, Green, Near Infrared images, and Thermal images to generate two tree health indexes namely NDVI (normalized difference vegetation index) and CTD (canopy temperature depression). As the measured health indexes have different threshold values based on the genus (and species) of the tree, the data related to genus (and species) ( I_{species} ) is indispensable. From the point of street level measurements, the value of light reflected, as is measured by the multi-spectral sensors depends on the input light radiation from the sun ( I_{radiation} ) and the solar orientation angle ( S_{angle} ) at the time of image capture. For measurement of CTD, air temperature ( T_{air} ) is essential. At the same time, image-to-image translation networks based on conditional GANs (generative adversarial networks) have been successfully utilized for generating output images with different characteristics (such as higher resolutions, additional channels) from input images in various application domains. Namely, in the remote sensing domain, Yuan et al. 2020 utilize a conditional GAN to generate NIR channel data as output using RGB images as input for the Sentinel-2 satellite. In the agriculture domain, Davidson et al. 2022 utilize a conditional GAN based on Pix2Pix architecture ( de Lima et al. 2022 ) to generate near-infrared and red-edge channel data from RGB optical images captured using drones. Finally, in the urban planning domain, Klimenka et al. 2025 utilize RGB optical images captured from street view to generate thermal imaging data and estimate urban facade temperatures. 2 Proposed Methodology In this work, we propose to train a predictive model which generates both near-infrared and thermal images from RGB images using Pix2Pix ( de Lima et al. 2022 ) architecture based on conditional GANs. The Pix2Pix architecture is modified with input variables namely, ( I_{radiation} ), ( S_{angle} ), ( I_{species} ) fed as an embedding vector in the Generator stage of Pix2Pix. This is shown in Figure 2 . Finally, the generated near-infrared and thermal images are in-turn used after canopy segmentation to estimate two tree health indexes namely NDVI and CTD (as shown in Figure 3 ). By training a model on input paired imaging dataset, we bypass the need for capturing near-infrared and thermal imaging data, instead relying on non-linear correlations and embedding features learnt by the proposed model. This decreases the operational costs associated with the multiple imaging sensors utilized in the technical approaches of vehicular mobile and remote sensing. Figure 2: Adding the meteorological variables as an embedding layer in Pix2Pix conditional GAN architecture to generate near-infrared from standard RGB images. The same methodology is used to generate thermal images. For real world testing, our pre-trained model is applied to street view images obtained from the Mapillary platform as input data, to generate near-infrared and thermal images which are subsequently utilized to estimate NDVI and CTD. Finally, the generated NDVI and CTD values are compared with an on-site campaign with professional equipments comprising of a multispectral camera (MAPIR Survey 3W) and a thermal camera (FLIR A50). Figure 3: Using street view images from Google Street view to generate R,G, near infrared and thermal images, though previously trained models. The CTD and NDVI is calculated after segmentation of canopy as performed in ( Gupta et al. 2024b 2.1 Datasets We propose to utilize the dataset from the study ( Gupta et al. 2024b ) which includes pairs of RGB, near infrared and thermal images along with the location metadata. This data is supplemented with additional datasets from the studies of Brown and Süsstrunk 2011 and Xu et al. 2020

Read the complete text