Scaling Up Forest Vision with Synthetic Data

Yihang She, Andrew Blake, David Coomes, Srinivasan Keshav · 2025

Read Scaling Up Forest Vision with Synthetic Data by Yihang She, Andrew Blake, David Coomes, Srinivasan Keshav (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-11201.txt.

Introduction

Yihang She and three co-authors tackle a data bottleneck in forest science in this 2025 computer vision paper. Splitting laser scans of forests into individual trees is essential for tracking carbon and ecosystem health, but real annotated scans are too few to train reliable systems.

The team built a pipeline that fakes the data instead, combining game-engine graphics with a physics-based simulation of laser scanning to produce a large, diverse, labeled three-dimensional forest dataset. A state-of-the-art segmentation model pre-trained on the synthetic forest and then fine-tuned on a single small real plot matched models trained on the full real dataset. The authors identify physics, diversity, and scale as the ingredients that make synthetic pre-training work.

Catalog record

CREATORYihang She, Andrew Blake, David Coomes, Srinivasan Keshav
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-11201

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scaling Up Forest Vision with Synthetic Data Yihang She, Andrew Blake, David Coomes, Srinivasan Keshav Abstract Accurate tree segmentation is a key step in extracting individual tree metrics from forest laser scans, and is essential to understanding ecosystem functions in carbon cycling and beyond. Over the past decade, tree segmentation algorithms have advanced rapidly due to developments in AI. However existing, public, 3D forest datasets are not large enough to build robust tree segmentation systems. Motivated by the success of synthetic data in other domains such as self-driving, we investigate whether similar approaches can help with tree segmentation. In place of expensive field data collection and annotation, we use synthetic data during pretraining, and then require only minimal, real forest plot annotation for fine-tuning. We have developed a new synthetic data generation pipeline to do this for forest vision tasks, integrating advances in game-engines with physics-based LiDAR simulation. As a result, we have produced a comprehensive, diverse, annotated 3D forest dataset on an unprecedented scale. Extensive experiments with a state-of-the-art tree segmentation algorithm and a popular real dataset show that our synthetic data can substantially reduce the need for labelled real data. After fine-tuning on just a single, real, forest plot of less than 0.1 hectare, the pretrained model achieves segmentations that are competitive with a model trained on the full scale real data. We have also identified critical factors for successful use of synthetic data: physics, diversity, and scale, paving the way for more robust 3D forest vision systems in the future. Our data generation pipeline and the resulting dataset are available at https://github.com/yihshe/CAMP3D.git . 1 Introduction Understanding forest dynamics and monitoring their status have long been a priority in the Sustainable Development Goals of the United Nations ( Swamy et al. 2018 ) . Forests can be monitored at different spatial scales from ground surveys to space missions, with varying trade-offs between accuracy, cost, and spatial coverage ( Lines et al. 2022 ; Liang et al. 2022 ; Dubayah et al. 2020 ) . Forest monitoring at a close range ( Liang et al. 2022 ) has become increasingly important for precise forest management. It also offers ground reference data to calibrate satellite remote sensing ( Lang et al. 2022 ) . For close-range forest sensing to obtain metrics for individual trees, the typical workflow involves obtaining point clouds of forests through laser scanning and segmenting individual trees. The next step is to extract the properties of interest, such as above-ground biomass ( Coomes et al. 2017 ) , species ( Puliti et al. 2024 ) , tree health ( Chan et al. 2021 ) , crown volume, canopy height and diameter at breast height (DBH) ( Liang et al. 2019 ) , among others. Accurate measurement of these properties is essential for understanding key ecosystem functions related to carbon cycling and beyond ( fig. 1 ). For example, canopy height and DBH are fundamental for quantifying forest metabolic scaling and allometry, which underpin the estimation of above-ground biomass ( Chave et al. 2009 ) . The shape of the canopy surface and the volume of the crown reveal competitive light dynamics, providing indicators of tree health and reflecting the role of the forest in the energy cascade of ecosystems ( Purves et al. 2007 ) . Metrics derived from canopy surfaces are also critical for studying canopy–atmosphere gas exchange, including the uptake and release of greenhouse gases such as methane and carbon dioxide ( Gauci et al. 2024 ) . The accurate segmentation of individual trees is a key bottleneck in the entire pipeline. Figure 1 : Individual tree segmentation is essential for deriving key forest properties in ecology. This figure shows examples of metrics derived from individual tree segmentation: canopy height and diameter at breast height for biomass and carbon estimation, and crown area, crown volume, and species composition for understanding forest structure, biodiversity, and ecosystem functions. The choice of algorithm plays a crucial role in building a robust tree segmentation system. Over the past decade, tree segmentation has changed from the use of the classical region-growing algorithm ( Coomes et al. 2017 ) to increasingly complicated neural network architectures ( Straker et al. 2023 ; Ball et al. 2022 ; Wielgosz et al. 2023 ; Xiang et al. 2024 ; Wielgosz et al. 2024 ) . It also expanded from 2D image operations ( Coomes et al. 2017 ; Ball et al. 2022 ; Straker et al. 2023 ) to the processing of 3D point clouds ( Wielgosz et al. 2023 ; Xiang et al. 2024 ; Wielgosz et al. 2024 ) . All of this progress has refined the segmentation of trees for a more accurate retrieval of forest properties as discussed above. The increasing complexity of tree segmentation algorithms also requires more data and computation for a stronger and more generalisable model performance ( Sutton 2019 ) . Despite extensive data collection efforts ( Weiser et al. 2022 ; Calders et al. 2022 ; Puliti et al. 2023 ) , the available 3D forest dataset for the development of tree segmentation algorithms still falls short of the scale typically required for robust machine learning ( Uddin and Lu 2024 ) . This is not surprising given that forest scanning requires field work that is limited by multiple factors, such as site accessibility, weather conditions, and seasons. In addition, the intricate canopy structures of forests make it challenging and time consuming to delineate the semantic labels (classifying each point by category, e.g., stem, branch, ground) and instance labels (assigning unique identifiers to individual trees). For example, Puliti et al. 2023 report that just annotating 2.79 hectares of forest plots takes two skilled researchers six months. ( Wielgosz et al. 2024 ) suggest that the scarcity of data has become a bottleneck for further improvement of algorithm performance. Recent advances in graphics technology enable synthetic data generation to tackle data scarcity in other vision domains ( Shotton et al. 2011 ; Sklyarova et al. 2023 ; Gaidon et al. 2016 ; Roberts et al. 2021 ) . In self-driving vehicles, the use of synthetic data has become a key factor for the rapid development of this field ( Song et al. 2023 ) . In the context of forest monitoring, procedural foliage generation in graphics engines such as Unreal Engine (UE) has allowed the creation of forest scenes on a large scale ( Holmberg 2025 ) . Researchers have repurposed game scenes to render synthetic 2D images for the development of tree segmentation algorithms ( Grondin et al. 2022 ; Lu et al. 2024 ; Feng et al. 2025 ) . Open source graphics engines such as Blender enable the creation of simulation packages tailored to natural scenes ( Liu et al. 2021 ; Winiwarter et al. 2022 ) . Simulators such as the Heidelberg LiDAR Operations Simulator (HELIOS) ( Winiwarter et al. 2022 ) resemble the real-world laser scan process during a forest survey. Synthetic data can significantly reduce the effort to collect data in the field and then to annotate the data, as many existing synthetic forest datasets have demonstrated ( Grondin et al. 2022 ; Lu et al. 2024 ; Wang 2020

Read the complete text