SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds

Trung Thanh Nguyen, Daniel Lusk, Kilian Gerberding, Janusch Vajna-Jehle, Tuan-Anh Vu, Duc Viet Le, Tu Vo, Phi Le Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide, Julian Frey, Teja Kattenborn · 2026

Read SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds by Trung Thanh Nguyen, Daniel Lusk, Kilian Gerberding, Janusch Vajna-Jehle, Tuan-Anh Vu, Duc Viet Le, Tu Vo, Phi Le Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide, Julian Frey, Teja Kattenborn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27491.txt.

Introduction

Nguyen leads a large team teaching a model to cut out any single tree from a three-dimensional forest scan at a user's click, in a 2026 remote sensing paper. Labelling tree instances by hand in dense forest scans is slow and error-prone. SelectAnyTree adapts the promptable segmentation idea to this world. A click-to-query encoder turns each click into one content query carrying position, polarity, and a pooled local feature. A canopy height model supplies treetops as an automatic first prompt.

A state-space decoder turns the query into one tree mask while capturing long-range context in linear time. Across seven forest regions and an independent held-out set, the model generalises well beyond its training domains. A single click reaches a segmentation overlap score of seventy-eight point two, about twenty-five points above the strongest promptable baseline in the reported evaluation.

Catalog record

CREATORTrung Thanh Nguyen, Daniel Lusk, Kilian Gerberding, Janusch Vajna-Jehle, Tuan-Anh Vu, Duc Viet Le, Tu Vo, Phi Le Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide, Julian Frey, Teja Kattenborn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.27491

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SelectAnyTree: A Promptable Instance Segmentation Model for 3D Forest LiDAR Point Clouds Trung Thanh Nguyen, Daniel Lusk, Kilian Gerberding, Janusch Vajna-Jehle, Tuan-Anh Vu, Duc Viet Le, Tu Vo, Phi Le Nguyen, Yasutomo Kawanishi, Takahiro Komamizu, Ichiro Ide, Julian Frey, Teja Kattenborn Abstract Automated instance segmentation of forest LiDAR point clouds is increasingly critical as forest monitoring moves toward scalable, detailed, 3D measurement. Yet, progress is constrained by label scarcity for tree instances; a single hectare can hold millions of points and hundreds of overlapping, complex crowns, making manual annotation from scratch with raw data laborious and error-prone. Annotations are often corrected from automatic pre-segmentations, but remain costly as these provide no interactive or AI-assisted refinement. Inspired by the promptable paradigm of foundation segmentation models, we propose SelectAnyTree , a promptable instance segmentation model that delineates any individual tree in a 3D forest point cloud from a few clicks. It introduces two key components: Click-to-query prompt encoder and Canopy Height Model (CHM)-guided first prompt. The former turns each click into a single content query, encoding its 3D position and positive/negative polarity together with a pooled local backbone feature. The latter provides treetops as a geometry- and ecologically guided first prompt without any user input. The resulting prompt query is then decoded into one tree mask by a state-space query decoder to efficiently capture long-range context in large-scale forest scenes with linear-time complexity. We evaluate SelectAnyTree in interactive and instance-level settings across seven diverse forest regions and an independent held-out test dataset, demonstrating strong generalization beyond the training domains. It segments a target tree to 78.2 Intersection over Union (IoU) from a single click, 24.8 points above the strongest promptable baseline, and reaches every accuracy target with the fewest clicks, while using far fewer parameters and less inference time than prior promptable models. 1 Introduction Forests are complex 3D ecosystems that play a central role in carbon storage, biodiversity, and climate regulation [ 28 , 11 ] . The combination of terrestrial and drone-based LiDAR with deep learning has made it possible to effectively convert the dense 3D structure of forests into quantitative information for inventory and ecological analysis [ 1 , 6 , 11 ] . A core task underlying these applications is individual tree instance segmentation ; assigning every point of a forest scan to the tree to which it belongs. Reliable per-tree delineation is a prerequisite for estimating biomass, crown architecture, and competition, as well as for downstream traits such as species or health status. Recent forest instance segmentation methods have advanced rapidly, progressing from clustering-based pipelines [ 9 , 33 ] to end-to-end query-based architectures [ 13 , 34 , 19 ] . Despite their strong benchmark performance, these models share a common operating mode; they segment all trees in a scene in a single fully-automatic forward pass. This design has two practical consequences. First, it provides no mechanism for a user to select a particular tree of interest, whether to annotate a reference plot, extract a single specimen, or build training data, without running and post-processing the entire scene. Second, in dense, multi-layered forest canopies where crowns are highly entangled and the models often fail [ 9 , 34 ] , there is no way to correct an individual mistaken instance; errors can only be addressed by retraining or costly by manual editing. In annotation-heavy ecological applications, where ground-truth masks are expensive and expert-driven, this lack of human control is a fundamental limitation. More fundamentally, progress in forest intelligence from LiDAR remains constrained by the scarcity of labels, particularly for individual tree instances and downstream properties such as species or functional type, which are costly to annotate at scale [ 11 , 34 ] . In 2D vision, the Segment Anything Model (SAM) [ 12 ] reframed segmentation as a promptable task, where a user specifies the object to be segmented through a few clicks and the model responds with a corresponding mask. This paradigm of AI-assisted annotation has since been extended to 3D point clouds by interactive and promptable models such as AGILE3D [ 36 ] , NPISeg3D [ 14 ] , and Point-SAM [ 38 ] , and to native 3D part segmentation by PartSAM [ 39 ] . However, these models are designed for generic indoor objects, man-made 3D shapes, or object parts, and are not optimized to encode forest-specific structures. Forest scenes differ sharply from these settings; they are large-scale and sparsely sampled, contain hundreds of trees with strongly overlapping but structurally diverse crowns, and exhibit a pronounced vertical ground-to-canopy organization that generic promptable models do not exploit. As a result, directly applying object-centric promptable models to forest LiDAR leaves substantial domain structure unused. To close this gap, we propose SelectAnyTree , a promptable model that segments any individual tree in a 3D forest point cloud in a few clicks. It encodes a forest scene once and reuses the features across all prompts for efficient multi-tree selection. Three design choices distinguish SelectAnyTree from generic promptable 3D models. First, a lightweight prompt encoder fuses each click’s position, polarity, and a cylinder-pooled backbone feature into a single query that the decoder turns into one mask, disambiguating a lone click among similar, densely entangled trees. Second, we exploit a forest geometry prior : The Canopy Height Model (CHM) [ 20 , 10 ] treetop provides a geometry-guided “free” first prompt that links the fully-automatic and interactive modes. Third, following SAM-style [ 39 , 38 ] , training and evaluation share a single click-simulation rollout, ensuring training and evaluation consistency. The main contributions of this work are as follows: • We introduce SelectAnyTree , a promptable instance segmentation model for 3D forest LiDAR point clouds, enabling a user to select and segment any individual tree in a few clicks and to interactively correct errors in dense canopies. • We propose a forest-aware promptable design that (i) converts each click into a single decoder query through a prompt encoder that fuses its position, polarity, and local backbone feature, and (ii) uses the CHM treetop as a geometry-guided prompt for interactive segmentation. • We adopt a shared interactive click-simulation rollout for both training and evaluation, ensuring consistency. SelectAnyTree segments a target tree to 78.2 Intersection over Union (IoU) from a single click and reaches every accuracy target with the fewest clicks, outperforming promptable 3D baselines. 2 Related work Figure 1 : Overview of the proposed SelectAnyTree. A forest point cloud is encoded once by the 3D Scene Encoder into reusable voxel features. Each click set is converted by the Prompt Encoder into a single content query that the Query Decoder turns into one tree mask. Training and evaluation share the same Interactive Rollout, with a Canopy Height Model (CHM) treetop as a free, geometry-guided prompt. Forest point cloud instance segmentation. Delineating individual trees from LiDAR has a long history in remote sensing, where classical pipelines detect local maxima in a CHM and grow regions with watershed-style rules [ 18 , 20 ] . These methods are effective for dominant, well-separated trees but degrade in dense, multi-layered canopies. Deep learning has substantially improved robustness: ForAINet [ 33 ] couples a sparse convolutional U-Net [ 23 , 7 ] with post-hoc

Read the complete text