PlantPose: Universal Plant Skeleton Estimation via Tree-constrained Graph Generation

Xinpeng Liu, Hiroaki Santo, Yosuke Toda, Fumio Okura · 2026

Read PlantPose: Universal Plant Skeleton Estimation via Tree-constrained Graph Generation by Xinpeng Liu, Hiroaki Santo, Yosuke Toda, Fumio Okura (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17773.txt.

Introduction

Xinpeng Liu and three colleagues build PlantPose, a tool that draws plant skeletons from images. Human skeletons have fixed joints, but plants branch in free-form trees, so the tool must guess arbitrary tree graphs. PlantPose mixes learned graph generation with classic graph methods that force tree shape during training. A broad mix of real, synthetic, sketch, and abstract plant images trains it to cope with varied styles and new plant types. Tests show sturdy, accurate skeletons across domains, even on unseen kinds of images.

Catalog record

CREATORXinpeng Liu, Hiroaki Santo, Yosuke Toda, Fumio Okura
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102067
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17773

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PlantPose: Universal Plant Skeleton Estimation via Tree-constrained Graph Generation Xinpeng Liu, Hiroaki Santo, Yosuke Toda, Fumio Okura Abstract Accurate estimation of plant skeletal structures ( e.g. , branching structures) from images is essential for smart agriculture and plant science. Unlike human skeletons with fixed topology, plant skeleton estimation presents a unique challenge, i.e. , estimating arbitrary tree graphs from images. To address this problem, we introduce PlantPose , a universal plant skeleton estimator via tree-constrained graph generation. PlantPose combines learning-based graph generation with traditional graph algorithms to enforce tree constraints during the training loop. To enhance the model’s generalization capability, we curate a large and diverse dataset comprising real-world and synthetic plant images, along with simplified representations ( e.g. , sketches and abstract drawings). This dataset enables the generalized model to adapt to diverse input styles and categories of plant images while preserving topological consistency. Our approach demonstrates robust and accurate plant skeleton estimation across multiple domains, including previously unseen out-of-domain scenarios. Further analyses highlight the method’s strengths and limitations in handling complex, heterogeneous data distributions. All implementations and datasets are available at https://github.com/huntorochi/PlantPose/ . keywords plant phenotyping, agriculture, image-to-graph generation Figure 1: We propose PlantPose , a method for single-image plant skeleton estimation combining learning-based graph generators with traditional graph algorithms ( i.e. , MST). The red lines show the predicted graph edges. Compared to (a) an unconstrained graph generator and (b) a naive tree-graph constraint implementation, (c) our method naturally imposes the constraint during the graph generation models’ training. Our method can be directly applied to plant science and agricultural applications, such as (d) time-series reconstruction of botanical roots. Furthermore, the PlantPose with large-scale training demonstrates strong generalization capabilities across diverse datasets, including significantly different out-of-domain plant species, as shown in (e). 1 Introduction Skeletal structures of plants ( e.g. , branches and roots) are key information for analyzing plant traits in agriculture and plant science. In particular, single-view estimation of plant skeletons has potential benefits for various downstream tasks, such as high-throughput plant phenotyping 20 ; 11 ; 59 and plant organ segmentation 19 ; 52 . As a similar task, single-view estimation of human poses has been widely studied, e.g. , OpenPose 12 . However, unlike human skeletons, which have a fixed graph topology, the plant skeleton is not organized because the number of joints and their relationships are unknown, posing a unique problem of estimating an arbitrary tree graph from an image. The estimation of graph structure from images has been studied to extract thin structures such as road networks in satellite images 26 ; 71 ; 72 . Recent end-to-end models using recurrent neural networks (RNNs) 2 , graph neural networks (GNNs) 40 ; 39 ; 46 , or transformers 8 ; 31 ; 38 ; 43 ; 60 show the ability to extract faithful unconstrained graph structures from images. However, inferring tree-constrained graphs with the existing graph generators becomes a non-trivial problem, where the output graph often violates the required constraints, as shown in Fig. 1 a. One reason for this difficulty is that tree graph generation, which requires finding a set of graph edges that satisfy the constraint defined on the entire graph, naturally falls into combinatorial optimization. Although constrained optimization in neural networks has been studied by designing differentiable layers 34 , using them to solve our problem is still not straightforward. A simple way to impose constraints on the graph generation is to convert the inferred unconstrained graphs to the closest graph that satisfies the given constraint using traditional graph algorithms, such as Dijkstra’s shortest path or minimum spanning tree (MST) algorithms. Such post-processing can work; however, because the graph generators are trained without any constraints, the output may be unrealistic, as shown in Fig. 1 b. For tree graph generation from single images, we propose a simple yet effective way to integrate state-of-the-art learning-based graph generation methods, which achieve high-quality image-based graph estimation, and traditional graph algorithms, which compute strictly constrained tree graphs such as shown in Figs. 1 c–d. Specifically, we propose to project an unconstrained graph into a tree graph by a non-differentiable MST algorithm during each training loop. Our selective feature suppression (SFS) layer then converts the inferred unconstrained graph to the MST-based tree graph in a differentiable manner, thereby naturally incorporating the constraints into the graph generation. By integrating our feature suppression layer with a state-of-the-art transformer-based graph generator, we develop PlantPose 1 1 1 This terminology is, of course, an analogy of OpenPose 12 for plants. , which infers tree structures from images capturing plants. Beyond the majority of plant-oriented vision methods that use domain-specific ( i.e. , species-specific) datasets for the training, we achieve a generalized model that estimates the plant skeletons for varieties of plants, as highlighted in Fig. 1 e. To this goal, we curate or collect 10 datasets spanning real-world, synthetic, and web-sourced plant images, ensuring the model is exposed to various structural patterns and contexts. We evaluate the effectiveness of PlantPose on different classes of plant images, including out-of-domain datasets that are significantly different from those used for the training. The results show that our constraint-aware graph generator accurately estimates the target tree structures compared to baselines. This paper is an extended version of a conference paper 48 presented in WACV 2025. Beyond 48 using the domain-specific training for each plant species, we achieve a significant advancement by focusing on the universal applicability of our method through the large-scale dataset creation and training, resulting in the plant skeleton estimation across various plant species by a single pre-trained model, as shown in Fig. 1 e. Contributions Our contributions are threefold: • We propose a novel method that tightly integrates learning-based graph generation methods with traditional graph algorithms using the newly proposed SFS layer, which modifies intermediate features in the network, effectively mimicking the behavior of the non-differentiable graph algorithms. • Building upon our constrained graph generation method, we develop the first end-to-end method inferring skeletal structures from a single plant image, which benefits the agriculture and plant science fields. • We collect or curate plant skeleton datasets, including diverse plant images and skeletons, contributing to the generalization of our method as well as future studies for plant skeleton analysis. 2 Related Work We propose constraining the graph structures given by image-based graph generators, whose primary goal is plant skeleton estimation. We, therefore, introduce the related work of plant skeleton estimation, graph generation from images, and constrained optimization for neural networks. 2.1 Plant skeleton estimation Plant skeleton estimation is actively studied since it becomes a fundamental technique for downstream tasks related to plant phenotyping and cultivation 55 . 3D plant skeleton estimation Several methods are proposed to derive plant skeletons from 3D observations 55 . These methods often use point clouds acquired by LiDAR 5 ; 68 or multi-view stereo (MVS) 65 ; 59 . Regardless of the 3D acquisition method, these works generally use a two-stage pipeline: Skeletonization 10 followed by graph optimization using MST or Dijkstra’s algorithm 17 ; 27 ; 52 ; 81 ; 13 , where the graph algorithms are required to convert a set of skeleton positions into a graph. 2D plant skeleton estimation Compared to

Read the complete text