T-REX: Vision-Based System for Autonomous Leaf Detection and Grasp Estimation
Srecharan Selvam, Abhisesh Silwal, George Kantor · 2025
Read T-REX: Vision-Based System for Autonomous Leaf Detection and Grasp Estimation by Srecharan Selvam, Abhisesh Silwal, George Kantor (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01654.txt.
Introduction
Srecharan Selvam, Abhisesh Silwal, and George Kantor built T-REX, a robot that picks leaves for sampling, in a 2025 paper. Farm labs need leaf bits to test plant health, a slow hand task. The gantry robot sees, picks, and grips a leaf on its own. A six-way arm rides the frame. Stereo sight builds depth maps and 3D leaf masks, while a known net cuts leaves from the view in real time.
A pick rule weighs clutter, sight lines, and range to pick the best leaf. A grip point rests on flat face, top reach, and edge room. A fine-needle grip then clamps the leaf in a mock sample act. Tests on fake plants in set poses hit a 66.6 percent grip rate. The team frame the rig as a step to self-run plant sampling in glass houses.
Catalog record
| CREATOR | Srecharan Selvam, Abhisesh Silwal, George Kantor |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01654
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
T-REX: Vision-Based System for Autonomous Leaf Detection and Grasp Estimation Srecharan Selvam, Abhisesh Silwal, George Kantor Abstract T-Rex (The Robot for Extracting Leaf Samples) is a gantry-based robotic system developed for autonomous leaf localization, selection, and grasping in greenhouse environments. The system integrates a 6-degree-of-freedom manipulator with a stereo vision pipeline to identify and interact with target leaves. YOLOv8 is used for real-time leaf segmentation, and RAFT-Stereo provides dense depth maps, allowing the reconstruction of 3D leaf masks. These observations are processed through a leaf grasping algorithm that selects the optimal leaf based on clutter, visibility, and distance, and determines a grasp point by analyzing local surface flatness, top-down approachability, and margin from edges. The selected grasp point guides a trajectory executed by ROS-based motion controllers, driving a custom microneedle-equipped end-effector to clamp the leaf and simulate tissue sampling. Experiments conducted with artificial plants under varied poses demonstrate that the T-Rex system can consistently detect, plan, and perform physical interactions with plant-like targets, achieving a grasp success rate of 66.6%. This paper presents the system architecture, implementation, and testing of T-Rex as a step toward plant sampling automation in Controlled Environment Agriculture (CEA). Introduction Plant diseases pose a persistent threat to global agricultural productivity, with estimated yield losses ranging from 20–40% annually due to pathogenic infestations and pest attacks ( Savary et al. 2012 ; Savary et al. 2019 ) . These losses have far-reaching implications—not only impacting food security but also the economic viability of growers. In many cases, outbreaks are exacerbated by the delayed detection of disease, where symptoms only manifest visibly at an advanced stage of infection. Traditional practices for identifying plant pathogens—such as visual inspection, culturing of microbes, and PCR-based molecular confirmation—are slow, manual, and not scalable for large production systems ( Savary et al. 2012 ; Pilli and George 2014 ) . These approaches often take several days to weeks from symptom detection to actionable diagnosis, allowing the disease to spread unnoticed across farms and greenhouses. Recent advancements in artificial intelligence (AI), robotics, and molecular biology have opened new frontiers in precision agriculture. In particular, AI-powered computer vision systems are capable of identifying early stress indicators in plants based on image-based features, even before visual symptoms are apparent to the human eye ( Chin, Catal, and Kassahun 2023 ) . Robotic platforms, including drones and ground rovers, have been used to scout large agricultural areas autonomously and collect image data for disease diagnosis ( Johnson, Vinatzer, and Li 2023 ; Trippa et al. 2024 ) . At the same time, genomic technologies—especially portable, real-time DNA sequencing platforms like Oxford Nanopore—have enabled high-resolution identification of plant pathogens directly from leaf tissue, eliminating the need for lengthy culture-based assays ( Espindola et al. 2015 ) . When combined, these technologies form the backbone of an emerging paradigm in agriculture: cyber-physical systems (CPS) for closed-loop plant health monitoring and control. Despite these promising developments, significant limitations remain. Many of the current AI-based disease detection systems rely solely on imaging, which can produce false positives/negatives when differentiating between abiotic stress (e.g., drought) and biotic infections (e.g., bacterial or fungal pathogens) ( Johnson, Vinatzer, and Li 2023 ) . On the other hand, while DNA sequencing offers definitive diagnosis, it still depends on manual tissue sampling—a process that is both time-consuming and labor-intensive. Furthermore, few systems have successfully integrated real-time imaging, autonomous sampling, and in-field sequencing into a cohesive, closed-loop disease control workflow. Controlled Environment Agriculture (CEA)—including greenhouses, vertical farms, and hydroponic systems—has emerged as a promising solution for high-yield, resource-efficient crop production. These systems allow year-round cultivation under tightly regulated environmental conditions and are increasingly adopted in urban and climate-sensitive regions ( Patel and Shah 2021 ) . However, CEA systems also face critical challenges: high energy demands for lighting and HVAC, limited scalability, and the need for skilled labor to manage complex microclimates ( EnTech Solutions 2023 ; CEAg World 2024 ) . These limitations underscore the importance of automation and robotic solutions to improve reliability and reduce manual intervention in such controlled setups. This paper focuses on the development and implementation of T-Rex, a gantry-type robotic system designed for autonomous detection and diagnosis of plant diseases through leaf grasping and DNA extraction. T-Rex (Robot for EXtracting leaf samples) operates within a high-density greenhouse setup, targeting crops such as tomato and soybean, and is capable of executing an entire disease detection pipeline—from plant scanning and leaf analysis to sampling and sequencing—without human intervention ( Silwal et al. 2024 ) . The imaging component of T-Rex uses stereo cameras and the YOLOv8 segmentation model to detect early disease symptoms such as wilting, discoloration, or stunted growth ( Trippa et al. 2024 ) . These cues are used to guide a six-degree-of-freedom (6-DOF) gantry manipulator, which physically interacts with selected leaves using a novel microneedle-based end-effector ( Silwal et al. 2024 ) . This end-effector clamps the leaf with minimal deformation and extracts tissue samples by piercing it with microneedle arrays. The extracted tissue is immediately transferred to a microfluidic sequencing pipeline, where low-volume samples undergo library preparation for real-time nanopore sequencing ( Espindola et al. 2015 ) . This enables the system to not only detect symptoms but also confirm the presence of pathogens using genetic evidence, all within the greenhouse environment. System Overview System Design Mechanical Design: The T-Rex robot is built as a gantry system spanning a 3 m \times 1.5 m growing area ( Silwal et al. 2024 ) . This design provides coverage of multiple plants arranged on a flat bench, similar to modern hydroponic greenhouse layouts. The robot’s kinematics consist of three prismatic axes (for X , Y , Z positioning) and three revolute joints on the end-effector, denoted 3P3R ( Silwal et al. 2024 ) . Together, these achieve a full 6-DOF manipulation workspace, allowing the end-effector to be positioned and oriented anywhere within the plant canopy volume. The prismatic axes move a carriage in the horizontal plane and raise/lower the end-effector, while the wrist joints adjust pitch, yaw, and roll of the sampling tool. This enables, for instance, approaching a leaf from directly above or at slight angles if needed. All actuators are high-torque digital servo motors controlled in firmware, offering positional accuracy sufficient for repeatable fine motions ( Silwal et al. 2024 ) . Figure 1: T-REX system architecture showing the main processing pipeline: initial stereo image acquisition, perception and leaf grasping analysis, motion planning for execution, leaf grasping operations, and the reload mechanism for continuous operation. The feedback loops (dotted lines) enable autonomous multi-sample collection without human intervention. End-Effector: The end-effector is a critical component of the T-Rex system, designed for precise leaf sampling (Figure 2 ). It consists of two side grippers that move laterally (left and right) controlled by a Dynamixel motor. When activated, these side components move toward each other to grasp the target leaf. The end-effector also incorporates a stepper motor that performs vertical (up and down) linear motion. After the side grippers secure the leaf, the stepper motor moves downward to initiate the sampling process, pressing the microneedle array against the leaf surface. The microneedle array is housed in a thin plastic holder with laser-etched