Learning to Pick: A Visuomotor Policy for Clustered Strawberry Picking

Zhenghao Fei, Wenwu Lu, Linsheng Hou, Chen Peng · 2025

Read Learning to Pick: A Visuomotor Policy for Clustered Strawberry Picking by Zhenghao Fei, Wenwu Lu, Linsheng Hou, Chen Peng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14530.txt.

Introduction

Zhenghao Fei and three co-authors teach a robot in 2025 to pick strawberries grown in tight clusters. Fruit hides among leaves, stems and other berries, so a clear reach is rare. A picking point sits on the stem just above the fruit crown.

Traditional plan then move systems struggle in that clutter. Their system learns from human demonstrations collected with a teleoperation interface for a four joint SCARA arm. An action chunking method with end pose help turns demonstrations into a visuomotor policy. The robot learns to nudge soft obstacles aside gently and approach the stem precisely. Demonstration learning suits farm tasks where modelling every leaf and berry in advance is hopeless.

Catalog record

CREATORZhenghao Fei, Wenwu Lu, Linsheng Hou, Chen Peng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14530

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning to Pick: A Visuomotor Policy for Clustered Strawberry Picking Zhenghao Fei, Wenwu Lu, Linsheng Hou, Chen Peng Abstract Strawberries naturally grow in clusters, interwoven with leaves, stems, and other fruits, which frequently leads to occlusion. This inherent growth habit presents a significant challenge for robotic picking, as traditional percept-plan-control systems struggle to reach fruits amid the clutter. Effectively picking an occluded strawberry demands dexterous manipulation to carefully bypass or gently move the surrounding soft objects and precisely access the ideal picking point—located at the stem just above the calyx. To address this challenge, we introduce a strawberry-picking robotic system that learns from human demonstrations. Our system features a 4-DoF SCARA arm paired with a human teleoperation interface for efficient data collection and leverages an End Pose Assisted Action Chunking Transformer (ACT) to develop a fine-grained visuomotor picking policy. Experiments under various occlusion scenarios demonstrate that our modified approach significantly outperforms the direct implementation of ACT, underscoring its potential for practical application in occluded strawberry picking. I Introduction Global demand for strawberries, a high-value crop, continues to rise. While China led production in 2023 with 3,336,690 tons, followed by the US with 1,055,963 tons [ 1 ] , harvesting remains labor-intensive due to the fruit’s fragility. This contrasts with mechanized harvesting of crops like corn and wheat. In both the US, where immigrant labor is crucial, and China, facing declining agricultural labor and rising costs, the need for robotic harvesting solutions is urgent for sustainable strawberry production. Although numerous studies on robotic strawberry picking have been conducted since the 1990s [ 2 ] , no commercially viable system has yet been developed. These are main caused by the specialty of strawberries. Strawberries are particularly delicate, and their fragility means that they must be handled with great care—similar to the way human pickers operate. Additionally, strawberries grow in dense clusters, with leaves and stems often obscuring the target fruit, making manipulation among the cluster particularly challenging. A successful robotic picking system must not only identify the target fruit but also understand the entire scene, including the spatial relationships and physical properties of surrounding elements. This comprehensive perception is critical for developing a strategy that avoids obstacles, gently displaces soft and cluttered items, and accurately reaches the optimal picking point—just above the fruit’s calyx. The current state-of-the-art approach, which typically follows a percept-plan-control framework, struggles to address these complexities in scene understanding, depth estimation, trajectory planning, and control, limiting its effectiveness in real-world applications. Fig. 1 : (a) & (b) Real world challenge cases of clustered strawberries ; (c) Table-top strawberry cultivation; (d) Trajectory for dexterous picking for clustered strawberries; Fig. 2 : The overall framework of our work Traditional robotic picking methods [ 3 , 4 ] , which typically rely on precise modeling strategies, are not very fit to address the complexities of strawberry harvesting as mentioned above. The challenges posed by fragile fruit hidden among leaves, stems and unripe fruits, as well as the need for dexterous manipulation in cluttered environments, go beyond the capabilities of conventional techniques. These methods often fail to adapt to the dynamic nature of the task and the variability of the environment. In contrast, recent advancements in imitation learning techniques, such as the Action Chunking Transformer (ACT) [ 5 ] and Diffusion Policy (DP) [ 6 ] , have shown promise in tasks that involve soft and non-rigid objects. For example, ACT has been used for cloth folding and shoe tying, while DP has been applied to dishwasher loading, both of which require delicate manipulation in cluttered environments. These methods, by learning from a few human demonstrations, might offer the flexibility and adaptability needed to handle the complexities of strawberry picking. Unlike traditional approaches, which struggle to model every aspect of the environment, imitation learning can focus on learning the core skills required for successful strawberry harvesting, providing a more promising solution. In our work, we propose a learning-based robotic system tailored to the specific challenges of strawberry picking. Strawberries typically grow in a vertical arrangement, which allows a 4-DoF robotic arm, such as a Selective Compliance Articulated Robot Arm (SCARA) [ 7 ] , to perform the necessary picking tasks effectively [ 8 ] . Rather than relying on complex, high-degree-of-freedom manipulators that replicate the full range of human motion, our system utilizes a cost-effective 4-DoF design optimized for efficiency in commercial applications. A dedicated human teleoperation interface is used to collect demonstration data, and an End Pose Assisted Action Chunking Transformer method—built on the ACT algorithm—is developed to learn a dexterous visuomotor picking policy from these demonstrations. The overall framework of our work is shown in Figure 2 . Laboratory experiments under various occlusion conditions demonstrate that our learned policy outperforms the traditional percept-plan-control approach in picking occluded strawberries. The remainder of this paper is organized as follows. In Section 2, related work of strawberry picking and imitation learning are reviewed. Section 3 describes our robotic strawberry picking system and the associated imitation learning approach. In Section 4, we detail the experimental setup and compare our method with the traditional percept-plan-pick pipeline. Finally, Section 5 concludes the paper and outlines directions for future research. The contribution of this paper: • We developed a remotely operated data collection system for strawberry picking that integrates a specialized 4-DoF SCARA robotic arm with a human teleoperation interface. • We demonstrated the feasibility of a learning-based visuomotor policy for clustered strawberry picking using a 4-DoF Selective Compliance Articulated Robot Arm (SCARA), providing an effective alternative to common 6-DoF arms. • We demonstrate, to the best of our knowledge, the first successful learning and deployment of a dexterous visuomotor policy for picking real, clustered strawberries through imitation. • We enhance the Action Chunking Transformer method by incorporating an end pose loss and a neural network based inverse-kinematic module, resulting in a success rate improvement for the picking tasks. • We conduct a comprehensive study to evaluate the impact of various configured occlusion scenarios on the performance of the proposed algorithm. II Related Work II-A Robotic Strawberry Picking Strawberry picking traditionally relies on a modular robotic manipulation pipeline [ 9 , 4 , 3 ] that includes perception [ 10 ] , planning, picking [ 11 ] , and placing. The perception system first detects the fruits and localizes their positions relative to the robot arm [ 12 ] . Next, a planning system determines the picking order and generates trajectories to reach the target positions [ 13 ] . Finally, the end-effector executes the detaching operation and places the picked fruit into a collecting container [ 4 , 14 ] . This classic pipeline fits the most cases of strawberry picking. However, there exist some challenging scenarios, particularly when fruits are clustered, which has led many researchers to introduce advanced techniques to address these issues [ 8 ] . Active perception techniques have emerged as effective solutions for overcoming occlusion challenges in robotic fruit picking [ 15 ] . These methods dynamically adjust the robot’s viewpoint or sensor positioning to improve the visibility of target objects. For instance, Magistri et al. [ 16 ] propose an active perception approach that integrates 3D shape completion with a 6-DoF

Read the complete text