Visual IRL for Human-Like Robotic Manipulation <<submission id>>
Ehsan Asali, Prashant Doshi · 2024
Read Visual IRL for Human-Like Robotic Manipulation <<submission id>> by Ehsan Asali, Prashant Doshi (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11360.txt.
Introduction
Ehsan Asali and Prashant Doshi present a way for factory robots to copy human workers in a 2024 paper. Teaching a robot by showing it a task is faster than programming every move. Their method learns from watching a person perform the task on video with depth data.
Each video frame gives 3D points on the person's body and the objects. Those points feed a learning method that works backward from actions to the reward the person seems to follow. A special model then maps human arm motion onto the robot arm's joints. The goal is not just to place the tool correctly but to move in a natural, human-like way. The team tested the approach on two real tasks: sorting onions by quality and pouring liquid. The results show the robot can pick up both the goal and the style of the work.
Catalog record
| CREATOR | Ehsan Asali, Prashant Doshi |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-11360
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Visual IRL for Human-Like Robotic Manipulation <<submission id>> Ehsan Asali, Prashant Doshi Abstract . We present a novel method for collaborative robots (cobots) to learn manipulation tasks and perform them in a human-like manner. Our method falls under the learn-from-observation (LfO) paradigm, where robots learn to perform tasks by observing human actions, which facilitates quicker integration into industrial settings compared to programming from scratch. We introduce \virl that uses the RGB-D keypoints in each frame of the observed human task performance directly as state features, which are input to inverse reinforcement learning (IRL). The inversely learned reward function, which maps keypoints to reward values, is transferred from the human to the cobot using a novel neuro-symbolic dynamics model, which maps human kinematics to the cobot arm. This model allows similar end-effector positioning while minimizing joint adjustments, aiming to preserve the natural dynamics of human motion in robotic manipulation. In contrast with previous techniques that focus on end-effector placement only, our method maps multiple joint angles of the human arm to the corresponding cobot joints. Moreover, it uses an inverse kinematics model to then minimally adjust the joint angles, for accurate end-effector positioning. We evaluate the performance of this approach on two different realistic manipulation tasks. The first task is produce processing, which involves picking, inspecting, and placing onions based on whether they are blemished. The second task is liquid pouring, where the robot picks up bottles, pours the contents into designated containers, and disposes of the empty bottles. Our results demonstrate advances in human-like robotic manipulation, leading to more human-robot compatibility in manufacturing applications. 1. Introduction Several factors are known to contribute to successful and sustainable human-robot collaborations ( Lin and Lukodono 2021 ; Ajoudani et al. 2018 ) . One of these is whether the collaborative robot is likable. Robots tend to be likable when they exhibit social intelligence and act in ways that make the human partner comfortable. One of these ways is for the robot to use naturalistic, human-like, motions in performing its tasks. Figure 1. Overview of the proposed method’s pipeline: The pipeline begins by learning the task from a human expert that starts with capturing the RGB-D stream during the human’s task performance. Object location prediction and human keypoint detection models extract state parameters for AIRL, which learns a reward function that reflects the expert’s preferences. When deployed on the cobot, the learned reward function is used to achieve a policy to perform the task. The calculated policy is input into the Neuro-Symbolic Dynamics Mapping model, which maps human joints to cobot joints and generates initial joint angles for the cobot. These angles are refined using the cobot joints IK model to adjust the end-effector positioning for accurate manipulation, allowing the cobot to perform the task successfully. Toward achieving this goal, we adopt the learn-from-observation (LfO) paradigm, which is a human-inspired methodology where robots learn to perform tasks by observing humans perform them. This approach can facilitate quicker integration of collaborative robots (cobot) in industrial settings without extensive programming from scratch. In this paper, we present a novel approach for cobots to learn manipulation tasks and execute them in a naturalistic, human-like manner. Our method leverages real-time human pose estimation and object detection on every RGB-D frame of the observed human task performance. Three-dimensional keypoints from these frames are extracted, polished, and then used directly as state features input into an adversarial inverse reinforcement learning (AIRL) technique ( Fu et al. 2018 ) . This technique learns a reward function that is interpreted as human preferences for the performed task. The learned reward function and corresponding behavior, which maps the pose keypoints to actions in terms of human wrist joint angles, is transferred to a cobot using a novel neuro-symbolic dynamics model. This model maps human kinematics to a cobot arm, allowing for similar end-effector positioning while minimizing joint adjustments, thus preserving the natural dynamics of human motion in robotic tasks. An overview of our proposed method’s pipeline is depicted in Fig. 1 . Our methodology offers two key contributions. First is the visual IRL that subscribes to the LfO paradigm and performs IRL directly with human pose keypoints as state features obtained from a state-of-the-art standard real-time pose estimation model such as YOLO ( Wang et al. 2023 ) . We demonstrate that IRL can scale to a stream of such state features and yield an accurate reward function. This mitigates the need to train custom state-action recognition models such as SA-Net ( Soans et al. 2020 ) or MVSA-Net ( Asali et al. 2023 ) to process video frames and obtain high-level states and actions as input to IRL algorithms. Second is the neuro-symbolic dynamics mapping model . This model enables the transfer of human arm motion dynamics to a cobot with differing degrees of freedom (DoF), maintaining the human-like characteristics of the motion. Unlike previous techniques that focus solely on end-effector placement, our method maps multiple joint angles of the human arm to corresponding cobot joints. After mapping the joints and calculating the initial joint angles for the cobot, an inverse kinematics model is used to make adjustments for accurate end-effector positioning with minimum changes to the initial joint angles. This approach ensures that the cobot performs the task while adopting the style and fluidity of human movement. In the Experiments section, we examine this methodology for learning two manipulation tasks with real-world applications and performing each with a different cobot. One task is produce processing which involves picking, inspecting, and placing onions at different locations based on whether the onion is blemished or not ( Asali et al. 2023 ; Sengadu Suresh et al. 2023 ) . Recordings of humans sorting onions in the lab are passed through the method and used to train the 7 DoF Sawyer cobot to perform onion sorting with human-like motion. The second task is liquid pouring which involves picking up colored bottles, pouring contents into a corresponding container (based on the color of the bottles), and disposing of the empty bottles into a bin. A 6 DoF KUKA cobot (LBR iisy) is used to perform the task in a human-like manner. These tasks are chosen due to their relevance to manufacturing automation and inherent complexity due to variations in human judgment and motion. We rigorously evaluate our method using multiple metrics, including Learned Behavior Accuracy (LBA), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), coefficient of determination (R²), and we also measure the averages of end-effector displacement, movement jerkiness, and the time in which it takes to perform the sorting action and compare it with the baseline path planner. The evaluation demonstrates advances in achieving human-like sorting actions, highlighting the robustness and applicability of our method in real-world scenarios. We summarize our contributions as follows: • We present an end-to-end perception-based inverse reinforcement learning approach that enables cobots to perform manipulation tasks in a human-like manner. • We develop a novel neuro-symbolic dynamics mapping model that enables transferring human arm motion dynamics to a cobot with different DoFs. • We conduct real-world experiments on two different cobots namely Sawyer and KUKA LBR iisy with 7 and 6 DoFs, respectively, showing