Collaborative Object Handover in a Robot Crafting Assistant
Leimin Tian, Shiyu Xu, Kerry He, Rachel Love, Akansel Cosgun, Dana Kulić · 2025
Read Collaborative Object Handover in a Robot Crafting Assistant by Leimin Tian, Shiyu Xu, Kerry He, Rachel Love, Akansel Cosgun, Dana Kulić (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19991.txt.
Introduction
Leimin Tian, Shiyu Xu, Kerry He, Rachel Love, Akansel Cosgun, and Dana Kulić teach a robot to hand objects to a person the way people do. Handovers pepper collaborative work, from restaurants to assembly lines. Safe, efficient transfers need the robot to read human context, not just positions. The team collected teleoperation data in a naturalistic crafting task.
A model trained on that human-guided data then ran the handovers itself. Cross-validation on the data set came first. A user study in the same crafting task followed, comparing autonomous and teleoperated episodes. Users' perceptions were measured alongside task success. The autonomous policy achieved genuinely collaborative handovers. Yet the comparison with human control exposed clear avenues for improvement. The paper maps both the competence reached and the social polish still missing.
Catalog record
| CREATOR | Leimin Tian, Shiyu Xu, Kerry He, Rachel Love, Akansel Cosgun, Dana Kulić |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-19991
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Collaborative Object Handover in a Robot Crafting Assistant Leimin Tian, Shiyu Xu, Kerry He, Rachel Love, Akansel Cosgun, Dana Kulić Abstract Robots are increasingly working alongside people, delivering food to patrons in restaurants or helping workers on assembly lines. These scenarios often involve object handovers between the person and the robot. To achieve safe and efficient human-robot collaboration (HRC), it is important to incorporate human context in a robot’s handover strategies. Therefore, in this work, we develop a collaborative handover model trained on human teleoperation data collected in a naturalistic crafting task. To evaluate the performance of this model, we conduct cross-validation experiments on the training dataset as well as a user study in the same HRC crafting task. The handover episodes and user perceptions of the autonomous handover policy were compared with those of the human teleoperated handovers. While the cross-validation experiment and user study indicate that the autonomous policy successfully achieved collaborative handovers, the comparison with human teleoperation revealed avenues for further improvements. I Introduction Object handover is a common task in human-robot collaboration (HRC), in which an item is transferred between a human and a robot [ 1 ] . To develop an autonomous human-robot handover policy, existing approaches either react to pre-defined triggers (e.g., specific hand gestures [ 2 ] ), or train machine learning models using data collected from isolated handover episodes between two people or in human-robot pairs (e.g., [ 3 , 4 ] ). However, using pre-defined triggers imposes learning costs for the person and interrupts the flow of activities, requiring additional physical and mental effort for the person to explicitly command the robot [ 5 ] . Further, as handovers are context-dependent [ 1 ] , training from isolated episodes limits the resulting model’s applicability in handovers that occur in naturalistic contexts with pre- and post-handover tasks. There has been growing interest in the social-interactive aspects of handovers in addition to the control and trajectory planning aspects, focusing on how a person and a robot may communicate and coordinate during handovers [ 6 , 7 ] . While these studies investigated communicating information such as intention or hesitation during individual episodes of object transfer, how people and robots collaborate in a complex task involving multiple handovers remains largely under explored. Handovers in a broader task context require providing objects at the right place and the right time, when the person may be engaged in the task and not giving explicit cues to the robot. Further, research on human behaviours in handovers largely focuses on hand locations, while additional informative behaviours such as gaze or whole body movements are under explored [ 8 ] . Evaluation of human-robot handover policies is often conducted in a short-term, handover-centred design without incorporating human and task contexts, e.g., focusing on the smoothness of the robot’s motions and trajectories [ 9 ] . This results in a gap between a handover policy’s outcomes in isolated episodes compared to when multiple handovers are serving an overall task goal. For example, a policy that emphasises optimising the instantaneous ergonomics risk factors in individual handovers may become over-assistive and reduce worker well-being in repeated interactions [ 10 ] . \HyColor@XZeroOneThreeFour \pc @goptd@color \pc@hyenc@color pdfcommentcolor \HyColor@XZeroOneThreeFour \pc @goptd@fontcolor \pc@hyenc@fontcolor pdfcommentcolor \HyColor@XZeroOneThreeFour \pc @goptd@icolor \pc@hyenc@icolor pdfcommentcolor \pdfmark [ ]pdfmark=/ANN,Subtype=/Widget,Raw=/TU (Top half of the image shows a photo of the experimental layout, with the work space where participants sit on the left and storage space where the experimenter sits on the right. The Fetch robot is moving between these two desks. A partially built birdhouse can be seen on top of the work desk. A tripod with a camera on top stands next to the work desk. A storage box can be seen next to the storage space. The robot also has a camera on its head. Below the photo, the state transitions during a handover episode are shown, with three timing adaptations and two handover location adaptations highlighted at key state transitions. \textCR ( \pc @goptd@deadline)) /T (tooltip zref@0) /C [ ] /FT/Btn /F 768 /Ff 65536 /H/N /BS ¡¡ /W 0 ¿¿ Fig. 1: Our autonomous handover policy performs three temporal adaptations: episode start, object transfer start, and object transfer completion timing. Spatial adaptation of object transfer position (left, middle, right handovers) is first performed at episode start, then updated at object transfer start. Therefore, we are motivated to understand human-robot handovers in a naturalistic task context where they serve a collaborative goal. In particular, we develop an autonomous handover policy, as shown in Figure 1 , trained with the Functional And Creative Tasks Human-Robot Collaboration dataset (the FACT HRC dataset) [ 11 ] . In the FACT HRC dataset, a mobile manipulator robot was teleoperated by a hidden human operator to serve as a crafting assistant to 20 users, each in an hour-long interaction session that involved multiple handovers of diverse objects used for the crafting task. We then replicate the crafting assistant task to evaluate the autonomous handover model with 20 participants. Further, by comparing the autonomous handover policy and the human operator’s handover strategies, we investigate design implications to improve collaborative handovers. Our main contributions are: 1. We developed a collaborative handover policy for a mobile manipulator robot, which adapts the handover timing and object transfer location based on a user’s upper body movements using supervised learning; 2. We conducted a user study and evaluated the developed autonomous policy during multiple purposeful handovers with mixed robot roles in a collaborative task context; 3. We compared the objective performance and subjective user perception of autonomous and teleoperated handovers, which informs future HRC research; 4. The code and data are open-sourced 1 1 1 https://doi.org/10.26180/25449640 . II Related Work As reviewed by Duan et al. [ 9 ] , recent handover research investigates how different methods for trajectory generation, motion planning, or timing control influence the objective handover performance (e.g., duration or success rate) and subjective human perceptions (e.g., trust or fluency). Existing studies focus on either human-to-robot (H2R) handovers (52.5%) or robot-to-human (R2H) handovers (37.5%), with limited research on bidirectional handovers (10%), i.e., exchanging objects. In H2R handovers, previous work focused on predicting human behaviours or intentions under uncertainty with machine learning models (e.g., [ 12 , 13 ] ). In R2H handovers, previous work focused on enhancing human comfort and increasing the movement efficiency of both parties [ 14 , 15 ] . Existing handover policies may adopt learning-based, control-based, or analysis-based approaches. For example, Käppler et al. [ 16 ] developed an adaptive method for a table-mounted robot arm, which updated the object transfer location based on the location of a user’s hand as observed from an RGB-D camera and proximity sensors on the gripper. This adaptive R2H handover model was compared with a non-adaptive model using a pre-defined object transfer location in a between-subject study, in which the robot handed a cup to a participant in four consecutive repetitions. They found that participants exhibited motor learning and adaptation to the robot’s handover, which necessitated evaluating handover models in repeated episodes. In another example, Kedia et al. [ 17 ] developed a transformer-based human intent prediction model pre-trained on human-human collaborative manipulation data and fine-tuned on human-robot