Enhancing Physical Human-Robot Interaction: Recognizing Digits via Intrinsic Robot Tactile Sensing

Teresa Sinico, Giovanni Boschetti, Pedro Neto · 2025

Read Enhancing Physical Human-Robot Interaction: Recognizing Digits via Intrinsic Robot Tactile Sensing by Teresa Sinico, Giovanni Boschetti, Pedro Neto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-00167.txt.

Introduction

Sinico, Boschetti, and Neto present a 2025 robotics paper on touch reading without extra sensors. Collaborative robots already carry torque sensors in their joints. The team used those built-in signals to read digits that people draw by hand. Users traced numbers zero to nine on a plain pad fixed to the robot arm.

Their data set pairs joint torque traces with forces at the tool tip from many writers. A trick that flips and rotates samples guards against odd drawing styles. A memory network sorted the digits online with ninety-four percent accuracy, even for new writers. A real robot then used the skill in a fruit delivery task for people with limited movement.

Catalog record

CREATORTeresa Sinico, Giovanni Boschetti, Pedro Neto
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-00167

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing Physical Human-Robot Interaction: Recognizing Digits via Intrinsic Robot Tactile Sensing Thanks: Citation : Authors. Title. Pages…. DOI:000000/11111. Teresa Sinico, Giovanni Boschetti, Pedro Neto Abstract Physical human-robot interaction (pHRI) remains a key challenge for achieving intuitive and safe interaction with robots. Current advancements often rely on external tactile sensors as interface, which increase the complexity of robotic systems. In this study, we leverage the intrinsic tactile sensing capabilities of collaborative robots to recognize digits drawn by humans on an uninstrumented touchpad mounted to the robot’s flange. We propose a dataset of robot joint torque signals along with corresponding end-effector (EEF) forces and moments, captured from the robot’s integrated torque sensors in each joint, as users draw handwritten digits (0–9) on the touchpad. The pHRI-DIGI-TACT dataset was collected from different users to capture natural variations in handwriting. To enhance classification robustness, we developed a data augmentation technique to account for reversed and rotated digits inputs. A Bidirectional Long Short-Term Memory (Bi-LSTM) network, leveraging the spatio-temporal nature of the data, performs online digit classification with an overall accuracy of 94% across various test scenarios, including those involving users who did not participate in training the system. This methodology is implemented on a real robot in a fruit delivery task, demonstrating its potential to assist individuals in everyday life. Dataset and video demonstrations are available at: https://TS-Robotics.github.io/pHRI-DIGI/ . K eywords Physical human-robot interaction \cdot intrinsic sensing \cdot tactile sensing \cdot digit classification \cdot robotics 1 Introduction Physical human-robot interaction (pHRI) has the potential to be a major enabler of natural interaction and collaboration between humans and robots in real-world scenarios [ 1 ] . However, achieving intuitive and safe pHRI remains a significant challenge due to several factors, including safety concerns and the lack of reliable sensors targeting pHRI. Current collaborative robot safety directives establish operational guidelines, but in light of recent advancements, the naturalness and effectiveness of human-robot interaction still have significant room for improvement [ 2 ] . Most collaborative robots are capable of detecting contact with humans. However, physical interaction is typically limited to hand-guiding, and most applications focus primarily on preventing contact rather than enabling, classifying, and leveraging it. Figure 1: Conceptual overview of the proposed pHRI system: the system uses intrinsic robot torque sensing to recognize digits (0 - 9) drawn on a touchpad, including standard, reversed, and rotated variations. Figure 2: System architecture illustrating the data flow from the robot’s intrinsic joint torque sensing, through dataset collection and data augmentation, into the Bi-LSTM classification network. The classified patterns are then used to define the robot interface and guide the corresponding task execution. Over the past decades, force/torque sensors and touch/tactile sensing devices have been successfully used for pHRI tasks such as hand-guiding and touch detection [ 3 , 4 ] . More recently, these sensing technologies, combined with machine learning, have been effectively used as robotic electronic skin (e-skin) to detect and classify touch patterns. While these technologies have shown promise for enhancing pHRI, their integration complexity, along with limited mechanical flexibility and durability, poses challenges for seamless integration into robotic systems. In this context, considering both the limited applicability of the aforementioned sensing technologies and the integrated sensing capabilities of today’s collaborative robots, two important questions arise: 1. Are these sensing technologies truly necessary to enable meaningful pHRI? 2. Could we instead leverage the intrinsic tactile sensing already present in collaborative robots, such as joint torque measurements, to enable richer and more intuitive physical interactions beyond simple contact detection? Inspired by recent pioneering work on intrinsic robot sensing [ 5 ] , this study leverages the intrinsic tactile sensing capabilities of collaborative robots to recognize digits drawn by humans, by sliding their finger, on an uninstrumented touchpad mounted to the robot’s flange, Fig. 1 . The primary goal of this study is to assess whether the sensed joint torques, and the corresponding EEF forces and moments, are sufficiently sensitive to accurately classify the digits. We also investigate the generalizability of the classification model, specifically its online performance with users who did not participate in the training phase and with digits drawn in reversed or rotated orientations. A detailed overview of the system architecture, including the data processing pipeline and the Bi-LSTM network, is provided in Fig. 2 . The key contributions of this paper are as follows: • An intuitive pHRI solution based on the intrinsic tactile sensing of collaborative robots: handwritten digits (0 - 9) are recognized by sliding the finger on an uninstrumented touchpad. • Publicly available dataset pHRI-DIGI-TACT: it includes 10 classes of tactile handwritten digits (0–9) collected from multiple participants. Each class contains data on 7-DOF robot joint torques, EEF forces and moments. • Enhanced robustness in classifying reversed and rotated digits: a data augmentation method enables the recognition of such variations when applied to the standard pHRI-DIGI-TACT dataset, increasing the system’s flexibility and general applicability. • High-accuracy online digit classification: a bi-LSTM network achieves an overall accuracy of 94% across various test scenarios, including those involving users who did not participate in the training phase. • Real-world application: a robotic cell for object delivery, where the user interacts through the proposed interface to select the desired items. The robot is controlled via digits drawn on the touchpad, while the system provides feedback through text-to-speech voice output. These natural interaction methods demonstrate the intuitiveness of the interface, highlighting its potential to assist inexperienced users in everyday tasks. 2 Related Work Physical human-robot interaction has gained increasing research interest in recent years due to its promising potential. Researchers are actively exploring both novel sensing technologies and methods for interpreting touch/tactile data within the robotics context, some of which powered by machine learning. The ability to sense and interpret physical contact between the robot’s structure and its surrounding environment, including interactions with humans, can open the door to new robotic applications and broaden accessibility for non-skilled users. Promising approaches to enhancing pHRI involve equipping robots with external tactile sensors, such as e-skin, to provide high-resolution tactile feedback. Notable examples include a control framework utilizing high-resolution e-skin that covers the entire robot body [ 6 ] , elastomeric e-skin with variable stiffness for safer human-robot interaction [ 7 ] , and robot e-skin solutions employing large-area tactile sensors to recognize human touch [ 8 ] . Soft and stretchable e-skin materials and their associated sensors offer adaptability to complex robot surfaces and are capable of integrating multimodal sensing, delivering high-resolution output, and providing extensive coverage [ 9 ] . Although e-skin sensors offer multiple advantages, they are not yet commonly available in commercial robots due to several challenges. These include difficulties in sensor integration, ensuring long-term reliability, managing system complexity, achieving adaptability to varying environmental conditions, and providing the mechanical flexibility needed to conform to curved robot surfaces. Combining robotics with recent developments in e-skin technology and machine learning holds great promise for addressing these challenges and advancing the real-world applicability of pHRI. A common approach involves mounting force/torque sensors on the robot’s flange to measure forces and torques at the robot EEF [ 10 , 11 ] . While practical, this method is limited in its ability to detect touch beyond the immediate vicinity

Read the complete text