Soft Vision-Based Tactile-Enabled SixthFinger: Advancing Daily Objects Manipulation for Stroke Survivors

Basma Hasanen, Mashood M. Mohsan, Abdulaziz Y. Alkayas, Federico Renda, Irfan Hussain · 2025

Read Soft Vision-Based Tactile-Enabled SixthFinger: Advancing Daily Objects Manipulation for Stroke Survivors by Basma Hasanen, Mashood M. Mohsan, Abdulaziz Y. Alkayas, Federico Renda, Irfan Hussain (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-06806.txt.

Introduction

Five researchers wrote a robotics paper in two thousand twenty-five on an extra robotic finger for stroke survivors. Weak grasp after stroke makes daily objects hard to handle. Their answer is a soft sixth finger worn beside the hand, fitted with vision-based touch sensing inside its surface.

The finger adjusts its grip force on its own when it senses slipping. A transformer model at its core was trained on a wide tactile data set spanning shapes, sizes, weights, textures, and hardness. In real-world trials the device handled a range of everyday objects. The authors argue the approach could raise independence for chronic stroke survivors.

Catalog record

CREATORBasma Hasanen, Mashood M. Mohsan, Abdulaziz Y. Alkayas, Federico Renda, Irfan Hussain
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-06806

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Soft Vision-Based Tactile-Enabled SixthFinger: Advancing Daily Objects Manipulation for Stroke Survivors Basma Hasanen, Mashood M. Mohsan, Abdulaziz Y. Alkayas, Federico Renda, Irfan Hussain Abstract The presence of post-stroke grasping deficiencies highlights the critical need for the development and implementation of advanced compensatory strategies. This paper introduces a novel system to aid chronic stroke survivors through the development of a soft, vision-based, tactile-enabled extra robotic finger. By incorporating vision-based tactile sensing, the system autonomously adjusts grip force in response to slippage detection. This synergy not only ensures mechanical stability but also enriches tactile feedback, mimicking the dynamics of human-object interactions. At the core of our approach is a transformer-based framework trained on a comprehensive tactile dataset encompassing objects with a wide range of morphological properties, including variations in shape, size, weight, texture, and hardness. Furthermore, we validated the system’s robustness in real-world applications, where it successfully manipulated various everyday objects. The promising results highlight the potential of this approach to improve the quality of life for stroke survivors. I Introduction Stroke frequently results in hand impairments and loss of grasping function. The devastating aftermath of a stroke often manifests in the form of hemiparesis—a debilitating unilateral weakness that disrupts daily life and compromises fine motor functions, especially those associated with grasping. While various rehabilitation devices exist to aid in hand function recovery after a stroke, only a small percentage of stroke patients fully recover hand functionality after six months [ 1 ] . The prevailing compensatory aids are often single-purpose, bulky, and confined to structured environments, like rehabilitation centers [ 2 ] . Assistive robotic devices designed for stroke survivors emerge as a beacon of hope, epitomizing the potential of technology in medical compensation [ 2 , 3 , 4 , 5 , 6 ] . In [ 7 ] , a supernumerary robotic finger, called the SixthFinger, has been proposed to compensate for the missing grasping abilities in hemiparetic upper limb. Unlike traditional assistive devices that often serve a singular, focused purpose, this robotic finger is designed to be an all-encompassing solution. It can assist post-stroke patients to overcome the multifaceted challenges in daily bimanual tasks [ 2 ] —challenges that are not only physical but also psychological in nature [ 8 ] . Based on tests with real patients, providing an extra finger can potentially build confidence and a renewed sense of independence in the user [ 7 ] . Different control strategies has been proposed for the device. In [ 9 ] , a trigger-based control technique was used to control the SixthFinger. Although this strategy is straightforward and intuitive, the control interface involves the human hand thumb, restricting the use of the thumb in task completion. In [ 10 ] , another EMG electrodes interface has been developed to make the device’s control more intuitive. Nevertheless, the user is required to utilize their non-paretic arm to provide gestures for regulating the gripping force, a task that becomes impractical if the non-paretic arm is already engaged with the grabbed object. When the patient’s non-paretic hand is busy, they might struggle to maintain consistent manual control over the device’s exerted force. This inconsistency can lead to problems: insufficient force might cause the grasped object to slip, as illustrated in Fig. 2 . Additionally, the manual tuning of the grasping force increases the cognitive load of the patient. Fig. 1: (a) The VTE-SF system assists the stroke survivor to grasp an object (an apple) (b) The CAD design of VTE-SF. (c) The image captured by GelSight mini when the device touches the object. (d) The Sequence of frames captured by Gelsight mini when slippage of the object occurs. Fig. 2: The figure depicts a grasping failure scenario during liquid pouring when both functional and non-functional hands are used and force is not auto-regulated. In this work, to address the limitations mentioned above, the design of the SixthFinger is enhanced by adding a tactile sensor in the fingertip. The inclusion of tactile sensing for grasping simulates the natural feedback loop of a biological finger: when slip is detected, grip force increases in human grabbing, according to early research [ 11 ] . Commercial tactile sensors exhibit deficiencies in multiple aspects. Typically, the measurement focus of these sensors lies on normal force or pressure, rather than shear force, hence disregarding the tangential forces. The determination of incipient slip or entire slip is challenging in the absence of shear field measurements and the observation of its temporal evolution. Vision-based tactile sensors, however, expand this horizon by utilizing visual cues to interpret tactile interactions. Through integrated cameras and advanced algorithms, these sensors can provide a more detailed understanding of the interaction between the manipulator and the object in contact, allowing for better grip adjustments and more precise interactions. By providing real-time feedback on grasp stability, it not only ensures the safety of the object being held but also instills confidence in the user. Our contributions in this paper lie in: (1) A novel, soft, Vision-Based Tactile-Enabled SixthFinger system (VTE-SF). (2) A transformer-based grasping framework for object grabbing that can detect the slippage and modulate gripping force using tactile images. We conducted ablation studies to enhance the performance and accuracy of the proposed approach. (3) In addition, we experimentally tested the proposed system in handling different daily objects using the VTE-SF device and demonstrated its practicality. II Related Work II-A The Gelsight Vision-based Tactile Sensor GelSight is a vision-based tactile sensing modality developed by [ 12 ] . It is made up of four primary parts [ 13 ] : 1) a transparent elastomer piece with a reflective surface on one side, 2) a clear supporting plate made of either glass or acrylic for this elastomer, 3) consistent lighting typically supplied by LEDs, and 4) a camera positioned behind the support plate to record the image imprinted on the elastomer. Upon engagement with an object’s surface, the elastomer molds in a manner that reflects the topography of the interfaced surface. Such deformations, when illuminated, are optically captured using the camera. Gelsight mini sensor is used in this paper. Distinguished by its high spatial resolution (3280 x 2464 pixels), GelSight mini is synergetic with computer vision models. Further enhancing its utility is the soft contact surface, ensuring minimal invasiveness during interactions, and making it ideal for delicate tasks. Notably, despite its cutting-edge capabilities, GelSight remains a cost-effective solution. II-B Deep learning for slippage detection in grasping and tactile perception Slip, often resulting from insufficient grip strength or incorrect grip placement, has been a subject of extensive research [ 14 ] . Such slips indicate unstable holding during robotic manipulation. By recognizing or anticipating these slips, robots can adjust their grip technique and strength to ensure successful grasping. This significance has led to the development of numerous sensors [ 15 ] . Understanding these contact dynamics is crucial for effective robotic handling. As computer vision progresses and with the advent of optical tactile sensors, there’s a renewed approach to how tactile images are processed. By synergizing tactile data with vision-based techniques, enhanced outcomes have been observed. Calandra et al. in [ 16 , 17 ] utilizes a singular visual and tactile image. This approach anticipates grasp stability

Read the complete text