Vibro-Sense: Robust Vibration-based Impulse Response Localization and Trajectory Tracking for Robotic Hands

Wadhah Zai El Amri, Nicolás Navarro-Guerrero · 2026

Read Vibro-Sense: Robust Vibration-based Impulse Response Localization and Trajectory Tracking for Robotic Hands by Wadhah Zai El Amri, Nicolás Navarro-Guerrero (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20555.txt.

Introduction

Wadhah Zai El Amri and Nicolás Navarro-Guerrero present Vibro-Sense, a way for robot hands to feel touch through vibration. Rich contact sensing is crucial for manipulation, yet traditional tactile skins are expensive and hard to integrate. Their alternative localizes touch and tracks its path using vibro-acoustic signals.

Seven low-cost piezoelectric microphones are fitted to the hand. An audio spectrogram transformer decodes the vibration patterns made during physical contact. Cheap whole-hand touch could give warehouse and service robots the feedback they need to handle objects gently, without costly skins. The mics are cheap parts, not custom lab gear.

Catalog record

CREATORWadhah Zai El Amri, Nicolás Navarro-Guerrero
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-20555

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vibro-Sense: Robust Vibration-based Impulse Response Localization and Trajectory Tracking for Robotic Hands Wadhah Zai El Amri, Nicolás Navarro-Guerrero Abstract Rich contact perception is crucial for robotic manipulation, yet traditional tactile skins remain expensive and complex to integrate. This paper presents a scalable alternative: high-accuracy whole-body touch localization via vibro-acoustic sensing. By equipping a robotic hand with seven low-cost piezoelectric microphones and leveraging an Audio Spectrogram Transformer, we decode the vibrational signatures generated during physical interaction. Extensive evaluation across stationary and dynamic tasks reveals a localization error of under 5 mm in static conditions. Furthermore, our analysis highlights the distinct influence of material properties: stiff materials (e.g., metal) excel in impulse response localization due to sharp, high-bandwidth responses, whereas textured materials (e.g., wood) provide superior friction-based features for trajectory tracking. The system demonstrates robustness to the robot’s own motion, maintaining effective tracking even during active operation. Our primary contribution is demonstrating that complex physical contact dynamics can be effectively decoded from simple vibrational signals, offering a viable pathway to widespread, affordable contact perception in robotics. To accelerate research, we provide our full datasets, models, and experimental setups as open-source resources. keywords Tactile Sensing, Vibro-Acoustic Sensing, Contact Localization, Deep Learning 1 Introduction Humans possess a comprehensive system for perceiving their physical surroundings that extends beyond exteroceptive senses like vision and hearing. The somatosensory system provides crucial information about direct contact and environmental forces 1 ; 16 . When an object contacts with the body, the resulting mechanical vibrations propagate through the skin and tissues, enabling immediate localization and characterization of the interaction. This distributed vibro-acoustic sensing is fundamental not only for object manipulation but for overall spatial awareness and safe interaction within a dynamic environment. The ability to detect and interpret these contact events is a key component of effective physical interaction 25 . Drawing from this biological paradigm, the field of robot perception is increasingly exploring structure-borne sound analysis to supplement conventional vision and force modalities 2 ; 19 . The integration of piezoelectric sensors, particularly contact microphones, presents a promising avenue for capturing the complex dynamics of robot-environment interactions. By leveraging the unique properties of acoustic signals, researchers have made notable progress in developing vibro-acoustic sensing techniques for contact-rich manipulation 14 ; 15 ; 12 ; 22 . In this paper, we propose a cost-effective yet accurate method that enables robots to perceive physical contact in a more natural manner. We demonstrate our approach on two real-world tasks: impulse response localization and trajectory tracking. Our method utilizes a robotic hand equipped with seven contact microphones to capture vibrational signals, which are then processed using an Audio Spectrogram Transformer (AST) architecture to predict the positions of external touches on the hand. To train and evaluate this system, we collected extensive datasets consisting of over 65,000 unique samples for impulse response localization and over 240,000 interactions for trajectory tracking. Our contribution is two-fold. Firstly, we present a robust method for vibro-acoustic sensing that can be applied to complex robotic geometries. We provide a detailed analysis of how material properties, specifically stiffness versus texture, distinctly influence localization accuracy across different interaction modes. Secondly, we demonstrate the effectiveness of our approach in dynamic scenarios, where the robot hand actively moves while interacting with its surroundings. Our results show that the system maintains effective localization accuracy even in the presence of significant motion and actuator noise. By leveraging vibro-acoustic sensing and deep learning, our work provides a scalable solution for whole-body contact perception. To facilitate further research, we open-source all our code, datasets, and experimental setups on our website, allowing researchers and practitioners to easily replicate and build upon our work: wzaielamri.github.io/publication/vibrosense . Figure 1: Setup of the impulse response localization task. The UR5e robotic arm is shown applying controlled pokes to the hand using a solenoid actuator with a metal indenter. 2 Related Works For robots to operate safely and effectively in the physical world, they must be able to perceive and interpret contact within their environment. This sense of touch is crucial for enabling robots to securely manipulate objects and avoid unexpected or damaging collisions 7 . To address this, researchers have explored various sensing modalities, each with distinct advantages and trade-offs. The dominant approaches include creating large-area robotic skin to directly mimic biological touch and developing high-resolution vision-based sensors for detailed, localized contact analysis 6 . As a distinct alternative, vibro-acoustic sensing has emerged as a method for capturing the rich dynamics of interaction events, leveraging sound and vibration 11 . This section provides an overview of the state-of-the-art approaches, highlighting their respective capabilities and limitations. 2.1 Tactile Sensor Arrays and Robotic Skin Tactile sensor arrays and robotic skin technologies enable robots and prosthetic devices to sense and interpret touch, mimicking how human skin detects pressure, texture, and temperature. These systems are built from grids of sensitive elements embedded in flexible materials, enabling accurate, responsive detection of physical contact and environmental interactions. An example is Touchlab 20 , which develops ultra-thin electronic skin. Their sensors equip robots with real-time touch sensing, enabling remote control with haptic feedback for healthcare and hazardous environments. Hardman et al. 6 developed another advanced tactile skin solution that uses a soft hydrogel embedded with over 860,000 conductive pathways to detect pressure, temperature, and damage, enabling highly sensitive, adaptive touch sensing for robots and prosthetics. However, these methods face significant drawbacks. They are often expensive to produce and implement, and their delicate construction makes them fragile and susceptible to damage from routine physical interaction. Furthermore, the sensing capability is localized, meaning it is confined only to the specific areas where the robotic skin is applied, limiting the robot’s overall environmental awareness 3 . 2.2 Vision-Based Tactile Sensing Vision-based tactile sensing takes a different approach by inferring contact information from camera observations of a deformable material. This approach can reconstruct detailed 3D contact geometry and force distribution from images. Examples of such sensors include GelSight 26 , which uses a camera to capture fine-grained texture and shapes of an illuminated elastomer. More compact designs such as DIGIT 10 have helped standardize this approach and broaden its use in manipulation research. These sensors provide rich data well-suited for grasping and in-hand manipulation, but they remain inherently local. Due to their bulk and reliance on internal optics, they are inherently difficult to scale for large surface areas and are not well-suited for whole-body tactile sensing. 2.3 Vibro-Acoustic Sensing in Robotics Distinct from surface-based electronic or visual methods, vibro-acoustic sensing exploits structure-borne sound to detect physical interactions. Contact microphones convert mechanical vibrations propagating through the robot body into electrical signals, providing a lightweight, inexpensive, and highly scalable sensing modality. It can provide a complementary source of information to vision and force sensors, especially in occluded settings 19 . In the following, we review relevant microphone-based work on (i) impulse response localization, which estimates contact locations from transient vibration patterns and (ii) trajectory estimation during contact-rich tasks, which tracks continuous motion from ongoing acoustic signals. 2.3.1 Impulse Response Localization Instead of covering the robot with dense arrays, impulse localization estimates contact coordinates by decoding the structural vibrations captured by a sparse microphone array. For instance, SonicBoom 11 equips a robot end-effector with a

Read the complete text