TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments
Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal, Taşkın Padır · 2026
Read TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments by Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal, Taşkın Padır (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24712.txt.
Introduction
Shivani Kamtikar and five colleagues present TACTFUL, a robot system that finds and identifies objects by touch alone. People do this easily, naming keys or a bottle inside a cluttered bag without looking. Robots, by contrast, lean heavily on vision and struggle in confined or dark spaces. TACTFUL uses a multi fingered hand with tactile sensors. It explores a confined workspace alone, finds objects through contact, and rebuilds their shapes from touch. Training happened entirely on real hardware, with no simulation.
One learned policy drives the exploration. A dynamic reward schedule balances wide searching of the workspace with close refinement of each surface found. On real world objects the system reached a 77 percent success rate. Its average reconstruction error was 0.015 metres. It also beat the baseline approaches tested. Touch, paired with structured learning, can thus serve as a primary sense, not a camera backup.
Catalog record
| CREATOR | Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal, Taşkın Padır |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.24712
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TACTFUL: Tactile-Driven Exploration For Object Localization and Identification in Confined Environments Shivani Kamtikar, Chung Hee Kim, Camilla Tabasso, Tye Brady, Joshua Migdal, Taşkın Padır Abstract Humans effortlessly locate and identify objects by touch alone, even without vision. In contrast, robotic systems rely heavily on vision and struggle with autonomous tactile exploration and object identification. We present TACTFUL, a vision-free tactile exploration framework that enables a multi-fingered robot to autonomously explore confined workspaces, discover objects through contact, and identify them via tactile reconstruction. Trained entirely on real hardware without simulation, our system learns a single policy that balances global workspace exploration with local surface refinement through a dynamic reward schedule. Our results demonstrate that tactile sensing, when paired with structured learning, can serve as an effective primary modality for object-level reasoning, achieving 77% success with 0.015 m average reconstruction error and outperforming baseline approaches on real-world objects. I INTRODUCTION Humans possess a remarkable ability to locate and identify objects in visually occluded scenarios, through a combination of tactile sensing and prior geometric knowledge, effortlessly distinguishing between a set of keys and a water bottle in a cluttered bag. Tactile sensors offer direct, meaningful feedback about contact interactions and local surface geometry [ 3 , 28 , 14 ] , particularly in occluded or confined settings. Yet, robotic manipulation has traditionally relied heavily on vision, with tactile sensing often used as a secondary or corrective modality [ 10 , 11 , 1 , 15 ] . While prior work has explored tactile-based object mapping and identification [ 2 , 16 , 26 , 17 ] , autonomous tactile exploration for object-level reasoning remains underexplored in real-world settings. Moreover, most prior tactile systems have primarily been evaluated in simulation using simplified or binary contact signals, with comparatively limited work demonstrating real-world performance using high-fidelity tactile data [ 31 , 20 ] . Consider a robot exploring a confined workspace containing multiple objects. When visual perception is available, object localization and identification are relatively straightforward, as camera-based systems can provide dense scene observations. In vision-denied scenarios, robots must rely on local tactile feedback to explore workspaces, localize objects, and reconstruct geometry. This requires simultaneously addressing workspace exploration for object discovery and contact-driven surface reconstruction for identification. Fig. 1 : (a) Active tactile exploration by the agent at various timesteps (no vision), (b) tactile point cloud accumulated at timestep t , and (c) shape-completed reconstruction, R_{t} (red), overlaid on the ground truth geometry, R_{GT} (green) We present TACTFUL , a real-world, vision-free tactile exploration and identification framework for settings where objects are drawn from a known object library but must be located without vision, such as warehouse bin-picking, medical instrument retrieval, or in manufacturing industries. Our framework makes the following contributions: 1) we develop a learned tactile exploration policy that enables a multi-fingered robot to autonomously explore and localize objects within confined workspaces, 2) we introduce a dynamic reward schedule that balances global exploration with local contact refinement, leveraging object-level geometric priors to guide 3D reconstruction, and finally, 3) we demonstrate reliable object identification based solely on reconstructed tactile geometry. II Related Work Fig. 2 : Pipeline of TACTFUL : (a) Offline behavior cloning (BC) model trained using proprioception, tactile signals, and tactile point cloud; used to initialize the actor in the PPO-based reinforcement learning (RL) policy. (b) Online PPO training with the same inputs and an additional region of interest (ROI) for the critic, resulting in an exploration policy for object-guided reconstruction in confined spaces. Tactile-based localization. Tactile sensing has been explored as a means for object localization when visual information is unreliable or unavailable. Early work demonstrated tactile SLAM using whisker-inspired sensors [ 8 ] , while subsequent approaches employed simple tactile probes for active localization [ 5 , 24 , 32 ] . However, these methods typically rely on low-dimensional contact signals and lack the rich spatial information afforded by multi-fingered tactile hands. Kissoum et al. [ 12 ] addressed simultaneous localization and reconstruction using particle filters, but their approach is limited to 2D simulated environments with simplified contact dynamics. In contrast, our work leverages high-resolution, real-world tactile sensing to jointly localize and reconstruct objects through learned exploration policies. Object reconstruction using tactile data. Reconstructing object geometry from touch has gained increasing attention in vision-denied settings, with some works exploring efficient shape exploration using rigid tactile arrays [ 7 ] and learning-based methods that fuse contact positions and normals to build 3D object representations [ 31 ] . Other approaches focus on reconstructing visually challenging objects, including transparent objects [ 19 ] . More recent systems, such as TactoFind [ 20 ] , demonstrate tactile-only object retrieval in clutter, but depend on hand-crafted exploration heuristics rather than learned policies. However, the majority of tactile reconstruction methods assume that the object has already been localized and focus exclusively on surface exploration. Our work addresses a more general and realistic setting in which the robot must first explore an unknown workspace to discover objects and then perform detailed tactile interaction to reconstruct their geometry, requiring policies that tightly couple global exploration with local shape refinement. Deep reinforcement learning for tactile perception. Deep reinforcement learning has been widely applied to tactile-driven perception and exploration tasks. Prior work has highlighted the importance of memory for sequential tactile decision-making, using recurrent architectures such as LSTM-augmented A3C [ 18 , 22 ] . More recent approaches leverage attention mechanisms to generalize tactile perception across manipulation primitives [ 24 ] . Similarly, Lee et al. [ 13 ] apply reinforcement learning for occluded object retrieval, but focus on reactive behaviors rather than systematic object reconstruction. In the context of exploration, intrinsic motivation strategies such as D-optimality [ 21 ] and contact-based exploration bonuses [ 26 ] have been proposed to encourage informative interactions. However, these methods typically operate under the assumption that object locations are known and focus on localized surface exploration. In contrast, our approach learns a single tactile-only policy that explores an entirely unknown workspace, localizes objects through contact, and incrementally reconstructs their geometry in the real world using a structured reward design that balances search, interaction, and shape understanding. III Tactile Robotic Setup Our system, shown in Fig. 1 , consists of a five-fingered Inspire RH56DFTP Series Dexterous hand mounted on a UR10e robotic arm. The robotic hand is equipped with resistive 1062 high-resolution taxels on the fingertips, fingerpads, and the palm area, measuring normal contact forces. The workspace consists of a bin with known bounds having 3D-printed objects placed in arbitrary locations. We do not use any cameras. IV Method We address vision-free object localization and identification in confined workspaces containing multiple objects. Given a bin of objects drawn from a known library, the robot is assigned a target object and must locate it using only tactile feedback. During exploration, it may encounter distractor objects and must distinguish the target through accurate tactile reconstruction. This requires jointly exploring the workspace to discover objects via contact and reconstructing surface geometry with sufficient fidelity for reliable identification. TACTFUL learns a single reinforcement learning (RL) policy that enables a multi-fingered robot to autonomously explore and identify objects using tactile sensing alone. The policy is trained with a dynamic