UniTac-NV: A Unified Tactile Representation For Non-Vision-Based Tactile Sensors*
Jian Hou, Xin Zhou, Qihan Yang, and Adam J. Spiers · 2025
Read UniTac-NV: A Unified Tactile Representation For Non-Vision-Based Tactile Sensors* by Jian Hou, Xin Zhou, Qihan Yang, and Adam J. Spiers (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-19699.txt.
Introduction
Robot touch sensors come in many designs, and software trained for one rarely works on another. Jian Hou, Xin Zhou, Qihan Yang, and Adam Spiers present a shared format in 2025 for touch sensors that do not use cameras. Their system gives each sensor its own encoder, then maps all readings into one common inner space.
They trained the system on two commercial sensors mounted on a robot arm, pressing shapes made of hard plastic and soft rubber. Because the inner space is shared, a model trained on one sensor's data could read contact shape from the other sensor without retraining. Tests included a harder unseen object to check generalization. The authors see the shared space as a base for portable touch skills.
Catalog record
| CREATOR | Jian Hou, Xin Zhou, Qihan Yang, and Adam J. Spiers |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-19699
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UniTac-NV: A Unified Tactile Representation For Non-Vision-Based Tactile Sensors* Jian Hou, Xin Zhou, Qihan Yang, and Adam J. Spiers Abstract Generalizable algorithms for tactile sensing remain underexplored, primarily due to the diversity of sensor modalities. Recently, many methods for cross-sensor transfer between optical (vision-based) tactile sensors have been investigated, yet little work focus on non-optical tactile sensors. To address this gap, we propose an encoder-decoder architecture to unify tactile data across non-vision-based sensors. By leveraging sensor-specific encoders, the framework creates a latent space that is sensor-agnostic, enabling cross-sensor data transfer with low errors and direct use in downstream applications. We leverage this network to unify tactile data from two commercial tactile sensors: the Xela uSkin uSPa 46 and the Contactile PapillArray. Both were mounted on a UR5e robotic arm, performing force-controlled pressing sequences against distinct object shapes (circular, square, and hexagonal prisms) and two materials (rigid PLA and flexible TPU). Another more complex unseen object was also included to investigate the model’s generalization capabilities. We show that alignment in latent space can be implicitly learned from joint autoencoder training with matching contacts collected via different sensors. We further demonstrate the practical utility of our approach through contact geometry estimation, where downstream models trained on one sensor’s latent representation can be directly applied to another without retraining. I INTRODUCTION Tactile sensing can enhance robotic perception, especially in scenarios involving visual occlusions or limited lighting. In addition, some object properties are inherently challenging to infer via vision, such as detailed surface shape, friction, stiffness and texture. Researchers have demonstrated that a wide range of use cases benefit from tactile sensing, including object pose estimation/contact characterization during in-hand manipulation (where fingers frequently occlude objects) [ 1 , 2 , 3 ] , object identification and characterization [ 4 , 5 , 6 , 7 ] , and grasp stability estimation [ 8 , 9 , 10 ] . Despite their importance, tactile algorithms and methods are not widely available, especially compared to the accessibility of computer vision methods. This is in part due to the wide range of sensing mechanisms and modalities present in current available tactile sensors, making generalizable approaches rare. For example, the DIGIT sensor [ 11 ] produces high-resolution RGB images capturing sensing surface deformations, whilst the Contactile PapillArray [ 10 ] produces 3-DoF force-torque readings from each distinct pea-sized sensing unit laid out in a matrix. Another challenge is sensor obsolescence: the once relatively popular BioTac [ 12 ] and TakkTile [ 13 ] sensors are now obsolete, with no comparable alternatives available. As a result, significant portions of research works involving these sensors are now challenging to continue or have lost relevance. Fig. 1: The UniTac-NV autoencoder architecture, which unifies tactile data from two different sensors into a shared latent space via training on matching tactile interactions collected on different sensors. This unified representation provides two main use cases: 1. The shared decoder can reconstruct tactile information from the latent space, enabling cross-sensor data transfer. 2. Downstream models trained on one sensor’s latent representation can be directly applied to another without retraining. Vision-based tactile sensors represent a special case, with examples such as the aforementioned DIGIT [ 11 ] , GelSight [ 14 ] , and TacTip [ 15 ] . These sensors use cameras to capture micrometer-level deformations of an elastomer sensing surface, ideal for recognizing complex and detailed contact shape features as demonstrated in [ 16 ] . They also share similar internal structures and produce image outputs of comparable formats, which led to applications and methods often being generalizable among them. Numerous recent works have demonstrated that a unified representation across different vision-based tactile sensors can be learned, which enables data collected on one sensor model to be shared across others. For example, [ 17 ] pretrained a self-supervised model on several large-scale datasets of vision-based tactile data, including Digit, GelSight, and GelSight Mini, to create a unified representation transferrable to each sensor. Another approach aligns tactile data from two different sensors via contrastive learning [ 18 ] . In [ 19 ] , task-specific decoders have been used to train individual encoders, even with large unpaired datasets. However, vision-based tactile sensors are not a one-size-fits-all solution. Their limited operating frequencies (often 30 Hz) and high latencies (between 100-200 ms) make them unideal for fast dynamic tasks, whilst their bulky form factors (often 40 mm) make them challenging to fully cover robotic hands [ 20 ] . In such cases, other non-vision-based tactile sensors are more suitable: the Xela uSkin provides various shapes and taxel configurations (ranging from 1 \times 1 to 4 \times 6 sensing units) with a thin profile (around 5 mm) and can cover large portions of robotic hands such as the Allegro hand [ 21 ] ; the Contactile PapillArray is better at measuring dynamic 3-DoF force and at high temporal resolution (1000 Hz maximum), enabling tasks such as real-time slip detection [ 10 ] . However, little work has focused on representation learning and cross-sensor information sharing for these non-vision-based sensors. To address this gap, we introduce the Cross-Sensor Tactile Representation Autoencoder (UniTac-NV), an encoder-decoder network featuring dedicated encoders for each sensor and a shared decoder for reconstructing tactile sensor readings (Fig. 1 ). Our approach encodes contact information into a unified latent space that is agnostic to the sensor type. The shared decoder then translates this latent representation into the raw readings for all tactile sensors involved. The advantages of our architecture are two-fold: Firstly, we demonstrate that the shared latent space can be used directly in application tasks with minimal loss of information. Conceptually, one can view the decoders as translators that convert sensor-specific data (raw sensor readings) into a universal ‘common language’ represented by the latent space. Consequently, downstream applications built on this common representation can seamlessly integrate any tactile sensor whose data can be translated into this universal format. Secondly, our loss function—derived from tactile reconstruction errors—implicitly aligns sensor-specific encodings, as demonstrated later. This streamlined approach simultaneously trains the encoders and the shared decoder, naturally supporting cross-sensor translation—an added benefit compared to methods that focus solely on latent alignment. We validate our approach using two commercial non-vision-based tactile sensors: the Xela uSkin Patch (Model: uSPa 46) and the Contactile PapillArray. Our experiments quantify both cross-sensor translation accuracy and latent space alignment. We further assess the framework by performing a downstream task: contact geometry reconstruction. In addition, we propose a reproducible tactile data collection procedure which ensures consistent contact interactions across different tactile sensors. This allows us, as well as the tactile sensing community, to expand the dataset with new contact interactions as well as sensors. II METHODOLOGY II-A The UniTac-NV Autoencoder Model II-A 1 Multi-Encoder Architecture UniTac-NV (shown in Fig. 1 ) was inspired by traditional autoencoder architectures, which learn compact latent representations by minimizing errors between original and reconstructed input data. In contrast to traditional autoencoders, UniTac-NV features multiple encoders (one for each tactile sensor) and one shared decoder, which reconstructs tactile readings from the latent space for all involved sensors. Each encoder takes the flattened sensor data as input (4 \times 6 \times 3 = 72 readings for uSkin and 3 \times 3 \times