Improving Robotic Manipulation: Techniques for Object Pose Estimation, Accommodating Positional Uncertainty, and Disassembly Tasks from Examples
by · 2025
Read Improving Robotic Manipulation: Techniques for Object Pose Estimation, Accommodating Positional Uncertainty, and Disassembly Tasks from Examples by by (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15865.txt.
Introduction
Viral Rasik Galaiya's 2025 thesis explores how touch sensing can help robots handle objects in messy, real-world settings. Cameras are the usual robot sense, but they suffer from blocked views and limited detail. The work asks what the sense of touch adds across three problems in robotic manipulation.
First, the thesis uses touch signals over time to work out an object's pose. Next, it trains a robot with reinforcement learning and touch feedback to grasp objects in fewer tries when camera estimates of position are off. Finally, touch data guides a learned policy that removes an object from a narrow passage, with training sped up by learning from human examples.
Catalog record
| CREATOR | by |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 91 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-15865
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Robotic Manipulation: Techniques for Object Pose Estimation, Accommodating Positional Uncertainty, and Disassembly Tasks from Examples by Abstract Abstract To use robots in more unstructured environments, we have to accommodate for more complexities. Robotic systems need more awareness of the environment to adapt to uncertainty and variability. Although cameras have been predominantly used in robotic tasks, the limitations that come with them, such as occlusion, visibility and breadth of information, have diverted some focus to tactile sensing. In this thesis, we explore the use of tactile sensing to determine the pose of the object using the temporal features. We then use reinforcement learning with tactile collisions to reduce the number of attempts required to grasp an object resulting from positional uncertainty from camera estimates. Finally, we use information provided by these tactile sensors to a reinforcement learning agent to determine the trajectory to take to remove an object from a restricted passage while reducing training time by pertaining from human examples. Acknowledgements I would never have even started my master’s program if it were not for Vinicius’ enthusiasm to share the work he was doing after a chance encounter with him while rock climbing. I’d like to thank my supervisors, Dr. Vinicius Prado de Fonseca and Dr. Xianta Jiang, for their invaluable support, guidance, and patience. I would also like to acknowledge our collaborator, Dr. Thiago Eustaquio Alves de Oliveira, for his ideas and advice on the work I have done, as well as Mohammed Asfour for his input in the second chapter. This would not have been possible without the encouragement of my family, Rasik Galaiya, Ila Galaiya and Kangana Galaiya. Many thanks to Yanitza Trosel for all her support and advice and her mighty sidekick Suki. A shoutout to my friends in Computer Science, Marius Seidl, for the engaging conversations, and Elahe Mohammadreza for the camaraderie as well as all my other lab mates and colleagues. Finally, all the friends I made in Newfoundland who helped make it my new home; Neus Campanya Llovet, Celyn Khoo, John Green, Haris Mubashir, Pranjal Patra, and Ali Ghamartale. Chapter 1 Introduction 1.1 Background Outsourcing human labor to a being or device has been conceptualized since as early as the Greek-Hellenistic age [ 1 ] , while there are even records of a musical automaton in the 1100s [ 2 ] . Although the idea of humanoid robots has been around in various ages and cultures ranging from Roman mythology to ancient India, the widespread robotics application in the 1950s was machine tools combined with manipulators to perform simple, repeatable activities without information about the environment. With time, these manipulators started becoming more sophisticated, incorporating advances in machine learning, sensor technology [ 3 ] , and mechanical and material advancements to come full circle to humanoids. These first implementations were very rudimentary compared to today’s standards because of the lack of information on the environment and were operated in highly structured environments. With the development of sensors, computing, and application breadth, robotic applications keep moving closer to unstructured environments. The ability to accurately infer the environment and adapt to this variable environment plays a significant role in robots’ wider range of uses. Zamalloa et al. [ 3 ] describes the trend as moving from mass manufacturing to mass customization. Since the development of robotics, the breadth and depth have increased tremendously. Despite manipulators being the first at mass adoption [ 3 ] and becoming just a subset of the whole field that now includes drones and quadrupeds, it is still a highly researched area due to the range challenges still far from being solved like gripper design [ 4 ] , control [ 5 ] and sensing [ 6 ] . Humans have an extensive range of sensing capabilities that enable us to perform very complex tasks seemingly efficiently. We rely highly on visual perception systems but also have other very powerful sensory and information input systems. These systems include sight, touch, sound, and smell. These systems can be used very well together but are also used well in isolation. Helbig et al. [ 7 ] suggests that since no information-processing system has the ability to provide sufficient information under all conditions, it is imperative to combine multiple systems. This line of thought has encouraged researchers to explore forms of perception systems other than vision directly, ranging from inertial measurement units [ 8 ] to rangefinders [ 9 ] . In particular, vision has limited capabilities for several grasping and manipulation tasks, specifically for near contact and contact activities. This is because we can’t generally obtain information on weight, force, and friction with an unmodified vision system. Albeit, creative methods of using cameras have been invented to obtain tactile information [ 10 ] . Despite the wide range of sensor data as well as the quantity of data, there is intrinsic uncertainty from the sensor’s accuracy, geometric uncertainty from sensor limitations like placement or types, physical uncertainty from properties such as material composition or weight as well as environmental disturbances from the environment, especially in unstructured environments. Reinforcement learning with tactile sensing allows for adapting to these uncertainties and changing environments [ 11 ] . 1.2 Literature Review For any manipulation task, it is vital to know the pose of the object being manipulated. It is also important to keep track of changes that are made when manipulating the object so the trajectory of the manipulator can be efficiently calculated. Amongst various other fields, object pose estimation has benefited highly from the advent of deep learning techniques. Du et al. [ 12 ] explores work done in 2D planner grasps six-degree grasp planning including constraint grasps, object pose estimation using template matching-based methods, voting-based methods, end-to-end grasp estimation using both RGB-D images. They concluded that there needs to be a close similarity of the 3D object model the network is trained on to have accurate grasps. Moreover, the lack of geometry information from the direction not facing the camera affects the accuracy of the shape completion and suggests the use of multi-source data. Therefore, other data sources have been explored, especially close to or in relation to the contact points, such as using tactile sensing or proprioception [ 13 ] . These methods generally complement vision-based sensing modalities to address blind spots, increase accuracy or both. Various ways of leveraging the information of tactile sensing for pose estimation have been used, including collisions [ 14 , 15 ] , simulation to real life [ 16 , 17 ] and reinforcement learning [ 18 ] . Grasping in unstructured environments must overcome significant uncertainties from things like occlusion and noise and accommodate many object properties like geometry and mass. Under-actuation and compliance are great low-cost rectifiers. This is because fewer motors are required to grasp an object, and compliance increases adaptability, reducing the effects of uncertainty. Although these techniques can result in position error because of forces dislodging the object, they can be reconciled using information obtained from the end-effector, such as tactile sensors [ 19 ] . Various compliant tactile sensors [ 20 , 21 , 22 , 23 , 24 , 25 ] have been developed to improve grasp force control,