PUGS: P erceptual U ncertainty for G rasp S election in Underwater Environments

Onur Bagoren, Marc Micatka, Katherine A. Skinner, Aaron Marburg · 2025

Read PUGS: P erceptual U ncertainty for G rasp S election in Underwater Environments by Onur Bagoren, Marc Micatka, Katherine A. Skinner, Aaron Marburg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09824.txt.

Introduction

Bagoren, Micatka, Skinner, and Marburg tackled robot grasping underwater in this paper from twenty twenty-five. Underwater sensors return patchy, noisy views of objects. A gripper that trusts every reading equally will grab at the wrong spots. Their method, PUGS, measures how unsure the 3D reconstruction is at each point.

Uncertainty from many camera views is carried through the rebuilding process. Grasp choices then favor spots the robot can actually see well. Tests ran on both simulated scenes and real underwater data. Accounting for doubt made grasp picks hold up against noise and missing views. The authors plan to release the code for other teams.

Catalog record

CREATOROnur Bagoren, Marc Micatka, Katherine A. Skinner, Aaron Marburg
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09824

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PUGS: P erceptual U ncertainty for G rasp S election in Underwater Environments Onur Bagoren, Marc Micatka, Katherine A. Skinner, Aaron Marburg Abstract When navigating and interacting in challenging environments where sensory information is imperfect and incomplete, robots must make decisions that account for these shortcomings. We propose a novel method for quantifying and representing such perceptual uncertainty in 3D reconstruction through occupancy uncertainty estimation. We develop a framework to incorporate it into grasp selection for autonomous manipulation in underwater environments. Instead of treating each measurement equally when deciding which location to grasp from, we present a framework that propagates uncertainty inherent in the multi-view reconstruction process into the grasp selection. We evaluate our method with both simulated and the real world data, showing that by accounting for uncertainty, the grasp selection becomes robust against partial and noisy measurements. Code will be made available at https://onurbagoren.github.io/PUGS/ I Introduction Underwater remotely operated vehicles (ROVs) equipped with manipulator systems allow for advancements in ocean cleaning operations [ 1 ] , deep sea specimen sampling [ 2 ] , and human-robot collaboration in underwater environments [ 3 ] . Standard underwater manipulator systems are often controlled through teleoperation [ 4 ] . Despite its success, reliance on teleoperation poses many challenges, primarily in operational cost and pilot training time. These challenges associated with teleoperation have motivated research advances in autonomous underwater manipulation [ 5 , 6 ] , where the operational cost is significantly lower. Yet, many challenges remain in deploying autonomous mobile manipulator systems in environments such as the underwater domain. Providing informative representations of the environment and objects that robots interact with is critical to achieving safe and reliable autonomy for robotic systems. For this, a common approach is using sensors that provide rich environmental information to map the surrounding scene [ 7 , 8 , 9 ] . However, underwater sensing and perception suffer from several challenges, including image degradation [ 10 , 11 ] , sensor noise [ 12 ] , and uncertainty due to a lack of absolute position measurements [ 13 ] , which lead to challenges for robotics tasks that rely on accurate perceptual information. A growing number of works have been successful in showing that incorporating uncertainty into perception systems, whether for detection [ 14 ] , simultaneous localization and mapping [ 12 ] , or 3D reconstruction [ 15 , 16 , 17 ] , plays a crucial role in improving performance. In addition to playing a crucial role in improving the robustness of perception systems, uncertainty plays a key role in embodied robotic tasks, such as planning and manipulation [ 18 ] . In this work, we model perceptual uncertainty in 3D reconstruction tasks to improve grasp selection and the manipulation of objects in underwater environments, using a method we call P erceptual U ncertainty for G rasp S election (PUGS). Fig. 1: Real-world underwater manipulation setup in the test tank at the University of Washington Applied Physics Laboratory (UW-APL). The test system includes a Reach Robotics Bravo 7 electric manipulator and Trisect subsea stereo sensor. The gripper in red shows the grasp pose from an out-of-the-box grasp selection model [ 19 ] . The proposed method successfully leads to a more reliable grasping location, shown in green, by using perceptual uncertainty to weigh more favorable grasp locations. The visualizations are taken from evaluations of a real-world dataset and overlaid to match the photo showing the robot during regular operation. We focus on modeling how uncertainty inherent from multi-view stereo can be leveraged for quantifying uncertainty in 3D reconstruction, specifically in representing occupancy in 3D space. We then show that the uncertainty of the occupied regions can be a useful for improving existing grasp selection methods and guiding toward more reliable and robust grasp selection. We present the following contributions: 1. We propose the construction of a fused occupancy field (FOF) informed by the uncertainty measurements and pose estimates. 2. We develop a novel method to quantify the predictive uncertainty associated with occupancy in 3D space using probabilistic regression methods. 3. We present an uncertainty fusion method to combine information from measurement and predictive uncertainty for modeling occupancy uncertainty. 4. We provide an experimental evaluation in both simulation and real-world underwater environments to validate the proposed methods. II Related Works Fig. 2: Overview of the proposed system. We use a SLAM pipeline to construct dense 3D reconstructions of objects of interest. PUGS quantifies the uncertainty inherent in the observation and pose estimation to construct an occupancy uncertainty representation. We use TSGrasp [ 19 ] as the baseline network to regress on grasp poses and confidences. The final grasp confidence and pose are determined by fusing the PUGS occupancy uncertainty output and the TSGrasp confidences. II-A Underwater Manipulation and Grasp Selection Compared to traditional robotic manipulation, manipulation with underwater vehicle manipulation systems (UVMS) has additional challenges associated with vehicle dynamics, sensing, and environmental disturbances [ 20 , 21 , 19 ] . For improving the perception in underwater environments, Billings et al. present a hybrid perception system for a UVMS with a stereo rig mounted on the vehicle and a fish eye camera mounted on the manipulator, with an accompanying mapping and pose estimation method [ 20 ] . Chang et al. present an information-theoretic approach to the underwater planning and grasping task, where the information associated with the object pose and shape is aimed to be maximized [ 21 ] . Different from [ 20 ] , our proposed system aims to use robust perception for grasping rather than pose estimation and mapping, where an improvement of the perception and grasping capabilities are achieved through an uncertainty-centric approach. TSGrasp, a grasp selection method tested in an underwater setting with a similar hardware setup, builds on previous work [ 22 ] by utilizing sparse spatiotemporal convolution to output grasp poses, gripper widths, and confidences on an input pointcloud [ 19 ] . This method is tested on point clouds accumulated from 1 or 4 frames of images and utilizes geometric information only for the grasp selection. PUGS is a method that builds upon TSGrasp’s grasping capabilities by formulating a fusion between grasp confidence and occupancy uncertainty. For this reason, along with the fact that it was developed for underwater grasp selection, we select TSGrasp as a baseline method to evaluate PUGS and show that our proposed method, accounting for uncertainty, improves grasping capabilities in adversarial situations. II-B Uncertainty Representation for 3D Reconstruction Uncertainty representations in 3D reconstruction aim to capture the uncertainty originating from the measurement model or multi-view stereo to improve the reconstruction quality [ 23 , 24 , 17 , 15 , 25 ] , as opposed to classical methods that treat each measurement equally in the reconstruction process [ 26 ] . Freundlich et al. compute the bias and covariance associated with 3D locations obtained from stereo imaging, showing that the calculated bias correction leads to more accurate representations [ 23 ] . Ulusoy et al. present a series of papers on using probabilistic methods for volumetric rendering [ 24 , 17 ] . The developed framework is compared against classical methods that perform point estimates on occupancy and show that

Read the complete text