Enhancing Transparent Object Pose Estimation: A Fusion of GDR-Net and Edge Detection

Tessa Pulli, Peter Hönig, Stefan Thalhammer, Matthias Hirschmanner, Markus Vincze · 2025

Read Enhancing Transparent Object Pose Estimation: A Fusion of GDR-Net and Edge Detection by Tessa Pulli, Peter Hönig, Stefan Thalhammer, Matthias Hirschmanner, Markus Vincze (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12027.txt.

Introduction

Tessa Pulli and four co-authors test whether edge detection helps robots estimate the pose of transparent objects in a 2025 paper. Clear objects are hard for robot vision because lighting, background, and reflections confuse standard methods. Their edges, though, stand out with strong contrast, which makes them stable features to build on.

The team added edge detection as a pre-processing step before a leading pose estimation pipeline and an object detector. They tried three edge methods, including one that uses color and one based on a neural network, on a rendered dataset of transparent objects. Edge pre-processing improved results for certain objects, which suggests the extra step is worth adding when reflections defeat ordinary features.

Catalog record

CREATORTessa Pulli, Peter Hönig, Stefan Thalhammer, Matthias Hirschmanner, Markus Vincze
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12027

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enhancing Transparent Object Pose Estimation: A Fusion of GDR-Net and Edge Detection Tessa Pulli, Peter Hönig, Stefan Thalhammer, Matthias Hirschmanner, Markus Vincze Abstract Object pose estimation of transparent objects remains a challenging task in the field of robot vision due to the immense influence of lighting, background, and reflections. However, the edges of clear objects have the highest contrast, which leads to stable and prominent features. We propose a novel approach by incorporating edge detection in a pre-processing step for the tasks of object detection and object pose estimation. We conducted experiments to investigate the effect of edge detectors on transparent objects. We examine the performance of the state-of-the-art 6D object pose estimation pipeline GDR-Net and the object detector YOLOX when applying different edge detectors as pre-processing steps (i.e., Canny edge detection with and without color information, and holistically-nested edges (HED)). We evaluate the physically-based rendered dataset Trans6D-32 K of transparent objects with parameters proposed by the BOP Challenge. Our results indicate that applying edge detection as a pre-processing enhances performance for certain objects. 1 Introduction 6D object pose estimation aims to determine the position and orientation of objects within a 3D space. It has been widely used in real-world robotics applications, such as object grasping [ 1 , 2 ] and manipulation [ 3 ] . While transparent items are among the most prevalent objects in daily life, their 6D pose estimation remains a challenging task. Their visual appearance is highly determined by the lighting conditions, backgrounds, and its reflective properties. Background patterns are reproduced by the object’s surface which complicates the extraction of recognizable features. Even if RGB-D methods can usually achieve higher performances [ 4 , 5 ] , the depth information of transparent objects is noisy and incomplete. Because depth information does not incorporate valuable information for transparent objects, we rely on RGB images only. Still, the edges of clear objects have the highest contrast presenting stable and prominent features bearing potential for feature extraction [ 6 ] . Edge detection filters emphasize the outlines of objects in RGB images, enabling the network to obtain more precise information about the transparent object’s boundaries. Several works [ 7 , 8 ] demonstrate the capabilities of introducing edge detection to CNNs. As their contribution suggests the effectiveness of this approach, we introduce two types of edge detectors to a state-of-the-art object pose estimation method and investigate the effects on 6D object pose estimation of transparent objects. Figure 1 illustrates the methodology of our experiments. The first set of experiments is conducted on the dataset Trans6D-32 K [ 6 ] comprising RGB images, while the second and third set of experiments incorporates an additional pre-processing step performing edge detection with Canny edge detection [ 9 ] and holistically-nested edge detection [ 10 ] . In a fourth set of experiments, we combine RGB information with canny edges [ 9 ] . Our work focuses on detecting objects and recovering the corresponding 6D pose with challenging features. We investigate the effect of edge representations by attempting to achieve a more precise object representation by depicting only the outlines of the corresponding objects. In summary, the paper has the following key contributions: • We propose a novel approach addressing the challenge of transparent 6D object pose estimation from RGB images. We introduce edge filters as a pre-processing step to highlight the object boundaries. • We design a set of experiments to demonstrate that edge detection can enhance the performance of an object pose estimation pipeline. The experiments prove that edge detection has the potential to improve 6D object pose estimates of transparent objects. The rest of this work is organized as follows: Section II introduces the related work of 6D object pose estimation on RGB images, 6D object pose estimation on transparent objects, and edge representation; Section III describes our method of the experiments, Section IV presents the experimental setup, evaluation, and results; and Section V concludes the work. Figure 1: Experimental setup. We conduct four sets of experiments: In the first experiment, the Trans6D-32K dataset without any augmentation (vanilla) is trained and tested. The second experiment investigates the performance when Canny is applied before training. The third experiment involves HED as an pre-processing step. Finally, we conducted a set of experiments incorporating color information and canny edges. 2 Related Work In this section, we discuss the related works by revisiting 6D object pose estimation on RGB images, for transparent object, and representation of edges. 2.1 6D Object Pose Estimation on RGB Images When considering the input data, 6D object pose estimation can be divided into RGB-D-based and RGB-based methods. RGB-D approaches consider 2D RGB data as well as depth information to achieve a pose estimate. While these methods achieve high performances on textured data [ 11 , 12 , 13 ] , errors occur when being applied to transparent objects. State-of-the-art sensor systems fail when it comes to obtaining depth data of transparent objects properly. For this reason, we concentrate on methods using only RGB information. Most current approaches rely on multi-stage networks instead of regressing the pose directly. To extract intermediate features approaches such as segmentation masks [ 14 ] , key points [ 15 ] , corner points of bounding boxes [ 16 ] , or feature maps [ 17 ] are used. This results in 2D-3D correspondences meaning that points in the 2D image are matched to their position in three-dimensional space [ 18 ] . Then, the pose is estimated based on the retrieved features by applying an additional Perspective-n-Point (PnP) algorithm [ 19 , 20 ] . The PnP problem describes the problem of finding the relative pose between an object and a camera from a set of n pairings between 3D points of the object and their corresponding 2D projections on the focal plane [ 21 ] . E.g. the Geometry Guided Direct Regression Network (GDR-Net) [ 17 ] combines direct and geometry-based indirect approaches, divides objects into fragments and outputs the pose estimate directly. The approach decouples 2D object detection and 6D object pose estimation. The method identifies all objects by means of a detector and identifies a Region of Interest (RoI) for each detection and predicts a geometric feature map of the RoI. The Patch-PnP algorithm is then employed to directly regress the 6D object pose. Even if the methods above achieve considerable results on textured objects, it is not explictily designed for transparent objects and their unique properties. 2.2 6D Pose Estimation of Transparent Objects Transparent objects cause reflections and refractions of light leading to a non-uniform appearance of the surface. Correspondences between different parts of the object are challenging to establish when the surface of an object does not have the same texture. Due to these unique visual properties, their depth information can not be captured by state-of-the-art sensors. The lack of distinct textures and features in transparent objects requires other strategies to extract the pose information. [ 22 , 23 , 24 ] propose networks explicitly designed for estimating poses of transparent objects. Zhou et al. [ 25 ] estimate the probability of being transparent for each pixel with a light-field camera. Sajan et al. [ 26 ] employ

Read the complete text