Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera
Peiyuan Ni, Chee Meng Chew, Marcelo H. Ang Jr, Gregory S. Chirikjian · 2025
Read Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera by Peiyuan Ni, Chee Meng Chew, Marcelo H. Ang Jr, Gregory S. Chirikjian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20207.txt.
Introduction
Peiyuan Ni and co-authors wrote this robotics paper, published in two thousand twenty-five, on teaching warehouse robots to pick metal parts out of bins. Cheap depth cameras struggle with shiny metal: depth data comes back patchy, and poses are often wrong.
Their system works in two stages. First, a reasoning step tests pose guesses against depth, collision, and edge rules to clean the raw data. A symmetry-aware filter then trims the pose list further, since many parts look the same when flipped. Second, the camera learns a scoring metric from the cleaned data, so it can train itself on new parts with no hand labels. Tests on a public data set and a new one from the team beat prior methods. The aim is bin-picking that needs far less human setup.
Catalog record
| CREATOR | Peiyuan Ni, Chee Meng Chew, Marcelo H. Ang Jr, Gregory S. Chirikjian |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-20207
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning and Learning a Perceptual Metric for Self-Training of Reflective Objects in Bin-Picking with a Low-cost Camera Peiyuan Ni, Chee Meng Chew, Marcelo H. Ang Jr, Gregory S. Chirikjian Abstract Bin-picking of metal objects based on low-cost RGBD cameras may suffer errors due to sparse depth information and reflective part texture, leading to a need for manual labeling. To reduce the need for human intervention, we propose a framework consisting of a metric learning stage and a self-training stage. Specifically, to automate the handling of the data captured by a low-cost camera (LC), a multi-object pose reasoning (MoPR) algorithm is proposed which optimizes the data with depth, collision and boundary constraints. To improve the accuracy of the retrieved pseudo labels, we generalize the traditional Bayesian Gaussian Mixture Model (BGMM) to the configuration manifolds and propose Symmetry-aware Lie-group based BGMM (SaL-BGMM). Moreover, to solve the ambiguity issue of perceptual metric learning for multiple objects, we propose a weighted ranking information noise contrastive estimation (WR-InfoNCE) loss. Experiments show that our algorithm outperforms related popular works both on ROBI and our proposed Self-ROBI dataset. I Introduction For bin-picking of reflective objects, traditional consumer-grade cameras can be an economical and easily accessible choice compared to high-performance cameras. However, this approach may suffer from several challenges when using traditional object pose estimation algorithms: 1) sparse depth information; 2) variable and reflective object texture. Using real data can improve these while manual labeling is quite time-consuming. Therefore, it is meaningful to conduct self-training for reflective object pose estimation based on a low-cost camera (LC). Although several works [ 1 ] [ 2 ] on self-training for reflective object pose estimation have been proposed, there are still two critical unsolved issues: 1) They are all based on analyzing individual objects without considering the relationships among all the objects holistically. 2) They require repetitive training iterations, object reconstruction, or isolating individual objects during self-training stage, which is not easy to use and may disrupt the target scene. Fig. 1: Overall framework. The blue and yellow parts respectively represent the processes of the metric learning and self-training stages. The green part denotes both of them. The green and blue poses respectively denote ground truths and the estimated pseudo poses. The red point clouds denote reconstructed data or data captured by the LC. To address these challenges, we propose a metric learning stage prior to self-training. During this stage, our algorithm will learn a holistic pose metric using object models reconstructed from data captured by the LC. The training for the metric uses models reconstructed from data obtained by a high-performance camera (HC) or QR codes as the ground truth. Then self-training can be conducted directly on real-world data or even unseen objects with only a LC. Since the configuration manifold for each object is not Euclidean space and there are ambiguities in the perceptual metric for multiple objects, we propose SaL-BGMM and WR-InfoNCE to address these issues. We summarize our contributions as follows: 1) We propose a two-stage framework to reduce the need for labeling reflective objects in the self-training stage based on an LC. 2) We present a MoPR framework based on SaL-BGMM which utilizes occlusion boundaries as cues to filter candidates. 3) We present a perceptual metric specifically designed for multiple cluttered metal objects based on a WR-InfoNCE. 4) We construct a dataset comprising household and industrial objects to validate the performance of self-training based on a Franka robot 1 1 1 https://github.com/ChirikjianLab/Self-ROBI . II RELATED WORK II-A Self-training for reflective object pose estimation Recent popular self-supervised object pose estimation often relies on texture learning [ 3 ] , multi-view pseudo flow [ 4 ] or domain adaption from simulation to real scene [ 5 ] . However, they are not suitable for reflective objects as the textures are not only changeable with the viewing angles but also suffer from low signal-to-noise ratio and image saturation [ 6 ] , which makes these popular methods difficult to generalize to such scenarios. For self-training of reflective objects, X. Li et al. [ 7 ] only use synthetic data to train but it still suffers from this problem. K. Chen et al. [ 1 ] propose an iterative self-training framework to facilitate cost-effective robotic grasping but with a general perceptual metric [ 8 ] . On the other hand, Y.-C. Lin et al. propose a Neural Radiance Field (NeRF) representation [ 2 ] of a scene to train object descriptors. T. Tan et al. [ 9 ] propose an occlusion-aware neural domain adaptation method via the CAD-like radiance field. These NeRF-based methods show good performance but they work best for isolated objects. Therefore, they may not be suitable for bin-picking because they regard occlusion as noise and the texture of single object differs from the texture in cluttered environments. Additionally, recent works [ 10 ] [ 11 ] [ 12 ] on zero-shot learning also require no manual labeling for unseen objects and can make predictions directly, but there are two issues: 1) Some works [ 11 ] [ 12 ] rely on rendered data, which often fail to capture real-world reflective textures. 2) Paper [ 10 ] relies on a general metric, which is not designed for reflective objects and LC. II-B Contrastive learning for object pose estimation Contrastive learning has widely been used on object pose estimation with discrete form [ 13 ] . C. Zhao et al. [ 14 ] apply a continuous contrastive loss with geodesic distance as weights but it can only be applied to the rotation prediction task. B. Wen et al. [ 12 ] propose pose-conditioned triplet loss for novel object tracking. All of these works mainly focus on single objects and do not compare any of two non-ground truth poses contrastively, which hinders the discrimination between different non-ground truth poses. On the other hand, V. N. Nguyen et al. [ 11 ] propose a local contrastive learning algorithm to predict 2D-2D correspondences. Rather than predicting correspondences, we consider predicting a metric value for multiple objects by improving work [ 15 ] . II-C Gaussian Mixture Models based on Lie theory Gaussian Mixture Models (GMMs) can intuitively provide a good fit for nonlinear space but cannot be directly used in a configuration manifold as its metric differs from the metric in Euclidean space. Zeestraten et al. [ 16 ] propose an extension for GMMs with inference that can work on Riemannian manifolds. S. Calinon et al. [ 17 ] show when GMMs are applied directly to data in non-Euclidean manifolds, the clustering performance tends to be poor. And they present a method to vectorize the manifold around each cluster center. W. Liu et al. [ 18 ] propose to use the derivatives of SE(3) to directly optimize GMMs in SE(3) without using Lie-group-based gaussian distributions. Existing related algorithms have not addressed the clustering or fitting of object poses, which is a common application in pose estimation, so we introduce [ 17 ] into BGMM with symmetry-awareness to enhance the fitting capabilities of the Cross Entropy Method [ 25 ] . III Method III-A Overall Frameworks Shown in Fig. 1 , the whole framework is divided into two parts: metric learning stage and