Targeted Hard Sample Synthesis Based on Estimated Pose and Occlusion Error for Improved Object Pose Estimation
Alan Li, Angela P. Schoellig · 2024
Read Targeted Hard Sample Synthesis Based on Estimated Pose and Occlusion Error for Improved Object Pose Estimation by Alan Li, Angela P. Schoellig (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04279.txt.
Introduction
Alan Li and Angela P. Schoellig improve robot estimates of object pose by training on hard cases, in a 2024 paper. Knowing an object's exact position and angle is vital for robot grasping. Bin picking is tough because items lack texture and hide behind each other. Even strong models fail on certain poses and levels of occlusion.
The method first maps where the model errs across pose and occlusion space. It then uses a simulator to make realistic training images aimed at those weak spots. The approach works with any pose model. Extra training on targeted hard samples lifts accuracy where it counts most. Warehouses and factories stand to gain from fewer failed picks.
Catalog record
| CREATOR | Alan Li, Angela P. Schoellig |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04279
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Targeted Hard Sample Synthesis Based on Estimated Pose and Occlusion Error for Improved Object Pose Estimation Alan Li, Angela P. Schoellig Abstract 6D Object pose estimation is a fundamental component in robotics enabling efficient interaction with the environment. It is particularly challenging in bin-picking applications, where objects may be textureless and in difficult poses, and occlusion between objects of the same type may cause confusion even in well-trained models. We propose a novel method of hard example synthesis that is model-agnostic, using existing simulators and the modeling of pose error in both the camera-to-object viewsphere and occlusion space. Through evaluation of the model performance with respect to the distribution of object poses and occlusions, we discover regions of high error and generate realistic training samples to specifically target these regions. With our training approach, we demonstrate an improvement in correct detection rate of up to 20% across several ROBI-dataset objects using state-of-the-art pose estimation models. I Introduction Object pose estimation, the task of determining the 3D position and orientation of an object within a scene, is fundamental to many computer vision applications. Its significance extends to autonomous navigation, object manipulation, and scene understanding. Over the years, various methods and algorithms have been developed to tackle this problem, including model-based, data-driven, and learning-based approaches. Despite the progress, several challenges still impede the accurate and robust estimation of object poses, particularly in bin-picking, where repeated instances of the same part are populated in an unorganized manner within a bin, with various poses and various levels of occlusion. In recent years, convolutional neural networks (CNNs) have demonstrated great success in the problem of object pose estimation, particularly on datasets such as LINEMOD and T-LESS [ 1 ] , where objects are typically rich in texture and features, resulting in near-optimal color and depth images. However, single-view object pose estimation performs poorly in harder cases when data is missing, sparse, or noisy due to surfaces being glossy or transparent, or due to occlusion and ambiguity from other objects [ 2 ] . In this paper, we explore the use of hard case mining, a technique rooted in machine learning and computer vision to mitigate these challenges and improve the reliability of object pose estimation systems. Fig. 1: The synthesis of hard samples using a set of pre-generated random bins. Poses are selected based on the error view-sphere, and realistic occlusions are added based on the occlusion model to generate the final training samples. After each training epoch, the model is evaluated again to update the error view-sphere and the occlusion model for the next round of hard sample synthesis. Fig. 2: Distribution of keypoint error across the view-sphere of Eye-bolt part. Each point on the sphere represents a single training sample where the camera is at a distance of approx. 500mm, and its viewpoint axis intersects with said point and the center of the object axis (left). The error distribution estimated from the training set error is shown on the right. Note that only half of the view-sphere is used due to symmetry of part. Pose estimation of surrounding objects is a crucial yet difficult task for robotic systems that need to interact robustly with their environments. As the complexity of the environment and the potential interactions between objects grow, the required data to cover the distribution increases exponentially. Preparing comprehensive training data for 6D pose estimation is a formidable challenge. Traditional data generation methods often limit sampling of training data to only the object of interest. For instance, in SSD-6D [ 3 ] , a roughly uniform sphere of views is generated around the object and used to render the object against randomized backgrounds. This approach, although widely used, fails to model the potential interactions between the object and other foreground and background objects. Another approach leverages physics simulators to generate realistic scenes of objects with natural household environments with realistic occlusions and interactions between objects and their backgrounds [ 4 ] . This was shown to produce higher overall performance, with the drawback that rarer poses and configurations are also underrepresented in the training data. These cases are typically harder cases which require more training samples rather than fewer to learn. Current approaches in pose estimation typically use a fixed training procedure and fixed training data, and are able to obtain decent overall performance, but are lacking in terms of reliability; certain arrangements of poses or occlusions may cause certain failure in these models [ 4 ] . We propose a novel approach in the context of object pose estimation which explores the input space of object poses and occlusions to sample additional novel training samples which specifically target these underrepresented and more difficult areas in the training data. The goal of this work is to demonstrate improvements to the reliability and overall accuracy of object pose estimation models compared to the state of the art, through three major technical innovations: 1. We develop a novel occlusion model from the training set which models the expected detection error given the parts of the object which are occluded, enabling the sampling of new occlusions based on error and likelihood. This is used to augment generated samples into hard cases to further train for specific types of occlusions. 2. We develop a method for synthesizing realistic new samples within a bin environment, given a desired object pose and occlusion areas, providing a way to generate novel hard cases in desired areas of the input pose space. 3. We demonstrate the effectiveness of our method using a continuous learning process where the pose error distribution is updated at each epoch and new samples are generated based on the updated error distribution, outperforming approaches with static training sets–converging up to 30% faster and improving overall correct detection rate by up to 20%. Fig. 3: An overview of the occlusion model generation, the goal of which is to estimate the expected error when a given point on the surface of the object is occluded. First a set of 5000 random points are sampled across the surface of the object, then pose estimator performance is evaluated for each training sample, the depth map is then used to determine the visibility of each of the sampled points, and the pose error is applied to points which should be visible based on the object pose but are occluded by other objects (blue). Fig. 4: Visualization of the estimated occlusion model for three parts, where redder areas represent higher expected error when occluded. Note that on the leftmost part, higher error is expected when the area around the slit is occluded, as the part is near-symmetric, and the visibility of the slit reduces the ambiguity of the pose. II Related Works Object pose estimation from images is a core problem in computer vision, with numerous applications in robotic manipulation, augmented reality, and autonomous driving. Methods such as hard negative mining, and hard sample exploration and synthesis are now only just being applied, finding growing success in the area of object pose estimation. II-A Monocular 6D Object Pose Estimation Recent advances have leveraged the power of deep learning to surmount the shortcomings of traditional methods, which estimate pose based on local feature