RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects
Jaeguk Kim, Jaewoo Park, Keuntek Lee, Nam Ik Cho · 2025
Read RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects by Jaeguk Kim, Jaewoo Park, Keuntek Lee, Nam Ik Cho (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-10841.txt.
Introduction
Jaeguk Kim, Jaewoo Park, Keuntek Lee, and Nam Ik Cho present RefPose, a method that finds the exact pose of objects a system has never seen before. It works from a single photo. Robots need this six-degree-of-freedom pose to grasp things, and unseen objects deny the system any object-specific training.
RefPose renders template views of the object to make a reference image and a first pose guess. It then matches geometric features between the photo and the references. The pose is refined by repeated render-and-compare steps. A new attention mechanism, guided by a correlation volume, sharpens the matching. Reference-guided refinement beats guessing from the photo alone.
Catalog record
| CREATOR | Jaeguk Kim, Jaewoo Park, Keuntek Lee, Nam Ik Cho |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-10841
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects Jaeguk Kim, Jaewoo Park, Keuntek Lee, Nam Ik Cho Abstract Estimating the 6D pose of unseen objects from monocular RGB images remains a challenging problem, especially due to the lack of prior object-specific knowledge. To tackle this issue, we propose RefPose, an innovative approach to object pose estimation that leverages a reference image and geometric correspondence as guidance. RefPose first predicts an initial pose by using object templates to render the reference image and establish the geometric correspondence needed for the refinement stage. During the refinement stage, RefPose estimates the geometric correspondence of the query based on the generated references and iteratively refines the pose through a render-and-compare approach. To enhance this estimation, we introduce a correlation volume-guided attention mechanism that effectively captures correlations between the query and reference images. Unlike traditional methods that depend on pre-defined object models, RefPose dynamically adapts to new object shapes by leveraging a reference image and geometric correspondence. This results in robust performance across previously unseen objects. Extensive evaluation on the BOP benchmark datasets shows that RefPose achieves state-of-the-art results while maintaining a competitive runtime. 1 Introduction 6D pose estimation is a key aspect of computer vision and robotics, focusing on accurately predicting an object’s position (3D translation) and orientation (3D rotation) in a given scene. This task is essential for a range of applications, such as autonomous driving [ 5 , 23 ] , augmented reality (AR) [ 25 , 38 ] , and robotic manipulation [ 2 , 36 ] . Despite significant research efforts in this field, estimating the 6D pose of previously unseen objects remains a considerable challenge. This challenge largely stems from the lack of prior knowledge and the limited generalization capabilities of existing models when faced with new objects [ 12 ] . In instance-level object pose estimation, various methods rely on geometric correspondence as a crucial element for achieving accurate pose estimation [ 35 , 3 , 7 , 19 , 42 , 47 , 11 , 45 , 34 ] . Geometric correspondence refers to identifying the 3D model points that correspond to each pixel in a 2D image, which provides essential information for determining an object’s pose. Typically, this correspondence is estimated using deep learning networks and is subsequently used to infer the object’s pose through methods such as PnP/RANSAC [ 16 ] or neural network regression. However, these techniques face challenges in accurately estimating 2D-3D geometric correspondences for unseen objects, mainly due to their reliance on pre-defined object models and the limited generalization capabilities of the correspondence estimation network. Figure 1 : Comparison between conventional methods and proposed method (RefPose). In contrast to conventional methods, RefPose leverages a reference image and geometric correspondence generated from the estimated pose in the coarse pose estimation stage to guide the query’s geometric correspondence and pose estimation. To address these limitations, we propose RefPose, a novel approach to object pose estimation for unseen objects. As illustrated in Fig. 1 , RefPose predicts the geometric correspondence of the query object by leveraging a reference image and geometric correspondence as guidance. This guidance provides crucial geometric information about the target object, allowing the network to avoid reliance on shape priors learned from a fixed set of objects during training. FoundPose [ 32 ] also estimates correspondences using pre-rendered templates through patch-wise matching, similar to our approach. However, these pre-rendered templates often lack proper alignment with the query image, leading to inaccurate matches and insufficient geometric information. To overcome this, we perform coarse pose estimation by processing the pre-rendered templates to obtain an initial pose, which is then used to render a reference image closely aligned with the query image, providing more reliable information. Additionally, rather than relying solely on direct matching, we carefully design a network that enhances correspondence estimation by integrating information from both the reference image and geometric correspondence guidance. Specifically, in the coarse pose estimation stage, we select multiple templates from a set of pre-rendered templates for the target object. This selection is based on the accuracy of optical flow predictions made by the optical flow network [ 43 ] , which will later help us estimate the geometric correspondence for the query object. We then use these selected templates in a warping-based approach that employs medoid voting to enhance robustness against outliers, yielding a reliable coarse geometric correspondence for the query object. Subsequently, we obtain an initial pose using PnP/RANSAC and render references accordingly. With synthesized reference guidance, we estimate a more reliable and precise geometric correspondence for the query during the refinement stage. To improve this estimation, we introduce a novel attention mechanism that leverages a correlation volume from the optical flow network, effectively integrating reference information. The estimated geometric correspondence for the query then serves as a fixed basis for further refining the pose. We iteratively update the initial pose by estimating the relative pose in comparison to the reference geometric correspondence. At each iteration, we re-render the reference geometric correspondence using the updated pose and compare it to the fixed geometric correspondence of the query. This render-and-compare process is repeated until we achieve an accurate final pose. We assess our method on seven key datasets from the BOP benchmark [ 12 ] . Our findings indicate that RefPose achieves superior accuracy in both coarse pose estimation and final pose accuracy compared to state-of-the-art methods. By optimizing runtime in the pose refinement stage, RefPose not only delivers the highest accuracy across all methods but also maintains competitive speed. Our contributions are as follows: • We propose RefPose, a method that leverages a reference image and geometric correspondence to guide the estimation of the query’s geometric correspondence and pose, eliminating the need for shape priors learned from predefined object sets. • We present a classifier based on optical flow for improved template selection. Additionally, we introduce a warping-based geometry estimation method that utilizes medoid voting to enhance robustness against outliers, leading to more accurate coarse pose estimates. • We propose a correlation volume-guided attention mechanism, improving the model’s ability to focus on relevant regions in a reference image corresponding to the query image. • We achieve state-of-the-art results on the BOP benchmark datasets while maintaining competitive runtime. Figure 2 : Overview of the RefPose pipeline. Given an input RGB image, the target object is first detected, cropped, and resized to create the query image, I_{q} . In Stage 1: Coarse Pose Estimation , a set of templates, \mathcal{S} , is selected from the pre-rendered template set, \mathcal{T} , to estimate an initial pose, P_{0} , for the query object. In Stage 2: Pose Refinement , the query’s geometric correspondence, G_{q}^{pos} , is estimated using the rendered reference image, I_{r} , and geometric correspondence, G_{r}^{pos} . The initial pose, P_{0} , is iteratively refined by estimating the relative pose, \Delta P , between the query and reference. At each iteration, G_{r}^{pos} is re-rendered to align with the updated pose, leading to an accurate final pose estimate. 2 Related work Geometric correspondence-based pose estimation. In instance-level object pose estimation, where training and testing are performed within a fixed set of objects, a common strategy