DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios
Xiangting Meng*, Jiaqi Yang*, Mingshu Chen, Chenxin Yan, Yujiao Shi, Wenchao Ding, Laurent Kneip · 2025
Read DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios by Xiangting Meng*, Jiaqi Yang*, Mingshu Chen, Chenxin Yan, Yujiao Shi, Wenchao Ding, Laurent Kneip (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19625.txt.
Introduction
Xiangting Meng, Jiaqi Yang, and colleagues release a benchmark for tracking moving objects with moving cameras in a 2025 computer vision study. Robots and headsets must estimate the pose of objects that move while the camera also moves. Real data for that double motion is scarce because labeling poses in such scenes is painstaking.
DynOPETs supplies both the data set and the pipeline used to gather and annotate it. The benchmark targets unconstrained settings rather than tidy lab scenes. With shared data and scoring, research groups can compare pose estimation and tracking methods fairly. Better pose tracking feeds augmented reality, robotics, and scene understanding.
Catalog record
| CREATOR | Xiangting Meng*, Jiaqi Yang*, Mingshu Chen, Chenxin Yan, Yujiao Shi, Wenchao Ding, Laurent Kneip |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19625
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DynOPETs: A Versatile Benchmark for Dynamic Object Pose Estimation and Tracking in Moving Camera Scenarios Xiangting Meng*, Jiaqi Yang*, Mingshu Chen, Chenxin Yan, Yujiao Shi, Wenchao Ding, Laurent Kneip Abstract secs/0-abstract I INTRODUCTION Object pose estimation, which involves determining an object’s 6-DoF pose (position and orientation) in 3D space, is a fundamental research challenge in computer vision and robotics. The task is crucial for various applications in embodied intelligence, augmented reality (AR), and mixed reality (MR) systems [ 137 ] , such as robotic manipulation [ 83 ] , dexterous hand control, and human-object interaction [ 84 ] . Despite significant advancements in recent years driven by the emergence of benchmark datasets, existing real-world datasets exhibit notable limitations. Most of the existing datasets predominantly assume static conditions - either fixed camera positions or stationary objects. This assumption contrasts sharply with real-world requirements where simultaneous camera ego-motion and object dynamics must be addressed. The limited availability of dynamic pose estimation datasets stems primarily from annotation challenges inherent to moving-camera-dynamic-object configurations. While static-object scenarios allow for indirect pose annotation via the state-of-the-art camera pose estimation technique, such as SLAM algorithm [ 149 ] or fiducial marker-based localization method [ 86 ] , dynamic settings require direct object motion tracking. In such dynamic settings, existing solutions for pose annotation rely on either labor-intensive manual annotation [ 87 ] or expensive motion capture (Mocap) systems in controlled environments [ 88 ] . Fig. 1 : The full set of dynamic objects in our dataset. Another approach [ 89 ] involves using state-of-the-art object pose estimation methods [ 137 ] to generate high-quality pseudo-labels for annotation. However, many common objects in daily life, such as those with symmetric geometries, transparent or reflective materials, or sparse/repetitive patterns, pose significant challenges for those algorithms. As a result, a significant fraction of the object pose annotations still require manual refinement due to the low quality of the pseudo-labels. To address the issue of scarce real-world datasets in this challenging scenario, we propose a novel dataset that includes diverse object categories and dynamic scenes. The dataset primarily consists of RGB-D video sequences, object CAD models, and high-quality object and camera pose annotations, supporting various tasks in the field of object pose estimation. To tackle the challenges of pose annotation, we introduce a novel automated pose annotation pipeline leveraging the fusion of absolute and relative object pose estimates, significantly improving data labeling efficiency and reducing human effort. Finally, we systematically investigate existing state-of-the-art methods and comprehensively benchmark their performance in dynamic scenarios, demonstrating the value of our work. We believe that our proposed dataset and efficient pose annotation pipeline can fill the gap in dynamic object pose estimation datasets, thereby promoting the development of applications in the fields of embodied intelligence and AR/MR. Our main contributions are summarized as follows: • We propose an efficient pose annotation approach that integrates an absolute pose estimator with a global EKF smoother, a relative pose estimator based on point tracking, and an object pose graph optimization module. This pipeline enables accurate, marker-free dynamic object pose labeling, without extensive manual effort. • We present DynOPETs , a comprehensive object pose estimation dataset comprising RGB-D sequences of 175 distinct object instances undergoing simultaneous camera and object motion, with synchronized 6-DoF pose annotations for both camera and object trajectories. • We conduct a systematic benchmark of various taxonomy-specific pose estimation and object pose tracking methods on our dataset. TABLE I : Comparison between existing 17 datasets with our proposed dataset. The table summarizes key attributes used for comparing datasets: object categories, whether the data is synthetic or real-world, data modalities (RGB, Depth), object details (number of instances, presence of dynamic objects, presence of moving cameras, CAD model availability), whether the videos are marker-free, and dataset size (number of videos, images, and annotations). Dataset Cat. Real Modality Object Details Marker-Free Vid. Img. Anno. RGB Depth Num. CAD Dyna-Obj Mov-Cam CAMERA25 [ 152 ] 6 ✗ ✓ ✓ 1085 ✓ ✗ ✓ ✓ - 300K 4M Omni6D [ 91 ] 166 ✗ ✓ ✓ 4688 ✓ ✗ ✓ ✓ - 0.8M 5.6M SOPE [ 92 ] 149 ✗ ✓ ✓ 4162 ✓ ✗ ✓ ✓ - 475K 5M YCB-Video [ 93 ] - ✓ ✓ ✓ 21 ✓ ✗ ✓ ✓ 92 133K 0.6M T-LESS [ 94 ] - ✓ ✓ ✓ 30 ✓ ✗ ✓ ✗ 20 48K 48K Linemod [ 95 ] - ✓ ✓ ✓ 15 ✓ ✗ ✗ ✗ - 18K 15k StereoOBJ-1M [ 96 ] - ✓ ✗ ✓ 18 ✓ ✗ ✓ ✗ 182 393K 1.5M YCBInEOAT [ 87 ] - ✓ ✓ ✓ 5 ✓ ✓ ✗ ✓ 9 7449 3.7K HOT3D [ 84 ] - ✓ ✓ ✗ 33 ✓ ✓ ✓ ✗ 425 3.7M 6M UOPE-56 (Ours) - ✓ ✓ ✓ 56 ✓ ✓ ✓ ✓ 56 64K 180K REAL275 [ 152 ] 6 ✓ ✓ ✓ 42 ✓ ✗ ✓ ✗ 18 8K 35K Wild6D [ 97 ] 5 ✓ ✓ ✓ 162 ✗ ✗ ✓ ✓ 486 10K 10K PhoCaL [ 98 ] 8 ✓ ✓ ✓ 60 ✓ ✗ ✓ ✗ 24 3.9K 91K HANDAL [ 89 ] 17 ✓ ✓ ✗ 212 ✓ ✓ ✗ ✓ 2.2K 308K 1.2M HouseCat6D [ 153 ] 10 ✓ ✓ ✓ 194 ✓ ✗ ✓ ✓ 41 23.5K 160K ROPE [ 92 ] 149 ✓ ✓ ✓ 581 ✓ ✗ ✓ ✓ 363 332K 1.5M COPE-119 (Ours) 6 ✓ ✓ ✓ 119 ✓ ✓ ✓ ✓ 119 140K 350K II RELATED WORK According to [ 100 ] , We categorize object pose estimation methods into 4 classes: instance-level object pose estimation (IOPE), unseen object pose estimation (UOPE), category-level object pose estimation (COPE), and object pose tracking (OPT). This section discusses representative and state-of-the-art methods within each class, followed by an overview of recently published datasets relevant to these areas. II-A Object Pose Estimation and Tracking Methods II-A 1 Instance-level Object Pose Estimation These methods predict the pose of only those objects present in the training set, often relying on priors such as CAD models or posed reference views during inference [ 157 ] . Object pose estimation can be achieved through template matching [ 102 ] or feature correspondence matching [ 103 ] given these priors. Some approaches directly predict object pose using neural networks [ 157 ] , while others employ voting schemes [ 104 ] . Instance-level methods are the most mature but also the most restrictive due to their inability to generalize to novel objects. Consequently, research has shifted towards the more generalizable UOPE and COPE methods, which are the primary focus of this paper and our proposed dataset. II-A 2 Unseen Object Pose Estimation Similar to IOPE methods, UOPE methods rely on CAD models or posed reference views as priors. However, UOPE methods can generalize to novel objects, enabling pose prediction for unseen objects during inference. Recent works in this area are broadly categorized into two main strategies: feature matching and template matching-based methods. Feature matching methods [ 105 , 142 , 107 , 108 ] extract robust image features and establish correspondences with generic, object-agnostic CAD model features. These approaches enable zero-shot estimation but