UnPose : Un certainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
Zhaodong Jiang Ashish Sinha Tongtong Cao Yuan Ren Bingbing Liu Binbin Xu · 2025
Read UnPose : Un certainty-Guided Diffusion Priors for Zero-Shot Pose Estimation by Zhaodong Jiang Ashish Sinha Tongtong Cao Yuan Ren Bingbing Liu Binbin Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15972.txt.
Introduction
Zhaodong Jiang and co-authors estimate object pose without a stored model in 2025. Robots often need a new object's full position and orientation, but building its CAD model is costly. Prior-based rebuilding from photos can invent false geometry instead.
Starting from one depth-camera frame, a diffusion model drafts the object in a splat-based 3D form, with per-pixel uncertainty. New views then refine only the parts the model flags as unsure. A pose graph ties all views into one coherent field. Tests show clear gains in pose accuracy and rebuild quality, plus demos on real robot handling tasks.
Catalog record
| CREATOR | Zhaodong Jiang Ashish Sinha Tongtong Cao Yuan Ren Bingbing Liu Binbin Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-15972
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
UnPose : Un certainty-Guided Diffusion Priors for Zero-Shot Pose Estimation Zhaodong Jiang Ashish Sinha Tongtong Cao Yuan Ren Bingbing Liu Binbin Xu Abstract Estimating the 6D pose of novel objects is a fundamental yet challenging problem in robotics, often relying on access to object CAD models. However, acquiring such models can be costly and impractical. Recent approaches aim to bypass this requirement by leveraging strong priors from foundation models to reconstruct objects from single or multi-view images, but typically require additional training or produce hallucinated geometry. To this end, we propose UnPose , a novel framework for zero-shot, model-free 6D object pose estimation and reconstruction that exploits 3D priors and uncertainty estimates from a pre-trained diffusion model. Specifically, starting from a single-view RGB-D frame, UnPose uses a multi-view diffusion model to estimate an initial 3D model using 3D Gaussian Splatting (3DGS) representation, along with pixel-wise epistemic uncertainty estimates. As additional observations become available, we incrementally refine the 3DGS model by fusing new views guided by the diffusion model’s uncertainty, thereby, continuously improving the pose estimation accuracy and 3D reconstruction quality. To ensure global consistency, the diffusion prior-generated views and subsequent observations are further integrated in a pose graph and jointly optimized into a coherent 3DGS field. Extensive experiments demonstrate that UnPose significantly outperforms existing approaches in both 6D pose estimation accuracy and 3D reconstruction quality. We further showcase its practical applicability in real-world robotic manipulation tasks. Video demos can be found at our project page: https://frankzhaodong.github.io/UnPose . Figure 1: We present UnPose for zero-shot model-free pose estimation, that takes (a) RGB-D frames and (b) object segments in the scene, to output (c) 3D priors and uncertainty from a diffusion model to (d) reconstruct and continually refine the geometry of the object represented as 3DGS [ 1 ] for 6D pose estimation. (d) shows the resulting object poses with globally consistent front and back views from the input viewpoint. Keywords: 6D Pose Estimation, Diffusion Model, Object Reconstruction, Uncertainty Estimation 1 Introduction 6D object pose estimation is a fundamental challenge in robotic manipulation, enabling tasks from precision grasping to complex assembly [ 2 , 3 ] . While deep learning (DL) has driven notable progress, prior approaches have predominantly relied on object-specific textured CAD models [ 4 , 5 , 6 , 7 ] or category-level training data [ 8 , 9 , 10 , 11 , 12 ] , severely limiting their application in dynamic, open-world environments. Even recent category-agnostic approaches [ 13 , 14 , 15 ] still depend on ground-truth CAD models during inference, essentially approximating the model-based paradigm rather than transcending it. As robots increasingly operate in open-world environments with novel objects, the need for methods that can estimate poses without prior object models has become increasingly urgent. Model-free pose estimation methods offer a promising alternative by eliminating dependence on pre-existing object models, instead leveraging reference views [ 14 , 16 , 17 , 18 ] , or image-to-3D reconstruction models [ 19 , 20 , 21 , 22 ] . However, despite their “model-free” designation, these methods still implicitly rely on reference object models generated in advance from either diffusion models or multiple reference views of the target objects. Therefore, a key issue arises when 3D representations are generated from limited observations: unobserved regions introduce variable confidence i.e. , epistemic uncertainty during reconstruction. Existing approaches typically fail to quantify this epistemic uncertainty [ 23 ] , treating all generated regions equally regardless of observability. This limitation becomes particularly problematic when integrating new observations, as the system cannot appropriately weigh the reliability of the prior model against newly observed data [ 24 , 25 ] . To this end, we introduce UnPose , a principled framework for zero-shot, model-free 6D object pose estimation that addresses these limitations through uncertainty-aware multi-view integration and pose graph optimization. Starting from a single RGB-D observation, UnPose generates an initial 3D representation using an image-to-3D diffusion model with pixel-level uncertainty estimates that quantify confidence in the diffusion priors and balance them with sensor measurements. This 3D representation is encoded using 3D Gaussian Splatting (3DGS) [ 1 ] , enabling efficient and high-fidelity textured models. As more observations arrive, UnPose incrementally refines the geometry and appearance via pose graph optimization, with uncertainty estimates guiding the fusion process to prioritize reliable measurements while refining uncertain regions. In summary, our contributions are: (1) we present a model-free approach that eliminates dependencies on object-specific CAD models, category-level training, or multi-view requirements with known camera poses; (2) we introduce an uncertainty-guided refinement that adaptively integrates observed data into diffusion priors; (3) We formulate pose estimation as an incremental factor graph optimization problem that incorporates both diffusion priors and observations, and ensures global consistency; and (4) We demonstrate significant improvements in both pose estimation accuracy and reconstruction quality compared to existing SOTA methods. Figure 2: We present an overview of our proposed pipeline. Starting from a single-view RGB-D frame, UnPose uses a multi-view diffusion model to estimate an initial 3D model using 3D Gaussian Splatting (3DGS) representation, along with pixel-wise epistemic uncertainty estimates. As additional observations become available, we incrementally refine the 3DGS model by fusing new views guided by the diffusion model’s uncertainty, thereby, continuously improving the pose estimation accuracy and 3D reconstruction quality. 2 Related Works 2.1 Model-based Object Pose Estimation Instance-level pose estimation techniques [ 6 , 4 , 5 , 7 ] typically rely on textured CAD models for specific objects, where both training and testing are conducted on the same set of instances. In contrast, category-level methods [ 9 , 10 , 8 , 11 , 12 ] aim to generalize to unseen instances within known categories, though they require costly category-specific annotations and are limited to predefined object classes. To address these limitations, category-agnostic approaches [ 13 , 14 , 15 ] aim to estimate the poses of novel objects without relying on predefined categories. However, many of these methods still depend on ground-truth CAD models to render reference views during inference, which restricts their practical use in open-world applications where such models are often unavailable or costly to obtain. More recent works [ 26 , 27 ] attempt to address this gap by retrieving CAD models from large repositories, though retrieval accuracy and high quality datasets continue to pose challenges. 2.2 Model-free Object Pose Estimation To reduce reliance on textured CAD models, recent research has shifted toward model-free object pose estimation methods using either reference images [ 28 , 14 , 20 ] or single/multi-view images or videos [ 29 , 27 ] . However, current SOTA approaches still rely on posed reference views [ 14 ] or reconstructed object models [ 30 ] and, thus, are not strictly model free . Several works attempt to relax this requirement by estimating relative orientation from a single anchor image [ 18 , 20 ] , using feature-matching [ 17 ] or language guidance [ 31 ] . Despite these advances, performance degrades when query views have limited overlap or severe occlusions. To tackle occlusion, recent works [ 21 , 20 , 22 ] leverage image-to-3D diffusion priors to reconstruct object’s geometry from limited views and estimate the 6D pose, alleviating the need for