SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation
Yufeng Jin, Niklas Funk, Vignesh Prasad, Zechu Li, Mathias Franzius, Jan Peters, Georgia Chalvatzaki · 2025
Read SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation by Yufeng Jin, Niklas Funk, Vignesh Prasad, Zechu Li, Mathias Franzius, Jan Peters, Georgia Chalvatzaki (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01501.txt.
Introduction
Yufeng Jin and six colleagues address a robot's basic uncertainty: from one camera view, where exactly is the object it wants to grab? Occlusions and symmetry often leave several poses that fit the image, yet most neural networks answer with a single confident guess. Their system, SE(3)-PoseFlow, instead estimates a whole distribution of possible six-dimensional poses.
The method uses flow matching on the manifold of rigid-body motions to sample likely poses, so downstream software can reason about doubt instead of ignoring it. It reaches state-of-the-art scores on the Real275, YCB-V, and LM-O benchmarks. The authors also show the samples helping real manipulation tasks, such as choosing a better viewpoint or planning a grasp that accounts for uncertainty.
Catalog record
| CREATOR | Yufeng Jin, Niklas Funk, Vignesh Prasad, Zechu Li, Mathias Franzius, Jan Peters, Georgia Chalvatzaki |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01501
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation Yufeng Jin, Niklas Funk, Vignesh Prasad, Zechu Li, Mathias Franzius, Jan Peters, Georgia Chalvatzaki Abstract Object pose estimation is a fundamental problem in robotics and computer vision, yet it remains challenging due to partial observability , occlusions , and object symmetries , which inevitably lead to pose ambiguity and multiple hypotheses consistent with the same observation. While deterministic deep networks achieve impressive performance under well-constrained conditions, they are often overconfident and fail to capture the multi-modality of the underlying pose distribution. To address these challenges, we propose a novel probabilistic framework that leverages flow matching on the SE(3) manifold for estimating 6D object pose distributions. Unlike existing methods that regress a single deterministic output, our approach models the full pose distribution with a sample-based estimate and enables reasoning about uncertainty in ambiguous cases such as symmetric objects or severe occlusions. We achieve state-of-the-art results on Real275, YCB-V and LM-O, and demonstrate how our sample-based pose estimates can be leveraged in downstream robotic manipulation tasks such as active perception for disambiguating uncertain viewpoints, or guiding grasp synthesis in an uncertainty-aware manner. I Introduction Estimating the 6D pose of objects is a fundamental problem in robotics, as it enables embodied agents to perceive, manipulate, and interact safely with their environment. In practical applications such as robotic grasping, assembly, and human–robot collaboration, it is not sufficient to output a single deterministic pose estimate. Instead, reasoning about uncertainty is critical for ensuring safe and reliable manipulation [ 1 , 2 ] . Probabilistic models that capture the multi-modality of pose distributions provide richer information than deterministic pose estimates, especially in safety-critical manipulation scenarios where downstream decisions rely on calibrated confidence. A central challenge in 6D object pose estimation arises from pose ambiguity . Symmetries in object geometry, partial observability, and occlusions often yield multiple feasible poses that are indistinguishable from sensor observations [ 3 , 4 , 5 , 6 , 7 ] . Deterministic deep learning approaches such as FoundationPose [ 8 ] have recently advanced the state of the art by leveraging large-scale synthetic training and transformer-based architectures, but they remain limited in their ability to represent multi-hypothesis pose distributions. As a result, they can be over-confident in ambiguous cases, which is undesirable for robotic planning and control. Fig. 1: We propose an uncertainty-aware 6D object pose estimation approach based on \mathrm{SE}(3) flow matching. Our probabilistic framework predicts full 6D pose distributions to handle ambiguities, enabling reliable robotic manipulation under challenging real-world conditions (partial observability, occlusions, and symmetries). SO(3) distributions are visualized on a Mollweide projection, where latitude (pitch) and longitude (roll) map the orientation, and color encodes yaw. To address these limitations, recent research has explored probabilistic formulations of object pose estimation . These methods aim to model the full distribution of feasible poses rather than commit to a single prediction. Early approaches leverage directional probability distributions such as the von Mises–Fisher or Bingham distribution to represent rotational uncertainty [ 9 , 10 , 11 ] . While theoretically principled, such parametric models are typically unimodal and require mixtures to capture multi-modal ambiguities, leading to computational inefficiency and numerical instability. More recently, generative models based on diffusion [ 6 , 7 , 3 ] and normalizing flows [ 12 ] have been proposed to directly model complex distributions on \mathrm{SE}(3) through sample-based estimates. These approaches naturally capture multi-modality and uncertainty, but often rely on intermediate representations or remain constrained to synthetic benchmarks. This motivates the development of new methods that combine the scalability of modern architectures with principled probabilistic modeling on the \mathrm{SE}(3) manifold for safe and robust robotics. Our main contributions are summarized as follows: • We propose a probabilistic framework based on flow matching on the \mathrm{SE}(3) manifold for 6D object pose estimation. Our method provides a sample-based estimate of the pose distribution that naturally captures uncertainty in ambiguous cases such as object symmetries or severe occlusions. • We introduce an adapted DiT module with masked cross-attention into the \mathrm{SE}(3) flow model, which improves robustness under occlusions and cluttered real-world scenes, thereby achieving competitive state-of-the-art performance across multiple benchmarks. • We demonstrate how the learned \mathrm{SE}(3) distribution can be leveraged for downstream robotic tasks, such as guiding active perception to resolve viewpoint ambiguity, and enabling reliable and effective single-view grasp generation under partial observability. II Related Works II-A Probabilistic Object Pose Estimation Recent work has sought to overcome the limits of deterministic regressors by explicitly modeling uncertainty in object pose estimation. One approach leverages directional probability distributions for rotational uncertainty: the von Mises–Fisher distribution for Euler angles [ 9 ] , and the Bingham distribution for unit quaternions [ 10 , 11 , 13 , 14 , 15 ] . These models handle symmetries well but have drawbacks: (i) computing normalization constants on non-Euclidean manifolds is costly, (ii) they are unimodal and require mixtures for multi-modality, risking mode collapse, and (iii) parameterizations can be unstable and scale poorly. Beyond closed-form distributions, generative probabilistic models better capture complex, multi-modal pose distributions. DiffusionNOCS [ 6 ] uses image-to-image diffusion to predict NOCS maps aligned with depth, naturally handling symmetry but incurring inference overhead. GenPose [ 7 ] applies score-based diffusion on point clouds, sampling multiple hypotheses but perturbing \mathrm{SO}(3) with Gaussian noise and requiring an auxiliary energy network for likelihood estimation. Möller et al. [ 16 ] adopt a particle-based diffusion formulation for point clouds, which discards texture cues important for fine-grained alignment. More theoretically, Hsiao et al. [ 3 ] and Liu et al. [ 12 ] study diffusion and normalizing flows directly on \mathrm{SO}(3) , showing promise for synthetic benchmarks but not extending to real-world robotics. In summary, probabilistic methods capture pose ambiguity and uncertainty better than deterministic ones but often trade efficiency or generality for expressiveness, motivating scalable methods that connect theoretical advances with robotic deployment. II-B Flows on Manifolds Flow matching [ 17 ] has emerged as an alternative to diffusion for generative learning. It trains continuous normalizing flows (CNFs) by regressing vector fields along probability paths, yielding simulation-free training, closed-form objectives, and faster inference. Chen and Lipman [ 18 ] extended this to Riemannian manifolds via Riemannian Flow Matching , which generalizes conditional flow matching using geodesic or spectral premetrics. Several works apply manifold-aware flow matching across domains. In robotics, Braun et al. [ 19 ] introduced Riemannian Flow Matching Policies for efficient motion generation, while Funk et al. [ 20 ] and Zhang and Gienger [ 21 ] demonstrated SE(3)-equivariant flows for action and affordance learning. Beyond robotics, Miller et al. [ 22 ] used it for crystalline material discovery, and SE(3)-flow matching has been applied to protein backbone generation [ 23 , 24 ] . These studies show the versatility of manifold-aware flows in robotics, materials science, and biology. However, applications on \mathrm{SO}(3) or \mathrm{SE}(3) remain largely confined to synthetic or simulation settings [ 25 , 12 ] . To our knowledge, our work is the first to employ flow matching on \mathrm{SE}(3) for real-world 6D object pose estimation , connecting manifold generative modeling with practical robotic deployment. III Method Given an RGB-D input, our goal is to provide a sample-based estimate of the 6D