Object Pose and Shape Estimation for Grasping: Does it Work?

Pavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna, Rajat Talak · 2026

Read Object Pose and Shape Estimation for Grasping: Does it Work? by Pavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna, Rajat Talak (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26944.txt.

Introduction

In their 2026 study, Karke and five colleagues ask whether object pose and shape estimation is mature enough to beat end-to-end grasp synthesis for robot grippers. Testing parallel-jaw grasps from single-view RGB-D images, they compare one end-to-end method against three modular methods that first estimate pose and shape, then sample antipodal grasps. The modular methods win in every test, even for small objects where the end-to-end method fails, though clutter degrades them. Pose and shape accuracy, they conclude, is the deciding factor.

Catalog record

CREATORPavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna, Rajat Talak
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35313
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26944

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Object Pose and Shape Estimation for Grasping: Does it Work? Pavan Karke, Kushal Shah, Gaurav Singh, Md Faizal Karim, K Madhava Krishna, Rajat Talak Abstract The problem of object pose and shape estimation has seen key advancements lately. Encoder-decoder ( e.g., SAM3D, LRM, CRISP) and diffusion-based models ( e.g., InstantMesh, Zero123, SceneComplete) have shown category-agnostic shape encoding capacity and open-set generalizability. In this work, we ask the question: Are the object pose and shape estimation methods mature enough, such that when used with antipodal grasp sampling, can outperform the end-to-end grasp synthesis methods? We explore this question in detail by scoping our study to parallel-jaw grippers, 7-DoF grasps, and single-view RGB(-D) image as input. We implement and compare a state-of-the-art, end-to-end grasp synthesis method and three modular methods, which first estimate the object pose and shape for all objects in the scene, and generates grasps using antipodal sampling. We observe that the modular methods outperform the end-to-end method in all our experiments. The modular methods are able to synthesize plenty of grasps, even for small objects, where the end-to-end methods fail. The effectiveness of the modular methods is contingent on the accuracy of the pose and shape estimation, and suffers partial degradation in cluttered scenes — a limitation of the existing pose and shape estimation methods. We also analyze the failure modes and run-times for the three modular methods, which use two different ways of object pose and shape estimation: one based on an encoder-decoder model, while another a diffusion model. Finally, we demonstrate that the single-view object pose and shape estimation methods can be augmented with vision-language models to yield language-conditioned grasps from just single-view RGB-D image as input. We notice comparable performance to the state-of-the-art LERF-TOGO baseline. † † aftertitle: Fig. 1: (a) We implement and analyze four methods for grasp synthesis (only three shown): a state-of-the-art, End-to-End Method ( AnyGrasp [ 11 ] ), and three Modular Methods ( SAM3D-Grasp , CRISP-Grasp and SC-Grasp ), which first estimate the object pose and shape for all objects in the scene and then synthesize grasps using antipodal sampling. We observe that Modular Methods synthesize more successful grasps ( green ) than the End-to-End Method (unsuccessful grasps: red ). (b) The End-to-End Method synthesize more unstable grasps compared to Modular Methods in our real-world experiments. (c) The Modular Methods are augmented with vision-language models to yield language-conditioned grasps from just single-view RGB-D input. We observe comparable performance with the state-of-the-art baseline LERF-TOGO [ 40 ] that uses multiple views. I Introduction The problem of estimating the complete geometry of an object from a single- or partial-view is very important in robotics. It can help robots navigate a scene as well as grasp and manipulate objects with very few observations. It is a capability that mobile manipulators aught to have. Classical works on grasping, for instance, relied on the assumption that such a complete object geometry is available for grasp synthesis [ 43 , 2 ] . Knowing the complete geometry, material, and physical properties, the grasp synthesis problem was framed as an optimization problem to seek the grasp that is most likely to succeed [ 2 , 3 ] . Recent works have used these physics constraints to develop grasp sampling approaches given a 3D mesh of the object [ 32 , 62 , 10 ] . The problem of estimating the complete geometry of objects ( i.e., shape and pose of the object) given a single-view RGB(-D) image has seen key advancements lately [ 51 , 52 , 28 , 14 , 8 , 26 , 55 , 19 , 42 , 48 , 20 , 59 , 27 , 23 , 58 , 33 , 1 ] . Encoder-decoder models have shown the capability to estimate the pose and shape of an object, across object categories [ 42 , 55 , 19 , 48 , 20 ] , and most recently, open-set generalizability [ 46 ] ( i.e., the capability to estimate the pose and shape of previously unseen objects). Previously, researchers were working on category-level object pose and shape estimation, where the template object shape would be known [ 52 , 50 , 63 , 14 , 7 , 8 , 47 , 26 , 30 ] . It was believed that such a category-agnostic generalization would not be possible. Diffusion models have also shown open-set generalizability [ 59 , 27 , 23 , 58 , 33 , 1 ] . These models leverage the progress on novel-view synthesis [ 53 ] , which once framed as a pose conditioned image-to-image generation problem and trained using a large diffusion model shows zero-shot generalizability. Today’s grasp synthesis approaches are dominated by end-to-end regression methods which regress the grasp pose directly from a single- or partial-view of a scene [ 32 , 12 , 44 , 45 , 16 , 11 , 37 ] . These methods were initially inspired by the success of the end-to-end deep learning methods in computer vision [ 3 , 37 ] . They have now shown remarkable capability in terms of increased accuracy, generalizability to unseen objects, and robustness compared to traditional machine learning or analytical methods. End-to-end methods have also enabled faster run times for real-time operations, unlike sampling-based methods [ 37 ] . Contributions. In this work, we inquire if this recent progress in object pose and shape estimation can be leveraged for grasping. We explore this question via experimental analysis. Scoping our study to parallel grippers, 7-DoF grasps, and single-view RGB-D image as input we analyze and compare a state-of-the-art, end-to-end grasp synthesis method and three modular methods, which first estimate object pose and shape of all the objects in the scene, and generate grasps using antipodal sampling [ 32 , 62 ] . We evaluate the grasp synthesis methods with collisions, force closure, and grasp stability. We conduct our experiments in a physics simulator, real-world datasets, and real-world grasping experiments with Xarm7 manipulator arm affixed with a parallel-jaw gripper and an Intel RealSense D455 camera. We demonstrate that the single-view object pose and shape estimation methods can also be leveraged for language-conditioned grasp synthesis with (qualitatively) comparable performance vis-a-vis a state-of-the-art multi-view baseline. II A Discussion with Related Works II-A Object Pose and Shape Estimation The earliest works on object pose and shape estimation were motivated by its application to grasping and manipulation. Early works focused on fitting templates ( e.g., cuboids, ellipsoids, quadrics). Instance-based object pose estimation gained traction for industrial applications where the object shape is known and could be easily retrieved [ 18 ] . Deep learning approaches showed pivotal role in increasing the accuracy of instance-based pose estimation methods [ 57 , 51 , 5 , 36 , 4 , 38 , 28 , 35 , 25 ] . The success of these methods prompted researchers to investigate category-level pose and shape estimation task, where an object template shape is assumed to be known [ 52 , 50 , 63 , 14 , 7 , 8 , 47 , 26 , 30 ] . These methods estimate the object pose and shape by modifying the template shape to fit the object instance in the input

Read the complete text