Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping

Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam · 2026

Read Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping by Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05029.txt.

Description

A robot entering a new room must understand objects it has never seen before. Octavio Arriaga and 5 co-authors built a system for that challenge in this 2026 robotics paper, submitted to a leading robotics letters journal. Instead of learning a black-box scene model from huge datasets, their approach fits a physically explicit scene.

The model combines neural foundation models with differentiable rendering, then solves a series of constrained optimization problems. From a single depth image plus bounding boxes, it estimates meshes, lighting, materials, and the six-degree-of-freedom poses of unseen objects. On standard few-shot pose benchmarks it beat existing algorithms, and its reconstructions were accurate enough to drive a zero-shot grasping task. The authors argue the route offers data-efficient, interpretable autonomy for robots in novel settings.

Catalog record

CREATOROctavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-05029

Browse the full library · About the Tower of Babel