Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping
Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam · 2026
Read Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping by Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-05029.txt.
Description
A robot entering a new room must understand objects it has never seen before. Octavio Arriaga and 5 co-authors built a system for that challenge in this 2026 robotics paper, submitted to a leading robotics letters journal. Instead of learning a black-box scene model from huge datasets, their approach fits a physically explicit scene.
The model combines neural foundation models with differentiable rendering, then solves a series of constrained optimization problems. From a single depth image plus bounding boxes, it estimates meshes, lighting, materials, and the six-degree-of-freedom poses of unseen objects. On standard few-shot pose benchmarks it beat existing algorithms, and its reconstructions were accurate enough to drive a zero-shot grasping task. The authors argue the route offers data-efficient, interpretable autonomy for robots in novel settings.
Catalog record
| CREATOR | Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto, Siddhant Kadwe, Rebecca Adam |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-05029