Generative Perception of Shape and Material from Differential Motion

Xinran Nicole Han, Ko Nishino, Todd Zickler · 2025

Read Generative Perception of Shape and Material from Differential Motion by Xinran Nicole Han, Ko Nishino, Todd Zickler (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02473.txt.

Description

Xinran Nicole Han, Ko Nishino, and Todd Zickler teach a model to read an object's shape and material from the way it wobbles, in work from 2025. A single still image leaves shape, material, and lighting hopelessly entangled, yet people resolve much of that ambiguity with a slight head movement.

Their diffusion model takes a short video of an object in small differential motion and generates maps of its shape and material properties at once. Trained on synthetic motion videos, it behaves sensibly under uncertainty. Given a still view it offers several plausible readings. Given motion, its answers converge toward the truth. It also performs well on real objects with clearer cues.

Catalog record

CREATORXinran Nicole Han, Ko Nishino, Todd Zickler
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-02473

Browse the full library · About the Tower of Babel