X-Sim : Cross-Embodiment Learning via Real-to-Sim-to-Real

Prithwish Dan Kushal Kedia Angela Chao Edward W. Duan Maximus A. Pace Wei-Chiu Ma Sanjiban Choudhury Cornell University · 2025

Read X-Sim : Cross-Embodiment Learning via Real-to-Sim-to-Real by Prithwish Dan Kushal Kedia Angela Chao Edward W. Duan Maximus A. Pace Wei-Chiu Ma Sanjiban Choudhury Cornell University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-07096.txt.

Description

Prithwish Dan and co-authors taught robots in 2025 from human videos without any robot demonstration data. Videos of people lack the action labels that imitation learning needs, and body gaps break direct motion mapping. Their frame, X-Sim, rebuilds a photo-true simulation from a depth video and tracks how objects move. Those object paths set rewards for a policy trained inside the simulation.

The policy is then poured into an image-driven diffusion policy using fake rollouts under varied views and light. An online trick aligns real and fake views during use. Across five tasks in two settings it beat hand-tracking and sim-to-real rivals by thirty percent on task progress, and matched cloning with a tenth of the data time.

Catalog record

CREATORPrithwish Dan Kushal Kedia Angela Chao Edward W. Duan Maximus A. Pace Wei-Chiu Ma Sanjiban Choudhury Cornell University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-07096

Browse the full library · About the Tower of Babel