Vid2Sid : Videos Can Help Close the Sim2Real Gap

Kevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3 · 2026

Read Vid2Sid : Videos Can Help Close the Sim2Real Gap by Kevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19359.txt.

Description

Matching a robot simulator to real hardware often relies on hand tuning or black-box optimizers that cannot say which physical mismatch drives the error. Kevin Qiu and three co-authors present Vid2Sid, a pipeline that identifies systems from video. It couples foundation-model perception with a vision-language model optimizer. That optimizer studies paired simulation and real videos, diagnoses concrete mismatches, and suggests physics parameter updates, each with a written rationale.

Tests cover a tendon-actuated finger in MuJoCo and a deformable continuum tentacle in PyElastica. On held-out real controls, Vid2Sid ranks best on average, matching or beating black-box optimizers while explaining each step. Simulation-to-simulation checks show it recovering true parameters most accurately, with mean relative error under 13 percent versus 28 to 98 percent for rivals.

Catalog record

CREATORKevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19359

Browse the full library · About the Tower of Babel