Vid2Sid : Videos Can Help Close the Sim2Real Gap
Kevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3 · 2026
Read Vid2Sid : Videos Can Help Close the Sim2Real Gap by Kevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19359.txt.
Description
Matching a robot simulator to real hardware often relies on hand tuning or black-box optimizers that cannot say which physical mismatch drives the error. Kevin Qiu and three co-authors present Vid2Sid, a pipeline that identifies systems from video. It couples foundation-model perception with a vision-language model optimizer. That optimizer studies paired simulation and real videos, diagnoses concrete mismatches, and suggests physics parameter updates, each with a written rationale.
Tests cover a tendon-actuated finger in MuJoCo and a deformable continuum tentacle in PyElastica. On held-out real controls, Vid2Sid ranks best on average, matching or beating black-box optimizers while explaining each step. Simulation-to-simulation checks show it recovering true parameters most accurately, with mean relative error under 13 percent versus 28 to 98 percent for rivals.
Catalog record
| CREATOR | Kevin Qiu 1,2 , Yu Zhang 3 , Marek Cygan 1,4 , Josie Hughes 3 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-19359