Interpreting Physics in Video World Models

Sonia Joseph, Quentin Garrido, Randall Balestriero, Matthew Kowal, Thomas Fel, Shahab Bakhtiari, Blake Richards, Mike Rabbat · 2026

Read Interpreting Physics in Video World Models by Sonia Joseph, Quentin Garrido, Randall Balestriero, Matthew Kowal, Thomas Fel, Shahab Bakhtiari, Blake Richards, Mike Rabbat (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07050.txt.

Description

Sonia Joseph and seven co-authors probe what video world models hold about physics inside. Video models score well on intuitive physics tests, yet it is unclear if they store physical variables in neat, factored form, like a physics engine, or in a distributed, task-tied form. This is the first study to look directly at physical representations inside large-scale video encoders.

Four tools drive the probe. The team probes layer by layer. They map subspace geometry. They decode at patch level. They ablate targeted attention heads. Across designs, a sharp mid-depth transition appears, which the team calls the Physics Emergence Zone. Physical variables become accessible there, peak just after, and fade toward the output layers. Speed and acceleration show up from early layers, but direction shows up only at the transition, coded in a high-dimensional population with circular geometry. The models predict physics without factoring it.

Catalog record

CREATORSonia Joseph, Quentin Garrido, Randall Balestriero, Matthew Kowal, Thomas Fel, Shahab Bakhtiari, Blake Richards, Mike Rabbat
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07050

Browse the full library · About the Tower of Babel