Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions

Yuanyuan Wang, Wenjie Wang, Kun Zhang, Mingming Gong · 2026

Read Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions by Yuanyuan Wang, Wenjie Wang, Kun Zhang, Mingming Gong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00115.txt.

Introduction

Wang and co-authors study a hard question in video learning. Can a model read true laws of motion from raw pixels alone? Their work looks at systems ruled by second-order equations that do not change over time. It asks when the hidden state can be pinned down with no doubt left.

The authors prove a slope test on level sets makes the learned space match the true state. Under that test, exact values for the physics can be read off. One short clip can be enough for systems that ring down slowly. Other cases need three runs that differ from each other. A new loss term also keeps the learned space from shrinking to nothing. Tests on made-up and real clips back the claims.

Catalog record

CREATORYuanyuan Wang, Wenjie Wang, Kun Zhang, Mingming Gong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE127 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00115

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions Yuanyuan Wang, Wenjie Wang, Kun Zhang, Mingming Gong Abstract Bridging the gap between visual realism and physical understanding is a core challenge for video-based world models. We study the structural identifiability of continuous-time physical laws from raw pixels, focusing on whether an encoder-only pipeline can uniquely recover the parameters of second-order linear ODEs. We prove that a level-set slope-coverage condition ensures the learned latent space is locally affine to the true physical state, enabling exact parameter recovery. Our theory provides the first characterization of minimal data requirements across damping regimes, establishing that underdamped systems are identifiable from a single video clip, whereas other regimes require three diverse trajectories. We further introduce a variance-floor regularizer to stabilize the decoder-free objective and prevent latent collapse. Validated on synthetic and real-world data, our approach demonstrates that interpretable physical constants can be reliably estimated from video without the need for compute-intensive pixel reconstruction, ensuring both physical correctness and transparency. Code is available at https://github.com/wenjiewang3/PhysicsFromVideo . † † affiliationnotice: Equal contribution 1 Introduction The quest for world models has renewed a central debate in AI: do modern generative video models truly internalize physical laws, or do they primarily act as powerful pixel predictors that achieve visual realism without reliable physical grounding? Recent physics-centric evaluations, such as Physics-IQ ( Motamed et al., 2025 ) (and related benchmarks ( Kang et al., 2024 ) ), suggest that while these models produce visually stunning videos, they exhibit a fundamental decoupling between visual fidelity and physical correctness. Specifically, they frequently violate basic physical invariants, revealing that high-fidelity synthesis can mask a systematic lack of grounding in the causal laws of the observed world. This discrepancy highlights a critical gap: achieving visual plausibility does not imply the recovery of the true, interpretable physical laws that govern the observed world. This gap motivates a complementary goal: recovering interpretable physical laws and parameters directly from raw videos. Commodity cameras are increasingly used as scalable, non-contact sensors across infrastructure monitoring ( Bai et al., 2023 ) , digital health and biomechanics ( Uhlrich et al., 2023 ; Boswell et al., 2023 ) , and robotics/graphics ( Zhu et al., 2025 ; Zhao et al., 2025 ) . If pixels can be mapped to reliable continuous-time dynamical models, video can enable monitoring, prediction, diagnosis, and design where dedicated instrumentation is unavailable or costly. Such recovered laws can also act as explicit constraints or priors for physics-grounded video prediction and generation, helping bridge the disconnect between visual realism and physical correctness. Yet identification from video is fundamentally harder than classical system identification: the state is latent, supervision is limited, and the compact coordinate system in which the dynamics are simple is unknown. This leads to a foundational, yet unresolved, theoretical question: Can we identify the true underlying physical parameters from raw video, and if so, under what formal conditions? Supervised regression from video to states or parameters requires high-quality ground truth that is often expensive or infeasible to obtain at scale ( Meijering et al., 2016 ; Varoquaux and Cheplygina, 2022 ) , motivating unsupervised approaches that incorporate known governing equations as inductive bias. Many pipelines nevertheless rely on reconstruction-driven objectives, either via autoencoder–decoder designs ( Kandukuri et al., 2020 ; Kandukuri et al., 2022 ) or analysis-by-synthesis through differentiable rendering and photometric matching ( Ma et al., 2022 ; Stotko et al., 2024 ; Zhao et al., 2025 ) . Many such methods mitigate appearance ambiguity in practice through structured decoders, restricted functional forms, or stronger rendering priors. Their identifiability, however, depends jointly on the dynamics model and the observation-model priors: when the reconstruction loss operates in pixel space and the decoder or renderer is expressive enough, it can in principle compensate for physically incorrect parameters, so the recovered parameters need not be unique without additional structural assumptions. Furthermore, high-fidelity reconstruction is data- and compute-intensive and can dominate training, even when the downstream task does not require pixel-level reconstruction. Recent decoder-free, encoder-only formulations ( Garcia et al., 2025 ) take a bold step by dropping reconstruction and enforcing physics directly in latent space, substantially reducing reliance on pixel synthesis. However, while removing the decoder mitigates appearance confounding, it exposes an even more fundamental theoretical vacuum: without the pixel-level constraint, the latent coordinate system is defined only up to an arbitrary reparameterization. Consequently, a good fit in latent space provides no assurance that the recovered parameters are unique or match the true physical system. Establishing structural identifiability is therefore a prerequisite for reliable physics-from-video, especially in ill-conditioned regimes ( Gutenkunst et al., 2007 ) . In this work, we study identifiability in an encoder-only , decoder-free setting, isolating when a video trajectory itself pins down a physically meaningful coordinate system and yields provable parameter recovery. Since identifying physical parameters from raw video is intrinsically challenging, we start with a simple yet widely used continuous-time model that enables sharp analysis: the homogeneous second-order linear time-invariant (LTI) ODE z^{\prime\prime}(t)+\gamma_{1}z^{\prime}(t)+\gamma_{0}z(t)=0 , a canonical abstraction for damped resonance and single-mode transients that underlies ring-down/modal analysis and damping estimation in mechanical vibration and structural monitoring ( Ewins, 2009 ; Li et al., 2022 ; Yokoyama, 2023 ; Noh et al., 2025 ) . Because many applications rely on the interpretability of (\gamma_{1},\gamma_{0}) , understanding when these parameters are uniquely recoverable from raw video is a prerequisite for using decoder-free vision as a reliable diagnostic instrument. Contributions. We (i) provide, to the best of our knowledge, the first structural identifiability guarantees for learning continuous-time physical parameters from raw video alone in an encoder-only framework; specifically, for the homogeneous second-order linear ODE, we characterize when a single clip suffices and when multiple trajectories are necessary across damping regimes; (ii) derive non-asymptotic finite-sample error bounds that separate measurement noise, finite-difference bias, and design conditioning; (iii) propose a simple variance-floor regularizer that improves conditioning and stabilizes training. 2 Related Work Reconstruction-driven physics and parameter inference from video. A large class of methods learns dynamics and physical parameters from raw video by coupling visual representation learning with differentiable time evolution and supervising via pixel-space reconstruction. In autoencoder–decoder pipelines, an encoder maps frames to latents, a dynamics module predicts their temporal rollout, and a decoder reconstructs future frames (often with masks, spatial transformers, or object-centric structure to encourage factorized explanations) ( Watters et al., 2017 ; de Avila Belbute-Peres et al., 2018 ; Asenov et al., 2019 ; Kandukuri et al., 2020 ; Kandukuri et al., 2022 ) . A related analysis-by-synthesis line treats physical parameters as optimization variables and fits them through differentiable simulation and rendering or photometric alignment ( Jaques et al., 2020 ; Murthy et al., 2020 ; Ma et al., 2022 ; Stotko et al., 2024 ; Zhao et al., 2025 ) . While these approaches can avoid manual labels, parameter estimation is fundamentally mediated by the observation model: nuisance factors such as lighting, texture, background and viewpoint can compensate for incorrect dynamics yet still achieve low pixel loss. As a result, the inverse problem is often not identifiable, leading to shortcut solutions, sensitivity to initialization, and degraded transfer under domain shift. Decoder-free latent-space physics from video. To reduce reliance

Read the complete text