Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi · 2026
Read Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation by Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28230.txt.
Introduction
Hassan and colleagues let frozen video generators judge their own physical plausibility. Proprio reads the model's flow residual under small latent nudges as a self-score: better-explained samples yield smaller, steadier residuals. Used for best-of-N search or gradient refinement, it lifted physics scores markedly and won roughly two-thirds of human comparisons for plausibility.
Catalog record
| CREATOR | Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63576 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.28230
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation Mariam Hassan, Kaouther Messaoud, Wuyang Li, Alexandre Alahi Abstract Modern video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio , a training-free framework that enables a frozen video generator to assess and improve the physical plausibility of its own outputs. Inspired by proprioception , the biological sense of one’s own movement, Proprio treats the model’s flow residual under controlled latent perturbations as a self-scoring signal. Samples that are better explained by the generator’s learned dynamics induce smaller and more stable residuals. We aggregate this signal across timesteps and perturbations, focus it on motion-relevant regions with a dynamic spatiotemporal mask, and use it for best-of- N N search, gradient-based self-refinement, or both. Across text-to-video and image-to-video benchmarks, Proprio consistently improves physical plausibility, outperforming VLM-based scoring, and external world-model baselines in several settings. With TurboWan2.2, Proprio improves Physics-IQ from 32.2 to 37.5 (+16.5%) and VideoPhy2-hard physical commonsense from 45.6 to 55.0 (+20.6%). Human evaluation further shows that raters prefer Proprio-selected or refined videos for physical plausibility in roughly two-thirds of comparisons. These results suggest that frozen video generators contain actionable internal signals for evaluating and improving the physical plausibility of their own outputs. Figure 1 : Proprio enables a video generator to evaluate and improve its own generation. Left: qualitative refinement comparisons for TurboWan2.2. Right: Proprio computes a latent self-score from the model’s denoising residual and uses it for either sample selection or inference-time refinement. 1 Introduction Diffusion models [ 14 , 25 ] have achieved remarkable progress, emerging as dominant paradigms for high-fidelity video generation [ 8 , 1 , 34 ] . Despite their impressive visual quality, these models still struggle to maintain physical plausibility , often producing videos that violate basic real-world constraints. Such failures not only limit perceptual realism but also undermine the utility of these models as reliable world models for downstream applications such as robotics, planning, and simulation. To address this, existing efforts typically rely on external sources of feedback to provide physical assessment and supervision. Some work inject physical structure through curated datasets, physical priors, or differentiable simulation [ 35 , 24 ] . Another line of work guides generation using external evaluators, including vision-language models, or pretrained latent world models such as V-JEPA [ 37 , 9 , 44 , 41 , 3 , 7 ] . These methods provide useful signals for selecting physically plausible generations and have shown promising results in improving physical realism. While effective, this external-feedback paradigm faces two key limitations. (1) External evaluators introduce their own biases and failures : VLMs struggle to assess physical plausibility because their judgments are often entangled with text-image alignment objectives and visual appearance [ 29 ] . Pretrained world models provide a more dynamics-oriented signal, but operate on decoded videos and map them into their own representation space, creating a representation mismatch with the generator’s latent space. (2) Misalignment with the generator’s internal dynamics : external evaluators are trained with different training objectives, so their feedback is not directly aligned with the generator’s sampling process and may miss generator-specific inconsistencies across denoising steps. These limitations motivate an intrinsic perspective on physical plausibility: Can a video generator leverage its own denoising dynamics as a self-contained signal for producing more physically plausible videos? Here, denoising dynamics refers to the model-predicted update direction at each noise level. We focus on its prediction error, which we call the flow residual : the discrepancy between the predicted denoising or velocity direction and the target induced by a controlled perturbation. Conceptually , the flow residual can be interpreted as an approximate likelihood signal [ 20 ] , providing inherent cues for assessing physical validity, as previously explored in curated simulated video pairs [ 40 ] . In contrast, we reinterpret this likelihood-preference signal as a generator-native objective. Since video generators are trained on large-scale real video datasets, their learned distributions implicitly capture regularities of real-world dynamics. It is therefore natural to treat consistency with such learned dynamics as an intrinsic proxy for physical plausibility. Empirically , we test this intuition with a small-scale diagnostic check. Given a set of real videos, we pair each video with a generated counterpart that exhibits a physical failure using a video model, and compare their residual scores under that model (see Appendix A.1 ). In 82.2% of the pairs, the residual signal prefers the ground-truth video over the model’s own physically implausible generation, suggesting the model’s internal denoising dynamics can distinguish natural video dynamics from its own failure cases. Motivated by this, we propose Proprio , a generator-native, training-free framework that converts internal denoising signals into an internal proxy for physical plausibility. Inspired by proprioception , the biological ability to sense one’s motion without external observation, Proprio enables a frozen video generator to assess and improve its own outputs. Specifically, we define a noise-conditioned self-consistency signal based on the model’s denoising residual under controlled latent perturbations. We robustly estimate the score by aggregating residuals across multiple timesteps with inverse-variance weighting , which downweights noisy estimates, and by applying a dynamic spatiotemporal mask to emphasize motion-relevant regions where physical failures occur. Beyond scoring, Proprio uses the score as a self-refinement objective. It refines generations by backpropagating through the frozen sampler and updating the initial noise to produce a more internally consistent video. We evaluate Proprio in three inference-time strategies: best-of- N search, self-refinement, and hybrid search-and-refine approach (see Fig. 1 ). For image-to-video (I2V) and text-to-video (T2V) generation, Proprio improves physical accuracy over VLM-based, and outperforms external pretrained world-model in several settings. These results suggest that a frozen video generator can be used to generate videos and to guide selection and refinement of its outputs. Our contributions are as follows: • Generator-native self-scoring: We introduce Proprio, a training-free framework that reuses a frozen generator’s denoising residual as an intrinsic score for physical plausibility. • Inference-time self-alignment: We turn this score into an optimization objective for a frozen generator, refining its outputs by updating only the initial noise, and optionally combine this refinement with best-of- N selection. • Empirical validation: We demonstrate consistent gains on I2V and T2V physics benchmarks, showing that a single frozen generator can act as both sampler and self-evaluator, guiding selection and refinement toward more physically plausible dynamics. 2 Related Work Physical plausibility in video generation. Recent video generative models [ 8 , 6 , 19 , 34 , 36 , 38 , 12 , 31 ] have achieved impressive visual fidelity, yet they still frequently violate basic physical principles [ 18 , 29 ] . Previous work explored three lines of work to solve problem. The first improves physical fidelity during training by introducing physics-aware data, supervision, or learning objectives [ 35 , 9 , 35 ] . The second incorporates explicit physical structure or simulation into the generation process [ 24 ] . The final approach addresses the problem at inference time through external guidance, for example using VLM-based planning or reward signals from pretrained world models. VLIPP and GPT4motion use a VLM as physics reward [ 37 , 26 ] , while WMReward uses VJEPA-2 as an external latent world