Demystifying Video Reasoning
Ruisi Wang 1, Zhongang Cai ✉ , 1 {}^{\textrm{{\char 0\relax}},1}, Fanyi Pu 1,2, Junxiang Xu 1, Wanqi Yin 1, Maijunxian Wang 3, Ran Ji 4, Chenyang Gu 1, Bo Li 2, Ziqi Huang 2, Hokin Deng 5, Dahua Lin 1, Ziwei Liu 2, Lei Yang 1 ✉ Corresponding Author 1 SenseTime Research 2 Nanyang Technological University 3 University of California, Berkeley 4 University of California, San Diego 5 Carnegie Mellon University · 2026
Read Demystifying Video Reasoning by Ruisi Wang 1, Zhongang Cai ✉ , 1 {}^{\textrm{{\char 0\relax}},1}, Fanyi Pu 1,2, Junxiang Xu 1, Wanqi Yin 1, Maijunxian Wang 3, Ran Ji 4, Chenyang Gu 1, Bo Li 2, Ziqi Huang 2, Hokin Deng 5, Dahua Lin 1, Ziwei Liu 2, Lei Yang 1 ✉ Corresponding Author 1 SenseTime Research 2 Nanyang Technological University 3 University of California, Berkeley 4 University of California, San Diego 5 Carnegie Mellon University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16870.txt.
Introduction
Wang and co-authors dissect how video diffusion models reason. Earlier work credited a Chain-of-Frames effect, with reasoning spread across frames. This team finds reasoning instead unfolds along denoising steps, a Chain-of-Steps. Early steps explore many candidate solutions, then narrow to one answer. They also spot working memory, self-correction, and perception before action. Middle transformer layers do the key reasoning work. As a proof of concept, they ensemble latent paths from identical models with different seeds. This training-free ensemble improves reasoning in their tests.
Catalog record
| CREATOR | Ruisi Wang 1, Zhongang Cai ✉ , 1 {}^{\textrm{{\char 0\relax}},1}, Fanyi Pu 1,2, Junxiang Xu 1, Wanqi Yin 1, Maijunxian Wang 3, Ran Ji 4, Chenyang Gu 1, Bo Li 2, Ziqi Huang 2, Hokin Deng 5, Dahua Lin 1, Ziwei Liu 2, Lei Yang 1 ✉ Corresponding Author 1 SenseTime Research 2 Nanyang Technological University 3 University of California, Berkeley 4 University of California, San Diego 5 Carnegie Mellon University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46201 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16870
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Demystifying Video Reasoning Ruisi Wang 1, Zhongang Cai ✉ , 1 {}^{\textrm{{\char 0\relax}},1}, Fanyi Pu 1,2, Junxiang Xu 1, Wanqi Yin 1, Maijunxian Wang 3, Ran Ji 4, Chenyang Gu 1, Bo Li 2, Ziqi Huang 2, Hokin Deng 5, Dahua Lin 1, Ziwei Liu 2, Lei Yang 1 ✉ Corresponding Author 1 SenseTime Research 2 Nanyang Technological University 3 University of California, Berkeley 4 University of California, San Diego 5 Carnegie Mellon University Abstract Recent advances in video generation have revealed an unexpected phenomenon: diffusion-based video models exhibit non-trivial reasoning capabilities. Prior work attributes this to a Chain-of-Frames (CoF) mechanism, where reasoning is assumed to unfold sequentially across video frames. In this work, we challenge this assumption and uncover a fundamentally different mechanism. We show that reasoning in video models instead primarily emerges along the diffusion denoising steps . Through qualitative analysis and targeted probing experiments, we find that models explore multiple candidate solutions in early denoising steps and progressively converge to a final answer, a process we term Chain-of-Steps (CoS) . Beyond this core mechanism, we identify several emergent reasoning behaviors critical to model performance: (1) working memory , enabling persistent reference; (2) self-correction and enhancement , allowing recovery from incorrect intermediate solutions; and (3) perception before action , where early steps establish semantic grounding and later steps perform structured manipulation. During a diffusion step, we further uncover self-evolved functional specialization within Diffusion Transformers, where early layers encode dense perceptual structure, middle layers execute reasoning, and later layers consolidate latent representations. Motivated by these insights, we present a simple training-free strategy as a proof-of-concept, demonstrating how reasoning can be improved by ensembling latent trajectories from identical models with different random seeds. Overall, our work provides a systematic understanding of how reasoning emerges in video generation models, offering a foundation to guide future research in better exploiting the inherent reasoning dynamics of video models as a new substrate for intelligence. \checkdata [Homepage] https://www.wruisi.com/demystifying_video_reasoning Figure 1 : Chain-of-Steps. We discover that video reasoning occurs along the diffusion steps with surprising emergent behaviors such as making multiple possible moves ( e.g. , paths) simultaneously at early steps, gradually pruning suboptimal choices during middle steps, and reaching a final decision at the late steps. This maze-solving example asks the model to start from the green circle in the top-left corner and find the red rectangle. Key regions of interest are color-coded and enlarged on the right. 1 Introduction Video generation models have transformed the landscape of movie, gaming, and entertainment industries. However, most research has focused primarily on their ability to produce high-fidelity, realistic, and visually appealing videos. Recent advances have revealed an unexpected phenomenon: diffusion-based video models exhibit non-trivial reasoning capabilities in spatiotemporally consistent visual environments [ 62 ] . Prior work attributes this behavior to a Chain-of-Frames (CoF) mechanism, suggesting that reasoning unfolds sequentially across video frames. Despite this intriguing discovery, the underlying mechanisms of video reasoning remain largely unexplored. With the recent release of large-scale video reasoning datasets and open-source foundation models [ 58 ] , we now have the opportunity to systematically investigate this capability. Leveraging these resources, we conduct the first comprehensive dissection of video reasoning and uncover a fundamentally different mechanism: reasoning in diffusion-based video models primarily emerges along the denoising process rather than across frames. Our key discovery challenges the prevailing Chain-of-Frames (CoF) hypothesis [ 62 , 66 ] , which assumes that video reasoning unfolds sequentially across frames. Instead, we find that reasoning does not primarily operate along the temporal dimension. Rather, it emerges along the diffusion denoising steps, progressing throughout generation. We term this mechanism Chain-of-Steps (CoS) . This finding suggests a fundamentally different view of how diffusion-based video models reason. Due to bidirectional attention over the entire sequence, reasoning is performed across all frames simultaneously at each denoising step, with intermediate hypotheses progressively refined as the process unfolds. Qualitative analysis reveals intriguing dynamics. In early denoising steps, the model often entertains multiple possibilities (populating alternative trajectories or superimposing candidate outcomes) before gradually converging to a final solution in later steps. Moreover, noise perturbation analysis shows that disruptions at specific denoising steps significantly degrade performance, whereas frame-wise perturbations have a much weaker impact. Further information propagation analysis identifies that the conclusion primarily solidifies during the middle diffusion steps. Furthermore, we uncover several surprising emergent behaviors in video reasoning models that are strikingly similar to those observed in early studies of Large Language Models (LLMs). First, these models exhibit a form of working memory that is crucial for tasks requiring persistent references ( e.g. , object permanence). Second, we observe that video models can self-correct errors during the CoS reasoning process, rather than committing to incorrect trajectories throughout generation. Third, video models exhibit a "perception before action" behavior, where early diffusion steps prioritize localizing target objects before subsequent steps perform more complex reasoning and manipulation. We further conduct a fine-grained analysis of the Diffusion Transformer by examining token representations within a single diffusion step. This reveals the self-evolved, diverse, task-agnostic functional layers throughout the network. Within a diffusion step, early layers focus on dense perceptual understanding ( e.g. , separating foreground from background and identifying basic geometric structures), while a set of critical middle layers performs the bulk of the reasoning. The final layers then consolidate the latent representation to produce the video state for the next step. Motivated by these insights, we present a simple training-free method as a proof-of-concept for improving video reasoning models. Given that the model inherently explores multiple reasoning paths during the diffusion process, we propose a inference-time ensemble strategy that merges latents produced by three identical models with different random seeds. This approach encourages the model to retain a richer set of candidate reasoning trajectories during generation. As a result, the model explores more diverse reasoning paths and is more likely to converge to the correct solution, illustrating a way to utilize our findings to design more effective video reasoning systems. In summary, we investigate the underlying mechanisms of video reasoning in diffusion models and identify Chain-of-Steps (CoS), a reasoning process that unfolds along the denoising trajectory. We further uncover several emergent reasoning behaviors that arise in these models. Building on these insights, we demonstrate how such mechanisms can be exploited through a simple training-free strategy for reasoning path ensembling. We believe our findings provide a foundation for understanding and advancing video reasoning, positioning it as a promising next-generation substrate for machine intelligence. 2 Related Works 2.1 Reasoning in Language and Multimodal Models Recent studies show that large language models (LLMs) exhibit remarkable reasoning capabilities. Early work identifies emergent behaviors that arise as models scale in size and data [ 60 ] , and demonstrates that Chain-of-Thought (CoT) prompting, which elicits intermediate reasoning steps, significantly improves performance [ 61 ] . Subsequent work explores mechanisms such as self-reflection, correction, and action [ 39 , 72 , 21 , 69 ] . Coconut further suggests that reasoning can also occur implicitly within latent representations [ 18 ] . Meanwhile, research has increasingly explored extending reasoning beyond language into multimodal settings. Early progress in vision-language models (VLMs) enables reasoning over images in addition to text [ 1 , 32 ,