Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
Min Zhao 1 , 2 ∗ {}^{1,2\penalty\ *} , Hongzhou Zhu 1 , 2 ∗ {}^{1,2\penalty\ *} , Kaiwen Zheng 1 , Zihan Zhou 3 , Bokai Yan 3 , Xinyuan Li 1 , Xiao Yang 1 , Chongxuan Li 3 , Jun Zhu 1 , 2 † {}^{1,2\penalty\ \dagger} 1 Tsinghua University 2 ShengShu 3 Renmin University of China ∗ Equal contribution. † Correspondence to: Jun Zhu. {gracezhao1997, suinibian74}@gmail.com; [email protected] · 2026
Read Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation by Min Zhao 1 , 2 ∗ {}^{1,2\penalty\ *} , Hongzhou Zhu 1 , 2 ∗ {}^{1,2\penalty\ *} , Kaiwen Zheng 1 , Zihan Zhou 3 , Bokai Yan 3 , Xinyuan Li 1 , Xiao Yang 1 , Chongxuan Li 3 , Jun Zhu 1 , 2 † {}^{1,2\penalty\ \dagger} 1 Tsinghua University 2 ShengShu 3 Renmin University of China ∗ Equal contribution. † Correspondence to: Jun Zhu. {gracezhao1997, suinibian74}@gmail.com; [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15141.txt.
Introduction
Min Zhao and eight others present Causal Forcing++, a pipeline for real-time video generation with frame-wise autoregression in only one or two sampling steps. Old few-step distillation works chunk-wise at four steps, but frame-wise work exposes student setup as the key bottleneck. Causal consistency distillation supplies teaching from a single online teacher step between nearby timesteps, with no stored full trajectories. The result beats the four-step chunk-wise predecessor on frame-wise benchmarks.
Catalog record
| CREATOR | Min Zhao 1 , 2 ∗ {}^{1,2\penalty\ *} , Hongzhou Zhu 1 , 2 ∗ {}^{1,2\penalty\ *} , Kaiwen Zheng 1 , Zihan Zhou 3 , Bokai Yan 3 , Xinyuan Li 1 , Xiao Yang 1 , Chongxuan Li 3 , Jun Zhu 1 , 2 † {}^{1,2\penalty\ \dagger} 1 Tsinghua University 2 ShengShu 3 Renmin University of China ∗ Equal contribution. † Correspondence to: Jun Zhu. {gracezhao1997, suinibian74}@gmail.com; [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47973 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15141
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation Min Zhao 1 , 2 ∗ {}^{1,2\penalty\ *} , Hongzhou Zhu 1 , 2 ∗ {}^{1,2\penalty\ *} , Kaiwen Zheng 1 , Zihan Zhou 3 , Bokai Yan 3 , Xinyuan Li 1 , Xiao Yang 1 , Chongxuan Li 3 , Jun Zhu 1 , 2 † {}^{1,2\penalty\ \dagger} 1 Tsinghua University 2 ShengShu 3 Renmin University of China ∗ Equal contribution. † Correspondence to: Jun Zhu. {gracezhao1997, suinibian74}@gmail.com; [email protected] Abstract Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong results in the chunk-wise 4-step regime by distilling bidirectional base models into few-step AR students, but they remain limited by coarse response granularity and non-negligible sampling latency. In this paper, we study a more aggressive setting: frame-wise autoregression with only 1–2 sampling steps. In this regime, we identify the initialization of a few-step AR student as the key bottleneck: existing strategies are either target-misaligned, incapable of few-step generation, or too costly to scale. We propose Causal Forcing++ , a principled and scalable pipeline that uses causal consistency distillation (causal CD) for few-step AR initialization. The core idea is that causal CD learns the same AR-conditional flow map as causal ODE distillation, but obtains supervision from a single online teacher ODE step between adjacent timesteps, avoiding the need to precompute and store full PF-ODE trajectories. This makes the initialization both more efficient and easier to optimize. The resulting pipeline, Causal Forcing++, surpasses the SOTA 4-step chunk-wise Causal Forcing under the frame-wise 2-step setting by 0.1 in VBench Total, 0.3 in VBench Quality, and 0.335 in VisionReward, while reducing first-frame latency by 50% and Stage 2 training cost by ∼ \sim 4 × 4\times . We further extend the pipeline to action-conditioned world model generation in the spirit of Genie3. Project Page: https://github.com/thu-ml/Causal-Forcing & https://github.com/shengshu-ai/minWM . 1 Introduction Video generation models are rapidly evolving from passive content generators into interactive world models [ 3 , 2 , 41 , 19 , 54 , 22 , 66 , 38 , 1 , 14 , 17 , 39 , 8 , 55 ] , where low latency, streaming rollout, and user-controllable interaction are essential. Autoregressive (AR) diffusion models [ 16 , 40 , 5 ] are a natural fit for this goal, as they perform causal rollout across frames or chunks while retaining diffusion-based generation within each segment. Recent AR diffusion distillation methods [ 58 , 13 , 67 , 14 , 38 ] have achieved promising results by distilling bidirectional video diffusion models, such as Wan [ 41 ] and Hunyuan [ 19 ] , into few-step AR students. However, these methods typically rely on chunk-wise autoregression with 4-step sampling, which still falls short of real-time interaction due to coarse response granularity and non-negligible sampling latency. We therefore push AR diffusion distillation to a more aggressive and largely underexplored regime: frame-wise autoregression with only 1–2 sampling steps . We identify the initialization of a few-step AR student before asymmetric DMD as the key bottleneck in this aggressive regime, where existing strategies fail in complementary ways. ODE initialization with a bidirectional teacher, as used in CausVid [ 58 ] and Self Forcing [ 13 ] , is architecturally misaligned with causal rollout: the teacher trajectory depends on future frames that are unavailable to an AR student, thereby providing an incorrect regression target. Directly using a multi-step AR diffusion model for initialization, as in LiveAvatar [ 14 ] and WorldPlay [ 38 ] , avoids this mismatch but lacks few-step generation capability; under frame-wise 1–2 step generation, its per-frame approximation error is severely amplified during self-rollout. Causal ODE initialization, as in Causal Forcing [ 67 ] , corrects the learning target by distilling from an AR teacher, but requires generating full multi-step PF-ODE trajectories for every training sample, making it costly to scale. Therefore, a satisfactory initialization for this regime must be simultaneously AR , few-step , and scalable . To this end, we introduce Causal Forcing++ , a principled and scalable pipeline that uses causal consistency distillation (causal CD) for few-step AR student initialization. Our key observation is that causal ODE distillation and causal CD aim to learn the same object: the AR-conditional flow map (or namely the consistency function [ 35 , 42 , 30 , 65 ] ) of the teacher. They differ, however, in how the supervision is obtained. Causal ODE distillation requires the AR teacher to generate an entire multi-step PF-ODE trajectory for each training sample, which must be precomputed and stored offline. In contrast, causal CD obtains supervision from a single online teacher ODE step between adjacent timesteps on real videos. Therefore, causal CD serves as a principled substitute for causal ODE initialization, while avoiding the expensive trajectory-generation bottleneck. Beyond efficiency, this local supervision also improves quality: adjacent-timestep consistency yields a smaller per-step optimization gap than causal ODE distillation, which regresses noisy intermediate states directly to clean endpoints [ 29 ] . As a result, causal CD is easier to optimize and empirically produces a stronger few-step AR student. Experiments on Wan2.1-1.3B [ 41 ] validate the effectiveness of Causal Forcing++ in this aggressive low-latency regime. Under frame-wise 2-step generation, Causal Forcing++ achieves the best overall performance among existing AR diffusion distillation methods, improving VBench Total, VBench Quality, and VisionReward over prior methods while reducing first-frame latency by 50%. Ablation studies further show that causal CD consistently matches or outperforms causal ODE initialization across 1-step, 2-step, and 4-step settings, while reducing the Stage 2 cost by about 4\times and requiring no auxiliary trajectory storage. We also examine causal score-distillation initialization and find that, although it can produce sharper early frames, its mode-seeking behavior makes it more sensitive to accumulated history errors during AR rollout, leading to stronger exposure bias. Finally, we demonstrate that Causal Forcing++ naturally extends to action-conditioned world model generation by distilling a camera-pose-conditioned generator into an interactive AR world models. Figure 1 : Overall framework of our Causal Forcing++ and the comparison with existing methods. (a) Causal Forcing (CF) fixes Self Forcing (SF)’s frame-level injectivity flaw but remains costly; our Causal Forcing++ (CF++) is theoretically sound, efficient, and scalable. (b) Our CF++ reduces training cost by 4 \times , requires no extra data curation, and achieves 50% lower latency and higher VBench scores than SF and CF. 2 Background 2.1 Autoregressive Diffusion for Interactive Video Generation Generative modeling through diffusion. Score-based diffusion models [ 11 , 37 ] typically perturb the data {\bm{x}}_{0}\sim p_{\mathrm{data}}({\bm{x}}_{0}) through the diffusion process {\bm{x}}_{t}=\alpha(t){\bm{x}}_{0}+\sigma(t){\bm{\epsilon}},\,{\bm{\epsilon}}\sim{\mathcal{N}}({\bm{0}},{\bm{I}}),t\in[0,1] , where \alpha(\cdot),\sigma(\cdot) are functions of the timestep t , namely the noise schedules. Trained in this process, the model learns score-related information about p_{\mathrm{data}}({\bm{x}}_{0}) . A recent trend is flow matching [ 28 , 26 ] , which sets \alpha(t)=1-t and \sigma(t)=t , and parameterizes the model using velocity prediction, denoted by {\bm{v}}_{\theta} . The model is trained by minimizing \mathbb{E}_{{\bm{x}}_{0},{\bm{\epsilon}},t}[||{\bm{v}}_{\theta}({\bm{x}}_{t},t)-{\bm{v}}_{t}||^{2}] , where {\bm{v}}_{t}:=\frac{\mathrm{d}{\bm{x}}_{t}}{\mathrm{d}t}={\bm{\epsilon}}-{\bm{x}}_{0} . By using the optimal model {\bm{v}}_{\theta} to solve the probability flow ordinary differential equation (PF-ODE) [ 37 ] \mathrm{d}{\bm{x}}_{t}={\bm{v}}_{\theta}({\bm{x}}_{t},t)\mathrm{d}t,\,t:1\rightarrow 0 , one can generate samples {\bm{x}}_{0}\sim p_{\mathrm{data}}({\bm{x}}_{0}) . Owing to their remarkable expressive power, diffusion models have achieved