InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO
Xueji Fang Liyuan Ma Zhiyang Chen Mingyuan Zhou Guo-jun Qi[1em] MAPLE Lab, Westlake University {fangxueji, maliyuan, chenzhiyang, zhoumingyuan}@[email protected] · 2025
Read InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO by Xueji Fang Liyuan Ma Zhiyang Chen Mingyuan Zhou Guo-jun Qi[1em] MAPLE Lab, Westlake University {fangxueji, maliyuan, chenzhiyang, zhoumingyuan}@[email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17574.txt.
Introduction
Xueji Fang, Liyuan Ma, Zhiyang Chen, Mingyuan Zhou, and Guo-jun Qi presented a framework for long video generation in two thousand twenty-five. Text-to-video models can make short scenes well, but long videos that cross scenes tend to drift. Keeping the full history also makes compute cost soar.
Their system, InfLVG, learns a policy that picks only the most relevant past context at each step. The policy is trained with a reinforcement method and a reward that mixes prompt match, cross-scene consistency, and fewer artifacts. Compute then stays fixed while content stays coherent. On a new cross-scene benchmark, the method stretched video length by up to nine times.
Catalog record
| CREATOR | Xueji Fang Liyuan Ma Zhiyang Chen Mingyuan Zhou Guo-jun Qi[1em] MAPLE Lab, Westlake University {fangxueji, maliyuan, chenzhiyang, zhoumingyuan}@[email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17574
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InfLVG: Reinforce Inference-Time Consistent Long Video Generation with GRPO Xueji Fang Liyuan Ma Zhiyang Chen Mingyuan Zhou Guo-jun Qi[1em] MAPLE Lab, Westlake University {fangxueji, maliyuan, chenzhiyang, zhoumingyuan}@[email protected] Abstract Recent advances in text-to-video generation, particularly with autoregressive models, have enabled the synthesis of high-quality videos depicting individual scenes. However, extending these models to generate long, cross-scene videos remains a significant challenge. As the context length grows during autoregressive decoding, computational costs rise sharply, and the model’s ability to maintain consistency and adhere to evolving textual prompts deteriorates. We introduce InfLVG, an inference-time framework that enables coherent long video generation without requiring additional long-form video data. InfLVG leverages a learnable context selection policy, optimized via Group Relative Policy Optimization (GRPO), to dynamically identify and retain the most semantically relevant context throughout the generation process. Instead of accumulating the entire generation history, the policy ranks and selects the top- K K most contextually relevant tokens, allowing the model to maintain a fixed computational budget while preserving content consistency and prompt alignment. To optimize the policy, we design a hybrid reward function that jointly captures semantic alignment, cross-scene consistency, and artifact reduction. To benchmark performance, we introduce the Cross-scene Video Benchmark (CsVBench) along with an Event Prompt Set (EPS) that simulates complex multi-scene transitions involving shared subjects and varied actions/backgrounds. Experimental results show that InfLVG can extend video length by up to 9×, achieving strong consistency and semantic fidelity across scenes. Our code is available at https://github.com/MAPLE-AIGC/InfLVG . 4 4 footnotetext: Work done at Westlake University. 3 3 footnotetext: Corresponding author. 1 Introduction Figure 1: Challenges in autoregressive long video generation across scenes. (Left) Baseline models tend to repeat initial scene elements (e.g., background, cup) due to unfiltered context accumulation, failing to follow new prompts. (Right) InfLVG addresses this by selectively preserving relevant context, achieving both better prompt alignment and content consistency – generating both face and environmental elements in accordance with the prompt and given context. Recent advances in diffusion models DDPMs ; DDIM ; FlowMatching and transformer-based architectures DiT ; SD3 ; DiT-MoE have significantly advanced video generation. While state-of-the-art approaches Sora ; opensoraplan ; CogVideoX ; WanX exhibit impressive visual fidelity and semantic alignment with textual prompts, their extension to longer videos remains constrained by the quadratic computational cost scaling with sequence length and the scarce availability of large-scale, high-quality training data for extended video sequences. Most current works based on bidirectional attention mechanisms CogVideoX ; WanX ; Mochi are trained on fixed-length video segments, inherently limiting their capacity for variable-length extrapolation. Autoregressive video generation NOVA ; PyFlow ; CausVid , while a seemingly straightforward approach for temporal extension through iterative next-frame prediction, suffers critical limitations in cross-scenario generation. As illustrated in Figure 1 , naively autoregressive extension through frame-by-frame denoising often fail to adapt to new prompt description, as the model remains anchored to initial scene semantics. The expanding context window accumulates increasingly irrelevant features that dominate the model’s attention, effectively suppressing its ability to focus on the specific context required for coherent scene generation. This necessitates an adaptive selection mechanism that selectively preserves content relevant to scene transitions while filtering irrelevant elements, thereby achieving an optimal balance between content continuity and prompt adherence. To address these issues, we propose InfLVG , an inference-time framework that adaptively adjusts the video context for each individual sample without requiring additional long-form training data. InfLVG introduces a context selection policy trained via GRPO, guided by hybrid rewards specifically designed to ensure content consistency and semantic alignment with the prompt. During autoregressive video generation, the policy assigns relevance scores to each context token, estimating their semantic contribution to the newly generated scene relative to preceding content. Based on these scores, the model performs top- K ranking to select the most relevant context features and uses them to progressively denoise subsequent frame sets. The reward function is carefully designed to jointly capture content consistency by preserving scene identity across transitions, ensure prompt alignment through accurate text-video semantic matching, and suppress artifacts by penalizing visual distortions in the generated frames. Through GRPO-based optimization, the policy is reinforced to prioritize context tokens that are critical for cross-scene coherence, while effectively filtering out irrelevant elements. Moreover, by enforcing a fixed-length context window during inference, our method maintains a bounded computational cost, even when generating long, multi-scene videos. As illustrated in Figure 2 , InfLVG enables flexible generation paradigms, including both single-scene extension and multi-scene transitions with contextual awareness. To evaluate cross-scene video generation performance, we propose CsVBench, a benchmark that constructs multi-scene descriptions featuring the same subject identity across varied actions and backgrounds. Our contributions are threefold. First, we propose an inference-time framework for generating multi-scene coherent long videos that effectively balances cross-scene consistency and alignment with dynamic textual prompts without any additional training on long-form video data. To support this, we design a GRPO-optimized context selection policy guided by hybrid rewards incorporating multi-faceted quality signals. This policy enables the model to identify and emphasize semantically relevant context tokens, thereby maintaining consistency between newly described scenes and previously generated content. Furthermore, by applying top- K ranking to select a fixed number of context features, our approach maintains a constant computational footprint while allowing the autoregressive model to scale to arbitrarily long video sequences. Experimental results show that our method achieves up to a 9\times increase in video length while preserving high generation quality. Figure 2: Different scene extension paradigms with InfLVG. (a) Single-scene extension, (b) Multi-scene transition with contextual awareness, and (c) Both single- and multi-scenes. 2 Related Work Video Generation Models. Recent years have witnessed substantial advances in video generation, driven by the development of diffusion and autoregressive generative modeling. A seminal breakthrough emerged through the scaling laws in the pretraining of Diffusion Transformer (DiT) architecture DiT ; DiT-MoE , where video generation quality exhibits remarkable improvements with increased model parameters and training data scale. However, conventional pretraining paradigms that condition on single text-video pairs Sora ; CogVideoX ; WanX ; Mochi inherently limit these models to generating videos within a single scene or narrative context. Parallel efforts to extend generation length without retraining attempt employ sliding window mechanism FreeNoise ; FIFO ; Ouroboros , enabling longer video sequences through local temporal context conditioning. However, these methods inherently depend on the quality of the pretrained backbone and often produce spatiotemporal inconsistencies in DiT-based frameworks, particularly when synthesizing dynamic scene transitions. Alternative approaches NOVA ; PyFlow ; CausVid integrate autoregressive modeling with diffusion processes, enabling theoretically unbounded generation lengths through iterative frame prediction. Nevertheless, their training protocols are typically optimized for single-scene generation and thus struggle to generalize to cross-scene narratives, while expanding training data to diverse scenarios incurs prohibitive computational costs. Our framework proposes a reinforcement learning framework built upon autoregressive video diffusion models, achieving consistent multi-scene generation and faithful adherence to new textual prompts without requiring additional training data. This is realized through adaptive history context selection and hybrid reward optimization. Aligning Visual Generation with Preference Feedback. Building upon the success of Reinforcement Learning from Human Feedback (RLHF) DeepSeek-r1 ; CoT-Survey-Hao-Fei ; JudgeLRM ; Sketch-of-Thought ; s1 in aligning language models with human preferences has inspired analogous approaches for diffusion-based generation. Pioneering works DDPO ; DPOK