Free-Lunch Long Video Generation via Layer-Adaptive O.O.D Correction

Jiahao Tian, Chenxi SongWei Cheng, Chi Zhang[0.4em] Westlake University · 2026

Read Free-Lunch Long Video Generation via Layer-Adaptive O.O.D Correction by Jiahao Tian, Chenxi SongWei Cheng, Chi Zhang[0.4em] Westlake University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25209.txt.

Introduction

Tian and co-authors stretch short-clip video models to long films without any retraining. Quality drops on long runs for two reasons: frame positions and context length both leave the trained range. FreeLOC fixes each flaw. One step re-encodes time positions at several scales so they feel familiar to the model. Sparse attention then keeps local detail plus far links across time. A probe checks each layer's sensitivity, so fixes land only where needed. Tests beat other no-training methods on steadiness and looks.

Catalog record

CREATORJiahao Tian, Chenxi SongWei Cheng, Chi Zhang[0.4em] Westlake University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66175
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25209

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Free-Lunch Long Video Generation via Layer-Adaptive O.O.D Correction Jiahao Tian, Chenxi SongWei Cheng, Chi Zhang[0.4em] Westlake University Abstract Generating long videos using pre-trained video diffusion models, which are typically trained on short clips, presents a significant challenge. Directly applying these models for long-video inference often leads to a notable degradation in visual quality. This paper identifies that this issue primarily stems from two out-of-distribution (O.O.D) problems: frame-level relative position O.O.D and context-length O.O.D. To address these challenges, we propose FreeLOC , a novel training-free, layer-adaptive framework that introduces two core techniques: Video-based Relative Position Re-encoding (VRPR) for frame-level relative position O.O.D, a multi-granularity strategy that hierarchically re-encodes temporal relative positions to align with the model’s pre-trained distribution, and Tiered Sparse Attention (TSA) for context-length O.O.D, which preserves both local detail and long-range dependencies by structuring attention density across different temporal scales. Crucially, we introduce a layer-adaptive probing mechanism that identifies the sensitivity of each transformer layer to these O.O.D issues, allowing for the selective and efficient application of our methods. Extensive experiments demonstrate that our approach significantly outperforms existing training-free methods, achieving state-of-the-art results in both temporal consistency and visual quality. Code is available at https://github.com/Westlake-AGI-Lab/FreeLOC . † † footnotetext: * denotes project lead. \dagger denotes corresponding author. Emails: {tianjiahao, chizhang}@westlake.edu.cn 1 Introduction Video generation has been established as a core research problem in computer vision and generative modeling due to its utility in content creation, simulation, education, and interactive media. In recent years, extensive research has been conducted in this area, where video diffusion models 16 ; 37 ; 32 ; 34 ; 7 ; 4 ; 29 ; 8 ; 1 have made remarkable progress. Benefiting from training on a large collection of annotated video data, these models can generate high-quality videos that are nearly indistinguishable from reality. Despite impressive generation quality, most video diffusion models are designed for short clips. Directly generating longer videos causes significant degradation due to the discrepancy between training and inference lengths. While retraining models is computationally prohibitive, existing training-based autoregressive (AR) methods 30 ; 11 ; 3 ; 39 ; 10 ; 38 still struggle to match the quality of native short video generation, posing an open challenge. Recently, increasing attention has been directed toward training-free approaches that adapt an off-the-shelf short video diffusion model to a long video generator at inference time. A popular line of work 25 ; 2 ; 33 partitions the target video into multiple temporally overlapping clips using a sliding window, and processes these clips in an asynchronous or tiled fashion. While such approaches effectively maintain consistency within localized video segments, the inherent constraints of the sliding window paradigm restrict interactions between distant frames and consequently impair global temporal coherence. Other works 21 ; 22 ; 19 attempt to generate long videos in a single pass, and video quality is subsequently refined by manipulating latents to balance spatial detail against temporal consistency. However, these empirical designs are still suboptimal and may still produce artifacts such as identity drift, inconsistent lighting, or abrupt scene transitions. Moreover, most existing training-free pipelines are built upon UNet-based diffusion architectures, making them incompatible with state-of-the-art DiT-based models such as Wan 32 . In this paper, we focus on training-free long video generation using DiT-based diffusion models and address this challenge from three complementary perspectives. Firstly, when extending video diffusion transformers to generate sequences substantially longer than their pre-trained temporal window, the 3D relative positional encodings (RoPE), inevitably fall outside the distribution encountered during training. This frame-level relative position out-of-distribution (O.O.D) phenomenon causes the model to misinterpret temporal dependencies, leading to degraded quality. This issue mirrors the extrapolation problem in RoPE-based LLMs 9 ; 13 ; 5 . However, naive solutions from LLMs like clipping 9 or grouping 13 fail for video because they overlook the hierarchical nature of video’s temporal dependencies: nearby frames require fine-grained precision for local dynamics, whereas distant frames contribute to global coherence. To address this, we introduce Video-based Relative Position Re-encoding (VRPR) , a multi-granularity scheme that adaptively remaps out-of-range frame-level relative positions back into the model’s familiar domain with varying precision, preserving both short-range fidelity and long-range consistency. While VRPR alleviates the positional extrapolation issue, generating significantly longer videos can induce another O.O.D problem that we refer to as context-length O.O.D problem. As the temporal sequence grows, the expansion of attention tokens leads to overly diffused attention distributions and increased attention entropy, weakening the model’s ability to focus on informative local regions. Existing methods, such as fixed sliding-window attention 21 ; 25 ; 19 , effectively constrain the receptive field but fail to capture long-range temporal dependencies. To address this trade-off, following the application of VRPR, we introduce Tiered Sparse Attention (TSA) , a hierarchical attention mechanism that adaptively allocates attention density across temporal scales. TSA preserves dense attention within short temporal neighborhoods to maintain fine-grained visual details, while employing progressively narrower striped attention patterns for more distant ones to sustain global coherence. This tiered design ensures that the model allocates sufficient attention to both local motion cues and long-range dependencies without overwhelming the attention space. Although VRPR and TSA effectively alleviate frame-level relative position and context-length O.O.D issues respectively, applying them uniformly across all transformer layers neglects the heterogeneous roles and sensitivities of individual layers in video DiTs. To better understand this issue, we conduct a detailed empirical study to quantify each layer’s sensitivity to these O.O.D effects. Specifically, we develop an automatic probing mechanism that quantifies the sensitivity of each DiT’s self-attention layer to the two O.O.D sources. Building upon this, we introduce a training- free l ayer-adaptive O .O.D c orrection ( FreeLOC ) for long video generation, which selectively applies VRPR and TSA to the most responsive layers. For frame-level relative position sensitive layers, we utilize aforementioned VRPR to mitigate frame-level relative position O.O.D problems. For context-length sensitive layers, we employ TSA to constrain the context length following VRPR. This targeted adaptation ensures that positional and contextual corrections are applied precisely where they are most effective, enhancing long-video stability and generation quality. To validate the effectiveness of FreeLOC, we conduct extensive experiments. The results demonstrate that FreeLOC substantially outperforms existing training-free baselines in both visual fidelity and temporal consistency. In summary, this paper makes the following contributions: • We propose a Video-based Relative Position Re-encoding (VRPR) strategy that remaps frame-level relative positions into the pre-trained range, effectively mitigating frame-level relative-position O.O.D issues. • We introduce a Tiered Sparse Attention (TSA) mechanism that preserves the attention distribution under extended context lengths, maintaining a balance between local detail and global temporal coherence. • We design a layer-adaptive strategy named FreeLOC which guided by automatic layer sensitivity probing, enabling selective and efficient application of our methods for optimal performance. • Extensive experiments verify that FreeLOC achieves state-of-the-art results among training-free approaches for long video generation. 2 Related Work Video Diffusion Model. Recently, significant progress has been made in video diffusion models 16 ; 37 ; 32 ; 34 ; 7 ; 4 ; 29 ; 8 ; 1 ; 27 , allowing for the synthesis of high-resolution videos that maintain coherent visual and motion continuity over dozens of frames. Early approaches adapted pre-trained Unet-based text-to-image (T2I)

Read the complete text