LoViC: Efficient Long Video Generation with Context Compression
Jiaxiu Jiang, Wenbo Li, Jingjing Ren, Yuping Qiu, Yong Guo, Xiaogang Xu, Han Wu, Wangmeng Zuo · 2025
Read LoViC: Efficient Long Video Generation with Context Compression by Jiaxiu Jiang, Wenbo Li, Jingjing Ren, Yuping Qiu, Yong Guo, Xiaogang Xu, Han Wu, Wangmeng Zuo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-12952.txt.
Introduction
Jiaxiu Jiang and co-authors tackle the cost of generating long videos with artificial intelligence. Modern video generators build every frame by attending to every other, so the computing bill grows quadratically with length, and earlier shortcuts tended to break the flow between scenes. Their framework, LoViC, instead generates a long video segment by segment, each new piece conditioned on a compressed memory of what came before.
The compressor, FlexFormer, squeezes video and text together into a compact code whose size can be dialled up or down. A single query token design, borrowed from question-answering architectures, makes that flexibility possible. Because the code also carries timing information, the same model can continue a video, fill a gap, work backwards, or stitch multiple shots. Trained on millions of open videos, the system is presented as a versatile engine for long-form generation.
Catalog record
| CREATOR | Jiaxiu Jiang, Wenbo Li, Jingjing Ren, Yuping Qiu, Yong Guo, Xiaogang Xu, Han Wu, Wangmeng Zuo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-12952
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LoViC: Efficient Long Video Generation with Context Compression Jiaxiu Jiang, Wenbo Li, Jingjing Ren, Yuping Qiu, Yong Guo, Xiaogang Xu, Han Wu, Wangmeng Zuo Abstract Despite recent advances in diffusion transformers (DiTs) for text-to-video generation, scaling to long-duration content remains challenging due to the quadratic complexity of self-attention. While prior efforts—such as sparse attention and temporally autoregressive models—offer partial relief, they often compromise temporal coherence or scalability. We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos, designed to produce long, coherent videos through a segment-wise generation process. At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations. It supports variable-length inputs with linearly adjustable compression rates, enabled by a single query token design based on the Q-Former architecture. Additionally, by encoding temporal context through position-aware mechanisms, our model seamlessly supports prediction, retradiction, interpolation, and multi-shot generation within a unified paradigm. Extensive experiments across diverse tasks validate the effectiveness and versatility of our approach. Figure 1: Videos generated by our model. Our model has the flexibility to do video continuation of any direction and generate multi-shot video with advanced efficiency. It shows the capability of retaining ID consistency within large temporal range, generating videos of large and smooth motion. Introduction Recent advances in text-to-video (T2V) generation have demonstrated impressive capabilities in synthesizing short video clips ( Polyak et al. 2024 ; Wang et al. 2025a ; Zheng et al. 2024b ; Ma et al. 2025 ; Liu et al. 2025 ; HaCohen et al. 2024 ; Lin et al. 2024 ; Kong et al. 2024 ; Yang et al. 2024 ; Zhou et al. 2024a ; Jin et al. 2024 ) , which leverage large-scale video-text datasets and scalable diffusion transformer (DiT) architectures ( Peebles and Xie 2023 ; Bao et al. 2023 ) . However, many real-world applications—such as first-person driving simulations, immersive 3D games, product promotional content, and user-generated storytelling—demand coherent long-form video generation. Naively extending DiT models to longer sequences is computationally prohibitive due to their quadratic self-attention cost and the inherent density of tokenized video representations, while alternative ad-hoc strategies often compromise temporal coherence and content consistency. Figure 2: Memory and time usage of single timestep DiT inference. With context compression, our method reduces memory usage and runtime, allowing more frames to be generated within the same resource constraints. A common strategy for long video generation is to synthesize short clips in a segment-by-segment manner, where the key challenge lies in efficiently scaling the context length—defined as the temporal span of the conditioning video that determines the model’s receptive field. Crucially, the appropriate context length is task-dependent and varies with the temporal complexity and dynamics of the scene. In relatively simple scenarios, such as natural scene videos with low temporal complexity, segments can often be generated independently and stitched together with minimal loss of coherence ( Zhu et al. 2023 ) . For moderately complex cases— e.g. , first-person driving videos or Minecraft gameplay streams—a block-autoregressive approach ( Voleti, Jolicoeur-Martineau, and Pal 2022 ; Harvey et al. 2022 ) can yield decent results, provided that transitions between blocks are smooth. In these cases, short-range context is typically sufficient to maintain temporal consistency. However, in more challenging settings with high visual and temporal complexity, maintaining content consistency often requires long-range temporal dependencies— i.e. , a context window spanning more than two segments ( Henschel et al. 2024 ) . To address this, several methods have attempted to extend the effective context. For instance, FDM ( Harvey et al. 2022 ) enhances block-wise autoregressive generation by sampling frames scattered across a broader temporal range in each step. NUWA-XL ( Yin et al. 2023 ) adopts a two-stage strategy by first generating sparse keyframes and then interpolating them in a coarse-to-fine manner. Despite these efforts, existing approaches have not demonstrated strong performance on open-domain, high-variability video generation tasks. These scenarios often demand substantially longer context windows per generation step to preserve both semantic and temporal consistency across frames. In this work, we propose a general framework, LoViC, for long-form video generation that accommodates a wide range of video settings. At each generation step, our model conditions on the full history of preceding video segments to synthesize the next segment. To mitigate the computational burden of self-attention over long temporal contexts, we introduce FlexFormer, a flexible Q-Former-based encoder that compresses context of arbitrary length—including both video and text tokens—under an adaptive compression ratio. FlexFormer employs a single learnable query token and a novel Interpolated RoPE (I-RoPE) positional encoding scheme to efficiently summarize long-range information. The resulting compressed context features are fed into a DiT-based decoder to generate the current video chunk. Our unified architecture supports a broad spectrum of generation tasks, including video prediction, retrodiction, interpolation, and multi-shot generation, by framing them under a single, coherent paradigm. We train our model on a million-scale open-domain video corpus and demonstrate its effectiveness and generality by extensive experiments. Our contributions are as follows: • We propose a novel framework, LoViC, that supports extended temporal contexts in DiTs-based video generation, enabling more coherent long-form video synthesis. • To efficiently encode long video-text context, we introduce FlexFormer, a flexible autoencoder that supports inputs of arbitrary shape with adaptive compression rates. FlexFormer employs a single-query token mechanism alongside a novel I-RoPE positional encoding scheme, which preserves spatial relationships among video tokens during compression. • Our framework supports both single-shot video extension and multi-shot video generation by introducing relative positional encoding to distinguish contextual segments from the current segment, enabling visually coherent results and realistic dynamics across diverse tasks. Related Work Video Diffusion Models. Earlier approaches ( Ho et al. 2022b ; Ho et al. 2022a ; Bar-Tal et al. 2024 ; Blattmann et al. 2023b ; Guo et al. 2023 ; Wang et al. 2023b ; Wang et al. 2025d ; Chen et al. 2023a ; Blattmann et al. 2023a ; Chen et al. 2024b ) adopt a U-Net ( Ronneberger, Fischer, and Brox 2015 ) architecture to model the diffusion process in the latent space of a VAE ( Rombach et al. 2022 ) for video generation. More recent methods ( Polyak et al. 2024 ; Wang et al. 2025a ; Zheng et al. 2024b ; Ma et al. 2025 ; Liu et al. 2025 ; HaCohen et al. 2024 ; Lin et al. 2024 ; Kong et al. 2024 ; Yang et al. 2024 ; Zhou et al. 2024a ; Jin et al. 2024 ) replace the U-Net with transformer ( Peebles and Xie 2023 ; Bao et al. 2023 ) to improve scalability. Figure 3: Model architecture. The left part of the figure features an autoencoder consisting of a FlexFormer encoder and a FlexFormer decoder. The encoder compresses multiple segments of video and text tokens separately. The number of query tokens is derived from the video token sequence length. The query token sequence is formed by copying the single learnable token multiple times. The decoder decode the context tokens into video and text features in a similar way. Each context video-text pair is compressed into some context tokens. The multiple chunks of context tokens are concatenated then fed into the