VideoMerge: Towards Training-free Long Video Generation
Siyang Zhang, Harry Yang, Ser-Nam Lim · 2025
Read VideoMerge: Towards Training-free Long Video Generation by Siyang Zhang, Harry Yang, Ser-Nam Lim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-09926.txt.
Introduction
Siyang Zhang, Harry Yang, and Ser-Nam Lim stretch short AI-made clips into long videos with no new training. Their 2025 paper, named VideoMerge, notes a core snag. Top video models learn from clips of set lengths, so asking for a longer clip at run time tends to fail or drift. Long clips also need steady flow across far-apart frames, which short clips do not demand.
Their method joins short clips from a trained model into one long piece. It keeps the model's range of motion and look while the user sets the new length. Side-by-side tricks guard smooth joins, steady scenes, and fresh change over time, which the authors say yields strong long clips.
Catalog record
| CREATOR | Siyang Zhang, Harry Yang, Ser-Nam Lim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-09926
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VideoMerge: Towards Training-free Long Video Generation Siyang Zhang, Harry Yang, Ser-Nam Lim Abstract Long video generation remains a challenging and compelling topic in computer vision. Diffusion‐based models, among the various approaches to video generation, have achieved state‐of‐the‐art quality with their iterative denoising procedures. However, the intrinsic complexity of the video domain renders the training of such diffusion models exceedingly expensive in terms of both data curation and computational resources. Moreover, these models typically operate on a fixed noise tensor that represents the video, resulting in predetermined spatial and temporal dimensions (i.e., resolution and length). Although several high-quality open-source pretrained video diffusion models, jointly trained on images and videos of varying lengths and resolutions, are available, it is generally not recommended to specify a video length at inference that was not included in the training set. Consequently, these models are not readily adaptable to the direct generation of longer videos by merely increasing the specified video length. In addition to feasibility challenges, long-video generation also encounters quality issues. The domain of long videos is inherently more complex than that of short videos: extended durations introduce greater variability and necessitate long-range temporal consistency, thereby increasing the overall difficulty of the task. To address these challenges, we propose VideoMerge, a training-free method that can be seamlessly adapted to merge short videos generated by pretrained text-to-video diffusion model. Our approach preserves the model’s original expressiveness and consistency while allowing for extended duration and dynamic variation as specified by the user. By leveraging the strengths of pretrained models, our method addresses challenges related to smoothness, consistency, and dynamic content through orthogonal strategies that operate collaboratively to achieve superior quality. Figure 1 : Comparison between our proposed VideoMerge and other state-of-the-art methods. Original prompt: “A woman is dancing in indoor garden.” Our method is able to preserve consistency in human identity in terms of face, clothes, hair style. We provide an extensive list of videos in the supplementary material that clearly demonstrates our superiority over current methods . 1 Introduction Recent advancements in video diffusion models have demonstrated remarkable performance in generating high-quality short videos by leveraging extensive training on meticulously curated image and video datasets [ 34 , 17 , 31 , 16 ] . However, extending these successes to long video generation remains a significant challenge due to two primary issues. First, even under default configurations, the inference cost of video diffusion models is substantial, and naively increasing the noise length for long videos results in extreme memory requirements that exceed typical GPU resource constraints. Second, the use of fixed-size noise tensors during training introduces a train-test discrepancy; forcefully imposing a larger video length can lead to a loss of high-frequency details [ 20 ] . Consequently, there is a consensus among training-free approaches for long video generation that the inference configuration should closely mirror that of training. Current training-free methods address long video generation primarily through two strategies: autoregressive generation and latent fusion. Autoregressive methods [ 11 , 2 ] generate long videos by using previously synthesized results as guidance for subsequent frames. However, this approach tends to accumulate errors by iteratively using synthesized frames, leading to notable quality degradation. In contrast, latent fusion approaches [ 23 , 20 , 4 , 15 ] avoid relying on synthesized data for guidance and thus circumvent error accumulation. Nevertheless, these methods often struggle to maintain video consistency, particularly in preserving the identities of humans or objects, because they perform denoising on independent noise tensors across different segments. Despite considerable efforts in prior research [ 23 , 20 , 4 , 15 ] , the issue of consistency in long video generation remains inadequately addressed. Although existing methods can achieve relatively smooth transitions over extended sequences, they still fail to preserve the global consistency of human or object identities (see Fig. 1 ). In this work, we introduce VideoMerge , a training-free method that adapts a pretrained text-to-(short)video model [ 16 ] to generate arbitrarily long videos while maintaining strong global consistency. Our approach addresses the consistency problem through three key innovations. First, rather than randomly initializing a long Gaussian noise which typically exhibits weak global consistency when denoised in short tiles, we reuse a short Gaussian noise of the default size configurated in the pretrained model, extending it to a longer noise tensor. We then blend this extended noise tensor with a new high-frequency component to preserve low-frequency semantic content while introducing dynamic motion details. Second, we propose a novel latent fusion strategy that merges the predicted noise tensors of different tiles, thereby enhancing the smoothness of transitions between denoising segments. Finally, we analyze the influence of text prompts, the sole conditioning input in the generation process, and demonstrate that fine-grained prompt refinement significantly improves the preservation of human or object identities. By integrating these three components, VideoMerge is capable of generating long videos that exhibit both smooth dynamic transitions and strong identity consistency. Our main contributions are summarized as follows: 1. We introduce a novel latent fusion method that effectively merges neighboring denoising tiles to achieve smooth transitions. 2. We propose a long noise initialization technique that shuffles and replicates a short Gaussian noise, subsequently augmenting it with the high-frequency component of a new noise, thereby ensuring both global consistency and dynamic variability. 3. We analyze the impact of fine-grained prompts and present a prompt refining strategy that further addresses the challenge of preserving human identity in generated videos. 2 Related Works Generative models in computer vision have evolved from GANs [ 9 ] to diffusion models [ 12 ] . One of the most widely studied problems is Text-to-Image (T2I), popularized by the success of diffusion models in T2I. One of the most effective diffusion models leverages a UNet [ 26 ] structure to model the reverse process of adding Gaussian noise to clear latents. Based on these well developed T2I models’ capability of understanding and handling 2D image knowledge, further extension to Text-to-Video (T2V) unleashes the possibility for creative works in the more complex video domain. 2.1 Video Diffusion Models Early works in video diffusion such as StableVideoDiffusion [ 2 ] , VideoCrafter [ 6 ] , and LaVie [ 30 ] ) typically extended a pretrained 2D UNet, originally developed for images, by incorporating additional modules to handle temporal attention. These models leverage a 2D+1D (i.e., pseudo-3D) attention mechanism within a 2D latent space derived from a pretrained 2D VAE, thereby obviating the need for temporal compression in the VAE. Subsequent video diffusion models (OpenSora [ 34 ] , Open-Sora-Plan [ 17 ] , CogVideoX [ 31 ] , Allegro [ 35 ] , and Hunyuan [ 16 ] ) have predominantly adopted Diffusion Transformers (DiT) [ 22 ] as denoisers. These models utilize full spatio-temporal attention across all dimensions and perform denoising within the latent space of a pretrained 3D-VAE, which more effectively captures features directly from video data. DiT-based diffusion models have established a robust foundation and baseline for text-to-video tasks, achieving state-of-the-art performance. 2.2 Long Video Generation Simply adapting the training methodology [ 13 ] used for short video generation models to long video models introduces several challenges, including the scarcity of high-quality long video datasets and limited