FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model

Yukang Cao, Chenyang Si, Jinghao Wang, Ziwei Liu · 2025

Read FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model by Yukang Cao, Chenyang Si, Jinghao Wang, Ziwei Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01953.txt.

Introduction

Yukang Cao, Chenyang Si, Jinghao Wang, and Ziwei Liu presented FreeMorph in twenty twenty-five, a tuning-free method for morphing between two images. Morphing demands a smooth, faithful blend, but older methods fine-tune a diffusion model per pair and break when the two images differ in meaning or layout. FreeMorph needs no per-instance training.

Two ideas carry it. A guidance-aware spherical interpolation modifies self-attention using explicit guidance from the inputs, preventing identity loss along the sequence. A step-oriented variation trend blends attention patterns from each input across denoising steps for controlled, consistent transitions. In evaluations it beat prior methods on quality while running ten to fifty times faster. The authors present it as a new state of the art for generalized morphing.

Catalog record

CREATORYukang Cao, Chenyang Si, Jinghao Wang, Ziwei Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01953

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FreeMorph: Tuning-Free Generalized Image Morphing with Diffusion Model Yukang Cao, Chenyang Si, Jinghao Wang, Ziwei Liu Abstract We present FreeMorph , the first tuning-free method for image morphing that accommodates inputs with different semantics or layouts. Unlike existing methods that rely on fine-tuning pre-trained diffusion models and are limited by time constraints and semantic/layout discrepancies, FreeMorph delivers high-fidelity image morphing without requiring per-instance training. Despite their efficiency and potential, tuning-free methods face challenges in maintaining high-quality results due to the non-linear nature of the multi-step denoising process and biases inherited from the pre-trained diffusion model. In this paper, we introduce FreeMorph to address these challenges by integrating two key innovations. 1) We first propose a guidance-aware spherical interpolation design that incorporates explicit guidance from the input images by modifying the self-attention modules, thereby addressing identity loss and ensuring directional transitions throughout the generated sequence. 2) We further introduce a step-oriented variation trend that blends self-attention modules derived from each input image to achieve controlled and consistent transitions that respect both inputs. Our extensive evaluations demonstrate that FreeMorph outperforms existing methods, being 10 × ∼ 50 × 10\times\sim 50\times faster and establishing a new state-of-the-art for image morphing. Figure 1 : Examples of image morphing obtained via FreeMorph. Given two input images, FreeMorph effectively generates smooth transitions between them within 30 seconds. † † footnotetext: ∗ Equal contributions, ‡ Project lead, † Corresponding author. † † footnotetext: Jinghao was a Master student at NTU during this work. 1 Introduction Given two distinct input images, image morphing [ 51 , 25 ] aims to gradually change attributes such as shape, texture, and overall layout to produce a series of intermediate images that transition smoothly from one to the other. This process is widely used in fields such as animation, film, and photo editing [ 1 , 45 , 46 ] , offering an effective means of enhancing creative expression. Historically, image morphing relied on image warping [ 37 , 44 , 9 ] for aligning corresponding points and on color interpolation [ 2 , 21 ] for blending. These methods, however, often fall short when handling complex textural and semantic transitions, making them less effective for images with intricate details. With advancements in deep learning, Generative Adversarial Networks (GANs) [ 11 , 17 , 4 , 34 ] and Variational Autoencoders (VAEs) [ 20 ] have significantly improved image morphing by enabling latent code interpolation. Despite their capabilities, these approaches still face challenges with real-world images due to limited training data and information loss during GAN inversion. This underscores the need for methods that better preserve identity and offer greater generalization. Recently, with the availability of large-scale text-image datasets, vision-language models (e.g., Chameleon [ 40 ] ), diffusion models (e.g., Stable Diffusion [ 39 , 32 , 31 ] ), and transformers (e.g., PixArt- \alpha [ 6 ] , FLUX [ 3 ] ) have demonstrated impressive capabilities in generating high-quality images from text prompts. These advancements have paved the way for new generative image morphing techniques. Specifically, Wang and Golland 2023 leverages the local linearity of CLIP-based text embeddings to create smooth transitions by interpolating latent image features. Building on this idea, IMPUS [ 47 ] introduces a multi-phase training framework that includes optimizing text embeddings and training Low-Rank Adaptation (LoRA) modules to better capture semantics. While this method yields more visually appealing results, it requires extensive training, typically around 30 minutes per case. DiffMorpher [ 49 ] proposes to directly interpolate latent noise and leverage Adaptive Instance Normalization (AdaIN) to improve performance. However, these methods still struggle to process images with diverse semantics and intricate layouts, limiting their practical effectiveness. Given these issues, our objective is to achieve image morphing without requiring further tuning. Nonetheless, this goal introduces two key challenges: 1) Non-directional transitions and identity loss 1 1 1 Non-directional transitions, akin to identity loss, result in generated images that deviate from the identity of the input images. . While converting input images into latent features using a pre-trained diffusion model and then applying spherical interpolation might seem straightforward, this approach often results in inconsistent transitions. This is due to the non-linear nature of the multi-step denoising process. Additionally, this method inherits biases from the pre-trained model, which can lead to identity loss in the generated images. 2) Achieving consistent transitions 2 2 2 Inconsistent transitions are those with abrupt changes. . A diffusion model does not inherently provide an effective " variation trend " to capture the gradual changes between images. Consequently, achieving smooth and gradual transitions in a tuning-free manner remains a significant challenge without additional adjustments. In this paper, we present FreeMorph , a novel tuning-free method capable of instantly generating directional and realistic transitions between two images. Our method introduces two novel components: 1) Guidance-aware spherical interpolation: We first enhance the pre-trained diffusion model by incorporating explicit guidance from the input images through modifications to its self-attention modules. This is achieved through spherical interpolation, which produces intermediate features used in two key ways. First, we perform spherical feature aggregation to blend the key and value features of the self-attention modules, ensuring consistent transitions throughout the generated image sequence. Second, to address identity loss, we introduce a prior-driven self-attention mechanism that incorporates explicit guidance from the input images to preserve their unique identities. 2) Step-oriented variation trend : To achieve consistent transitions, we introduce a novel step-oriented variation trend . This method blends two self-attention modules, each derived from one of the input images, enabling a controlled and consistent transition that respects both inputs. To further improve the quality of the generated image sequences, we designed an improved reverse denoising and forward diffusion process that seamlessly integrates these innovative components into the original DDIM framework. As shown in Fig. 1 and Fig. 4 , our approach adeptly handles diverse input types, whether they have similar or distinct semantics and layouts, producing smooth and realistic transitions. To thoroughly assess FreeMorph and benchmark it against current methods, we also collect a new evaluation dataset that includes four distinct sets of image pairs, categorized by their semantic and layout similarity. Our extensive evaluations demonstrate that FreeMorph substantially outperforms existing approaches. FreeMorph produces high-fidelity image sequences with smooth and coherent transformations in under 30 seconds, making it \boldsymbol{50\times} faster than IMPUS [ 47 ] and \boldsymbol{10\times} faster than DiffMorpher [ 49 ] . 2 Related Work Text-to-Image Generation. Recently, diffusion models [ 31 , 28 , 32 , 30 ] have emerged as the de facto standard for text-to-image generation. These models employ a series of denoising steps (e.g., DDIM, DDPM) [ 15 , 38 ] to transform Gaussian noise into images, effectively capturing and interpreting details from textual prompts. Trained on billions of text-image pairs [ 35 ] , these models exhibit a remarkable ability to understand the distribution of real-world images, generating high-quality, diverse outputs while maintaining strong generalization capabilities. Our work harnesses the capabilities of diffusion models, particularly their ability to generate smooth transitions between two specified images [ 33 , 19 , 29 ] , to address the image morphing task. Image Morphing. Image morphing is a long-standing computer vision and graphics problem. Before the deep

Read the complete text