FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation Conference: ; ;
Zebin Yao 1 , Lei Ren 2 , Huixing Jiang 2 , Chen Wei 2 , Xiaojie Wang 1 , Ruifan Li 1 , Fangxiang Feng 1 · 2025
Read FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation Conference: ; ; by Zebin Yao 1 , Lei Ren 2 , Huixing Jiang 2 , Chen Wei 2 , Xiaojie Wang 1 , Ruifan Li 1 , Fangxiang Feng 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15958.txt.
Introduction
Yao and six co-authors present FreeGraftor, a method to place a subject from a photo into new AI images with no training, in a paper from twenty twenty-five. Subject-driven generation must keep identity true while following text prompts. Tuning methods are slow and costly, while instant methods often lose the subject. FreeGraftor grafts features across images to bridge that gap.
Semantic matching finds matching parts between reference and draft images. A position-aware fusion step then blends those details in. A new noise start preserves the subject's shape for better matching. Tests show higher fidelity and better text fit than rival instant methods. The method extends to scenes with several subjects at once.
Catalog record
| CREATOR | Zebin Yao 1 , Lei Ren 2 , Huixing Jiang 2 , Chen Wei 2 , Xiaojie Wang 1 , Ruifan Li 1 , Fangxiang Feng 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15958
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FreeGraftor: Training-Free Cross-Image Feature Grafting for Subject-Driven Text-to-Image Generation Conference: ; ; Zebin Yao 1 , Lei Ren 2 , Huixing Jiang 2 , Chen Wei 2 , Xiaojie Wang 1 , Ruifan Li 1 , Fangxiang Feng 1 Abstract . Subject-driven image generation aims to synthesize novel scenes that faithfully preserve subject identity from reference images while adhering to textual guidance, yet existing methods struggle with a critical trade-off between fidelity and efficiency. Tuning-based approaches rely on time-consuming and resource-intensive subject-specific optimization, while zero-shot methods fail to maintain adequate subject consistency. In this work, we propose FreeGraftor, a training-free framework that addresses these limitations through cross-image feature grafting. Specifically, FreeGraftor employs semantic matching and position-constrained attention fusion to transfer visual details from reference subjects to the generated image. Additionally, our framework incorporates a novel noise initialization strategy to preserve geometry priors of reference subjects for robust feature matching. Extensive qualitative and quantitative experiments demonstrate that our method enables precise subject identity transfer while maintaining text-aligned scene synthesis. Without requiring model fine-tuning or additional training, FreeGraftor significantly outperforms existing zero-shot and training-free approaches in both subject fidelity and text alignment. Furthermore, our framework can seamlessly extend to multi-subject generation, making it practical for real-world deployment. Our code is available at https://github.com/Nihukat/FreeGraftor . Figure 1. Subject-driven generation results of our FreeGraftor, achieved without any training or tuning process. These results demonstrate the superior properties of FreeGraftor, including (a) pixel-level detail preservation, (b) flexible text-guided control, and (c) support for multiple reference subjects. 1. Introduction The democratization of text-to-image generation has ignited widespread interest in personalized content creation, where users seek to synthesize novel scenes containing specific visual concepts—be it a cherished pet, a unique artifact, or a fictional character. While diffusion models ( Nichol et al. 2021 ; Saharia et al. 2022 ; Ramesh et al. 2022 ; Rombach et al. 2022 ; Podell et al. 2023 ) excel at generating diverse imagery from textual prompts, adapting them to preserve subject identity without costly per-subject optimization remains a critical challenge. Current solutions for subject-driven generation ( Ruiz et al. 2023 ; Gal et al. 2022 ; Kumari et al. 2023 ; Li et al. 2023 ; Ye et al. 2023 ) face a critical dilemma between fidelity and efficiency. Tuning-based methods , such as DreamBooth ( Ruiz et al. 2023 ) and Textual Inversion ( Gal et al. 2022 ) , capture subject identity by fine-tuning models or learning embeddings on user-provided subject datasets. These methods rely on subject-specific optimization to achieve high subject consistency, which are resource-intensive and time-consuming, prone to catastrophic forgetting, and struggle with multi-subject composition. Zero-shot methods , such as IP-Adapter ( Ye et al. 2023 ) and BLIP-Diffusion ( Li et al. 2023 ) , extract subject identities from reference images via additional trainable modules (e.g., image encoders or adapters) and integrate them into the text-to-image generation process. While these methods directly generate images containing reference subjects, they often fail to retain fine visual details, especially for novel concepts, as the limited parameter capacity of auxiliary modules fundamentally restricts their ability to reconstruct high-fidelity visual details. It is worth noting that pre-trained text-to-image models inherently possess powerful visual representation capabilities, as evidenced by their ability to synthesize intricate textures and structures from textual prompts. However, current approaches fail to fully exploit this potential. Tuning-based methods modify the image generation distribution through fine-tuning, while zero-shot approaches offload visual representation learning to auxiliary modules. In contrast, directly tapping into the base model’s own feature space for cross-image knowledge transfer offers a promising way to bridge the gap between fidelity and efficiency. Recent advances in appearance transfer ( Epstein et al. 2023 ; Alaluf et al. 2024 ; Mou et al. 2023 ; Mou et al. 2024 ; Go et al. 2024 ) lend credence to this idea, demonstrating that diffusion models can transplant visual attributes such as style and texture between images through semantic feature matching. For instance, Eye-for-an-eye ( Go et al. 2024 ) successfully transfers localized appearances by establishing cross-image correspondences in U-Net’s feature space. By computing pixel-wise similarity (detailed in Sec. 3.2), we demonstrate that such cross-image semantic correspondences can also be identified in the feature space of modern MM-DiT-based ( Esser et al. 2024 ) models (e.g., FLUX.1 ( Labs 2024 ) ). As illustrated in Figure 2 , two pixels that are similar in FLUX.1’s feature space are also semantically corresponding. Figure 2. Cross-image semantic correspondence in FLUX.1. For each specified pixel in the reference image, we identify the most similar pixel in the target image within FLUX.1‘s feature space. These corresponding pixel pairs are then visually connected by colored lines. Building on this insight, we propose FreeGraftor, a training-free framework that resolves the fidelity-efficiency dilemma in subject-driven generation via cross-image feature grafting. Our approach extracts visual features from reference images using rectified flow inversion ( Rout et al. 2024 ; Wang et al. 2024b ; Deng et al. 2024 ) and injects them into the generation process. Specifically, we introduce a novel cross-image feature grafting mechanism that transfers appearance details through semantic matching and position-constrained attention fusion in the feature space of text-to-image diffusion models. To bind reference patches to their corresponding generated patches, we implement a position embedding replication strategy, enabling semantically matched patches to share positional information during attention computation. Furthermore, we identify that proper noise initialization is crucial for robust semantic matching and structural preservation of reference subjects. Thus, we propose a structure-consistent initialization strategy: first constructing a collage containing the reference subject, then inverting it as the initialization noise for generation. Our method achieves pixel-level detail preservation (e.g., text, patterns) with fast inference and low computational overhead, requiring no additional training or optimization. Extensive experiments demonstrate that our method outperforms existing approaches in both subject consistency and text alignment while maintaining competitive efficiency. When combining multiple subjects, FreeGraftor preserves individual identities and harmoniously integrates all subjects into a coherent scene without layout conflicts or attribute confusion. Our contributions are summarized as follows: • We introduce FreeGraftor, a plug-and-play framework for subject-driven image generation that achieves pixel-level detail preservation and flexible text-guided control without fine-tuning or training. • We develop a cross-image feature grafting technique that transfers visual characteristics of reference subjects to semantically corresponding regions in generated images via semantic matching and position-constrained attention fusion. • We propose a novel noise initialization strategy that preserves reference geometry through automated collage construction and rectified flow inversion, seamlessly extending our method to multi-subject scenarios. 2. Related Work 2.1. Multimodal-Diffusion Transformer Text-to-image diffusion models have achieved remarkable success in generating high-quality and diverse images, exemplified by DALL-E 2 ( Ramesh et al. 2022 ) , Imagen ( Saharia et al. 2022 ) , GLIDE ( Nichol et al. 2021 ) , and eDiff-I ( Balaji et al. 2022 ) . Early popular open-source models, such as Stable Diffusion ( Rombach et al. 2022 ) and SDXL ( Podell et al. 2023 ) , are based on U-Net ( Ronneberger et al. 2015 ) architecture and utilize cross-attention between image and text to introduce textual conditions. In contrast, Diffusion Transformer (DiT) ( Peebles and Xie 2023 ) replaces the U-Net backbone with