TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models

Xin Jin [email protected], Yichuan Zhong [email protected], Yapeng Tian [email protected] · 2026

Read TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models by Xin Jin [email protected], Yichuan Zhong [email protected], Yapeng Tian [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08011.txt.

Introduction

Xin Jin, Yichuan Zhong, and Yapeng Tian blend a new object and a new style into images in one pass. Text-conditioned diffusion editors handle single object replacement well. They struggle when an object and a style must change together. Their system, TP-Blend, receives two separate prompts: one for the blend object, one for the target style. Both are injected into a single denoising trajectory. The 2026 preprint describes the training-free framework.

Twin-prompt attention pairing keeps the two prompts from interfering. Training-free means users can apply it to existing models without new data. Precise blending serves designers who need a specific object rendered in a specific look. Keeping object and style controls separate matches how people describe edits. The paper serves computer vision scholars in image editing. It adds fine control without added training cost.

Catalog record

CREATORXin Jin [email protected], Yichuan Zhong [email protected], Yapeng Tian [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-08011

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Models Xin Jin [email protected], Yichuan Zhong [email protected], Yapeng Tian [email protected] Abstract Current text-conditioned diffusion editors handle single object replacement well but struggle when a new object and a new style must be introduced simultaneously. We present Twin-Prompt Attention Blend (TP-Blend), a lightweight training-free framework that receives two separate textual prompts, one specifying a blend object and the other defining a target style, and injects both into a single denoising trajectory. TP-Blend is driven by two complementary attention processors. Cross-Attention Object Fusion (CAOF) first averages head-wise attention to locate spatial tokens that respond strongly to either prompt, then solves an entropy-regularised optimal transport problem that reassigns complete multi-head feature vectors to those positions. CAOF updates feature vectors at the full combined dimensionality of all heads (e.g., 640 dimensions in SD-XL), preserving rich cross-head correlations while keeping memory low. Self-Attention Style Fusion (SASF) injects style at every self-attention layer through Detail-Sensitive Instance Normalization. A lightweight one-dimensional Gaussian filter separates low- and high-frequency components; only the high-frequency residual is blended back, imprinting brush-stroke-level texture without disrupting global geometry. SASF further swaps the Key and Value matrices with those derived from the style prompt, enforcing context-aware texture modulation that remains independent of object fusion. Extensive experiments show that TP-Blend produces high-resolution, photo-realistic edits with precise control over both content and appearance, surpassing recent baselines in quantitative fidelity, perceptual quality, and inference speed. 1 Introduction Text-driven image editing with diffusion models 11 ; 12 ; 125 ; 107 ; 92 ; 136 ; 22 ; 2 ; 42 ; 128 ; 36 ; 87 has excelled at tasks like object replacement but still lacks a robust solution for object blending, where two objects must fuse seamlessly into a single coherent entity. Achieving such morphological transitions is challenging: the system must preserve each source object’s defining characteristics (e.g., color, shape, texture) while synthesizing intermediate attributes that accurately reflect the intended blend. This capability is especially valuable in creative design, film production, product prototyping, and scientific or educational visualization, where smooth transitions (e.g., morphing a car into a spaceship or combining organisms to study evolutionary traits) are often essential. Most style transfer methods still rely on reference images, limiting users to existing examples and requiring substantial effort 27 ; 167 ; 141 ; 171 ; 85 ; 100 ; 150 . By contrast, text-driven approaches 53 ; 194 ; 98 ; 159 specify styles in natural language (e.g., “sketch-like,” “art nouveau”) and could offer greater flexibility, yet they remain underexplored. Additionally, current style transfer techniques face major obstacles in achieving fine-grained, multi-scale, and region-specific control. They often fail to capture high-frequency textural details, losing subtle stylistic cues (e.g., brushstrokes, grain, intricate material features) even at high resolutions 27 ; 167 , thereby compromising overall texture fidelity. Motivated by these challenges, we propose Twin-Prompt Attention Blend (TP-Blend), a training-free framework that extends Classifier-Free Guided Text Editing (CFG-TE) 11 ; 12 ; 125 ; 107 ; 92 ; 136 ; 22 ; 2 ; 42 to support fine-grained object blending and style fusion through separate textual prompts, as illustrated in Figure 1 . TP-Blend introduces two new modules: Cross-Attention Object Fusion (CAOF), which integrates features from a blend object prompt using attention maps and an Optimal Transport framework; and Self-Attention Style Fusion (SASF), which injects style via Detail-Sensitive Instance Normalization (DSIN) and replaces self-attention Key/Value matrices with those from the style prompt. Unlike prior image-based approaches, TP-Blend enables direct textual control of both content and style, offering precise and independent modulation of blending strength and texture details. By unifying object replacement, blending, and style transfer within a single denoising process, TP-Blend enhances controllability without incurring additional computational overhead. Main Contributions. (1) Dual-Prompt Mechanism decouples object and style prompts, preventing interference and ensuring precise content representation and faithful style transfer within a unified denoising process; (2) CAOF with Optimal Transport aligns and integrates blend-object features into a replaced object by treating attention maps as distributions, enabling seamless morphological transitions and preserving semantic integrity; (3) SASF leverages DSIN to extract and transfer high-frequency style features, preserving intricate textural details without over-smoothing while allowing adaptive modulation of stylistic attributes across different spatial extents and granularities; (4) text-driven Key/Value substitution replaces self-attention Key/Value matrices with those derived from the style prompt, enforcing localized style modulation while maintaining spatial coherence and object fidelity. Code Availability. Code is available at https://github.com/felixxinjin1/TP-Blend . Original Image Object Replacement Object Blending Style Blending Figure 1 : Demonstration of our method’s capabilities. Row 1: Original object “ Knight ” is replaced by “ Leonardo DiCaprio ”, blended with “ Batman ”, and styled with “ Pop Art ”. Row 2: Original object “ Robot ” is replaced by “ Knight ”, blended with “ Thanos ”, and styled with “ Cyberpunk Style ”. Row 3 : Original object “ Cat ” is replaced by “ Dog ”, blended with “ Horse ”, and styled with “ Oil painting ”. Row 4 : Original object “ Chameleon ” is replaced by “ Dinosaur ”, blended with “ Fish ”, and styled with “ Oil painting ”. 2 Related Work Diffusion models have emerged as the dominant paradigm for text-guided image generation and editing, beginning with unconditional DDPMs and latent variants together with classifier-free guidance, which underpin early editing systems but leave open challenges in multi-concept disentanglement and precise regional control ( 37 ; 122 ; 54 ; 12 ; 11 ) . Unified generation and editing continues to broaden capability and instruction following, spanning single or dual diffusion formulations, universal dynamics-aware frameworks, adapter-based unification, and interactive or conversational editors that couple instruction tuning with in-context visual reasoning ( 96 ; 181 ; 182 ; 161 ; 198 ; 162 ; 164 ; 23 ; 38 ; 41 ; 89 ; 66 ; 77 ; 21 ; 170 ; 197 ; 88 ; 157 ; 67 ; 75 ; 104 ; 17 ; 20 ; 101 ) . Compositionality, multi-conditioning and regional control are advanced by balancing or transplanting representations, ordering and counting constraints, language-guided tokenization and unified tokenizers, as well as training-free or attention-repositioning guidance that improves spatial faithfulness and occlusion handling ( 102 ; 69 ; 28 ; 29 ; 90 ; 83 ; 10 ; 186 ; 55 ; 118 ; 50 ; 188 ; 144 ; 187 ; 120 ; 160 ) . Personalization and identity preservation leverage feature caching and standardize-then-personalize pipelines, continual concept learning with mitigation of forgetting and confusion, parameter-efficient or time-aware frequency guidance, consistency from a single prompt and localized attention, LoRA merging and hypernetworks, and targeted human preference optimization ( 1 ; 165 ; 46 ; 82 ; 95 ; 58 ; 97 ; 163 ; 124 ; 108 ; 200 ; 84 ; 145 ; 99 ) . Style control has moved from exemplar dependence toward text-driven and detail-faithful modulation through state-space modeling, selective style element control, object-centric style editing, causal intervention and conflict-free guidance, including emotional manipulation ( 93 ; 78 ; 113 ; 60 ; 70 ; 176 ; 14 ; 59 ; 34 ) . Efficiency and scheduling improve with learned time prediction, one-step distillation and time-independent encoders,

Read the complete text