Imagine for Me: Creative Conceptual Blending of Real Images and Text via Blended Attention

Wonwoong ChoYanxia Zhang, Yan-Ying Chen, David I. Inouye · 2025

Read Imagine for Me: Creative Conceptual Blending of Real Images and Text via Blended Attention by Wonwoong ChoYanxia Zhang, Yan-Ying Chen, David I. Inouye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-24085.txt.

Introduction

Wonwoong Cho, Yanxia Zhang, Yan-Ying Chen, and David Inouye automate the blending of ideas into new images in a 2025 study. People merge concepts from a photo and a phrase into a fresh visual idea, a spark for design work. Human blending, though, sticks near familiar combinations, a bias known as design fixation.

Their adapter for text-to-image diffusion, called IT-Blender, encodes a real image without losing its details and mixes it with text through blended attention. Automating the mix helps designers escape local ruts and sample stranger combinations. The tool is framed as a creativity partner, widening the space a human designer can search.

Catalog record

CREATORWonwoong ChoYanxia Zhang, Yan-Ying Chen, David I. Inouye
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-24085

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Imagine for Me: Creative Conceptual Blending of Real Images and Text via Blended Attention Wonwoong ChoYanxia Zhang, Yan-Ying Chen, David I. Inouye Abstract Blending visual and textual concepts into a new visual concept is a unique and powerful trait of human beings that can fuel creativity. However, in practice, cross-modal conceptual blending for humans is prone to cognitive biases, like design fixation, which leads to local minima in the design space. In this paper, we propose a T2I diffusion adapter “IT-Blender” that can automate the blending process to enhance human creativity. Prior works related to cross-modal conceptual blending are limited in encoding a real image without loss of details or in disentangling the image and text inputs. To address these gaps, IT-Blender leverages pretrained diffusion models (SD and FLUX) to blend the latent representations of a clean reference image with those of the noisy generated image. Combined with our novel blended attention, IT-Blender encodes the real reference image without loss of details and blends the visual concept with the object specified by the text in a disentangled way. Our experiment results show that IT-Blender outperforms the baselines by a large margin in blending visual and textual concepts, shedding light on the new application of image generative models to augment human creativity. Our project website is: https://imagineforme.github.io/ . Figure 1 : Visual and textual conceptual blending results of IT-Blender based on FLUX.1-dev. 1 Introduction “Conceptual integration is at the heart of imagination” — Fauconnier and Turner 2008 Conceptual integration/blending ( Fauconnier and Turner 1998 ; Fauconnier and Turner 2008 ; Coulson 2001 ) is a theory in Cognitive Science, which can describe the human’s cognitive process combining a visual and textual concepts into a new idea. It is one of the most essential virtues in the creative industries (e.g., product design, character design, fashion design, interior design, graphic design, art, and advertisement) because conceptual blending can provide inspirational and creative design ideas by creating new combinations or reinventing existing ones ( Gabora 2002 ) . Prior works Yang 2009 ; Hyun and Lee 2018 ; Cai et al. 2023 have shown that exploring the design concepts and space as much as possible can produce better design results especially during the early phase of the design process (e.g., Conceptual Design ( Otto 2003 ) and the SCAMPER method ( Eberle 1996 ) in Concept Generation ( Ulrich and Eppinger 2016 ) ). However, there can be two challenges to perform cross-modal visual and textual conceptual blending in practice. First, human’s creativity easily got stuck in the suboptimal as shown in design fixation (i.e., a tendency of a designer to overly adhere to a limited set of solutions) ( Jansson and Smith 1991 ) and Einstellung effect (i.e., a cognitive bias from past experiences or familiar solutions to a problem, preventing them from exploring better alternatives) ( Luchins 1942 ) . Second, cross-modal conceptual blending itself is not a trivial task. It can be achieved by selective projection process determining what and where to integrate the given multiple concepts ( Fauconnier and Turner 1998 ) . It involves the laborious process for identifying features in each condition and comparing the semantic correspondence to find a way to meaningfully blend them together. Recent significant advances of text-to-image (T2I) diffusion models ( Rombach et al. 2022 ; Saharia et al. 2022 ) and their applications for adding an image condition led us to the question, “Can pretrained diffusion models be used for cross-modal conceptual blending to augment creativity?” If so, it can be very useful by 1) providing numerous conceptual blending results to explore broader design possibilities and 2) automating the conceptual blending process to minimize the time required to manually illustrate all design ideas. For example, suppose that we want to come up with a creative product design for sneakers. Instead of struggling with imagining what to combine with and how to apply the selective projection, we can simply give a prompt like “a photo of sneakers, creative design.” and give a set of reference images with a target concept and appearance, e.g., a sport car image for “sleek” or any knitted items for “warm” and “cozy” (e.g., Fig. 1 ). We may also apply the same style to the multiple objects (e.g., bicycle and car) or add multiple visual concepts to the generated results. Even random reference images can be used to provide a serendipitous inspiration. The question is how to perform selective projection in diffusion models, which must be done to achieve cross-modal conceptual blending. We think the key is the attention module ( Vaswani et al. 2017 ) (which is one of the most crucial components of modern diffusion models ( Rombach et al. 2022 ; Black Forest Labs 2024 ) ) because its mechanism, comparing similarity and selectively applying the value, is conceptually close to the selective projection. Earlier work, such as IP-Adapter ( Ye et al. 2023 ) and BLIP-Diffusion ( Li et al. 2023 ) proposed encoder-based methods to incorporate a reference image into text-guided generation with additional training. Although they show decent performance in blending visual and textual concepts with a fast inference time, their methods are limited in 1) disentangling textual and visual conditions and 2) preserving the detailed visual concept of the reference image due to the dependencies on the text cross-attention module and an external image encoder. Meanwhile, RIVAL ( Zhang et al. 2023a ) and StyleAligned ( Hertz et al. 2024 ) have shown the potential of the pretrained self-attention module of the T2I diffusion models in blending cross-modal concepts. Although they showed impressive performance in disentangling cross-modal concepts and applying detailed visual concepts from their own denoising chain to another, their performance is limited when a real reference image is conditioned due to the distribution shift of the inversion chain ( Zhang et al. 2023a ) . They also have a slower inference time than the encoder-based methods. Filling the gap between both baseline approaches, we propose a novel image adapter “Image-and-Text Concept Blender” (IT-Blender) that can imagine for us by blending cross-modal concepts with fast inference time. IT-Blender learns to blend visual concepts from a real image without loss of details, in a disentangled manner from the textual concept (i.e., text determines semantics while a reference image determines visual concepts such as texture, material, color, and local shape). Briefly, instead of using an external image encoder, we leverage the denoising network as an image encoder to maintain the details of visual concepts. As opposed to recent related literature without an external image encoder ( Wu et al. 2025 ; Tan et al. 2024 ) , our proposed method does not have any architectural dependency (i.e., applicable to both UNet-based ( Rombach et al. 2022 ) and DiT-based diffusion models ( Black Forest Labs 2024 ) ). We design a novel Blended Attention on top of the self-attention module, where detailed visual concepts can be preserved, and textual concepts are physically separated, encouraging disentanglement of textual and visual concepts. Blended Attention is trained to be specialized in finding a semantic correspondence between two latents; one from the real reference image and the other from the generated image. Our baseline experiment results on

Read the complete text