Z-Magic: Z ero-shot M ultiple A ttributes G uided I mage C reator

Yingying Deng ∘, Xiangyu He, Fan Tang, Weiming Dong · 2025

Read Z-Magic: Z ero-shot M ultiple A ttributes G uided I mage C reator by Yingying Deng ∘, Xiangyu He, Fan Tang, Weiming Dong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12124.txt.

Introduction

Yingying Deng, Xiangyu He, Fan Tang, and Weiming Dong introduce Z-Magic in this 2025 paper. Their tool makes images with more than one chosen trait at once, such as style, color, or content. Past tools could add traits, but the traits did not always fit well with each other in the final image. The work also tackles the case where no training examples cover the trait mix.

The authors frame the task with chance and condition rules. Each new trait should follow from the traits set before it. That view keeps the traits in step and links the task to multi-task learning, which cuts the computing cost. In tests, Z-Magic beat prior models at this zero-shot task.

Catalog record

CREATORYingying Deng ∘, Xiangyu He, Fan Tang, Weiming Dong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12124

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Z-Magic: Z ero-shot M ultiple A ttributes G uided I mage C reator Yingying Deng ∘, Xiangyu He, Fan Tang, Weiming Dong Abstract The customization of multiple attributes has gained popularity with the rising demand for personalized content creation. Despite promising empirical results, the contextual coherence between different attributes has been largely overlooked. In this paper, we argue that subsequent attributes should follow the multivariable conditional distribution introduced by former attribute creation. In light of this, we reformulate multi-attribute creation from a conditional probability theory perspective and tackle the challenging zero-shot setting. By explicitly modeling the dependencies between attributes, we further enhance the coherence of generated images across diverse attribute combinations. Furthermore, we identify connections between multi-attribute customization and multi-task learning, effectively addressing the high computing cost encountered in multi-attribute synthesis. Extensive experiments demonstrate that Z-Magic outperforms existing models in zero-shot image generation, with broad implications for AI-driven design and creative applications. † † footnotetext: \circ These authors contributed equally to this work. † † footnotetext: \diamondsuit Corresponding author: Fan Tang. 1 Introduction Figure 1 : We visualize the cosine similarity between gradients introduced by two conditions (i.e., landmark g_{0} and face ID g_{1} ) during the diffusion process of face synthesis. In the vanilla setting, each condition is applied separately without considering contextual coherence, causing the gradient directions to be nearly orthogonal, as random vectors in high-dimensional space are often orthogonal. In contrast, our method adjusts each condition based on the preceding ones, resulting in an obtuse-angled optimization direction where later conditions refine the gradients of earlier ones. In other words, our approach has two degrees of freedom to construct the gradient direction: angle and length. With the rapid advancement of generative AI, particularly in the widespread adoption of image generation applications, the paradigm of controlled image creation has gained considerable attention from both industry and the image synthesis community [ 38 , 34 , 36 , 37 ] . In this context, guiding the image creation process through multimodal conditions, such as text prompts and reference images, becomes more popular. In reality, conditional dependence is a fundamental property in multi-attribute guided creation. For example, in fashion design, the selection of a color scheme is often influenced by the style or cut of the clothing, and vice versa; similarly, in face synthesis, the presence of gender implies facial features, and this contextual information should guide subsequent attribute choices. These examples intuitively illustrate that attributes do not exist in isolation but are mutually informative and conditionally dependent, forming a naturally coherent whole. While existing diffusion-based approaches have achieved impressive results in conditional generation [ 48 , 22 , 5 , 19 , 4 , 44 , 3 , 15 ] , the underlying structure that inherently links multiple attributes into a cohesive creation has been rarely discussed. Besides, for the sake of practical simplicity, previous methods often assume independence among different attributes, which limits their capacity to model inter-attribute relationships. In the score-based conditional diffusion model, conditioning is facilitated through a gradient-based approach, formulated as g_{\mathbf{c}}=\nabla_{\mathbf{x}_{t}}\log p(\mathbf{c}|\mathbf{x}_{t}) , where the gradient g_{\mathbf{c}} represents the conditioning mechanism. When each attribute is generated under the assumption of independence, the gradients associated with distinct attributes exhibit near-zero cosine similarity. As shown in Figure 1 (a), for any two conditional gradients, g_{\mathbf{c}_{0}} and g_{\mathbf{c}_{1}} , during the diffusion process, we observe that \mathbb{E}\left[\frac{g_{\mathbf{c}_{0}}^{T}g_{\mathbf{c}_{1}}}{\|g_{\mathbf{c}_{0}}\|\|g_{\mathbf{c}_{1}}\|}\right]\approx 0 . This behavior aligns with the properties of random high-dimensional vectors, which are typically nearly orthogonal in high-dimensional spaces † † In high-dimensional spaces, two random vectors are almost always nearly orthogonal [ 39 ] . . This result is somewhat surprising, as these conditions apply to the same image, and ideally, they should exhibit a degree of contextual correlation to support coherent and harmonious synthesis. In light of this, we present a novel approach that explicitly models conditional dependencies, ensuring each attribute is generated in the context of previously selected ones. Specifically, our method employs a conditional distribution framework, where each subsequent attribute is derived from a multivariable conditional distribution informed by the attributes generated earlier, aligning with the joint distribution p(\mathbf{c}_{0},\mathbf{c}_{1}|\mathbf{x}_{t}) . Each conditionally dependent gradient progressively refines image generation. Initially, we apply g_{0}=\nabla{\mathbf{x}_{t}}\log p(\mathbf{c}_{0}|\mathbf{x}_{t}) to capture the influence of the first attribute, followed by g_{1}=\nabla{\mathbf{x}_{t}}\log p(\mathbf{c}_{1}|\mathbf{c}_{0},\mathbf{x}_{t}) , which adjusts the generation to align with the contextual influence of the first attribute. As shown in Figure 1 (b), this sequential conditioning introduces a gradient refinement mechanism whereby g_{1} partially aligns with g_{0} and corrects the vector length along the g_{0} direction, enhancing attribute coherence. Further, while models can be fine-tuned for specific attribute combinations, training-based approach is resource-intensive and lacks scalability, particularly in the face of high-dimensional attribute spaces. Thus, exploring methods for zero-shot multi-attribute synthesis has the potential to unlock new avenues for practical AI-driven creative tools. In response to these challenges, we introduce Zero-shot Multiple Attributes Guided Image Creator (Z-Magic), an approach designed to address the complex requirements of multi-attribute customization. By reformulating multi-attribute synthesis as a problem of conditional dependency across attributes, Z-Magic leverages principles from conditional probability and multi-task learning to generate coherent images without extensive attribute-specific tuning. In our experiments, this method not only reduces human labeling overhead for attribute combinations but also paves the way for harmonious customization across diverse domains. 2 Related Works 2.1 Conditional Diffusion Model Recent advances in Denoising Diffusion Probabilistic Models (DDPMs) [ 18 ] have demonstrated exceptional capabilities in various image synthesis tasks, including image manipulation and conditional generation. Classifier-Guidance [ 13 ] introduces an additional classifier to guide the generation toward specific categories, while classifier-free guidance models [ 17 ] combine conditional and unconditional models to enhance generation quality. Several landmark architectures have further shaped this field. GLIDE [ 31 ] employs a pre-trained CLIP model [ 35 ] for text-guided image synthesis, and Stable Diffusion [ 38 ] improves computational efficiency by performing text-conditioned denoising in latent space. ControlNet [ 48 ] incorporates a parallel U-Net architecture to enable diverse visual conditions, including landmarks, edge maps, and skeletal structures. Similarly, T2I-Adapter [ 30 ] proposes a lightweight adapter network to integrate various visual control signals. Despite their impressive performance, these approaches demand substantial computational resources for training. In contrast, our work explores a training-free paradigm for controlling diffusion models, offering greater flexibility and broader applicability. 2.2 Multiple Attributes Customization Multi-attribute control in image generation—encompassing aspects such as style, structure, textual descriptions, and semantic segmentation—has become a critical research focus in computer vision, particularly in tasks like style transfer and facial image synthesis. StyleGAN [ 21 ] pioneered this field by introducing a style-based generator architecture that enables precise control over visual features at multiple scales, setting new standards in high-quality face image generation. Subsequent advances have further extended the capabilities of attribute-controlled generation. HairCLIP [ 45 ] built on the StyleGAN framework by integrating both textual and reference image conditioning. TediGAN [ 46 ] advanced semantic control by combining GAN inversion techniques with style-based generation. In diffusion models, recent research has shown significant progress: for instance, [ 25 ] achieved control over both identity and expression attributes, while [ 22 ] demonstrated the complementary strengths of diffusion models and GANs for text and visually guided

Read the complete text