HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning
Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, Jianbing Shen · 2025
Read HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning by Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, Jianbing Shen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19965.txt.
Introduction
Hongji Yang and six co-authors present HiCoGen, a system for images that must match complex, layered prompts. Diffusion models handle simple prompts well but often drop objects, mix up parts, or compose scenes poorly when prompts have many objects and nested structure. The authors replace one-shot generation with a step-by-step build.
A language model first splits a complex prompt into minimal semantic units. HiCoGen then synthesizes the units one by one, each new image giving visual context for the next step. Reinforcement learning tunes the process with rewards judged at global, subject, and relationship levels, and a decaying randomness schedule widens exploration early on. A new benchmark, HiCoPrompt, tests the method on layered prompts.
Catalog record
| CREATOR | Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, Jianbing Shen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-19965
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning Hongji Yang, Yucheng Zhou, Wencheng Han, Runzhou Tao, Zhongying Qiu, Jianfei Yang, Jianbing Shen Abstract Recent advances in diffusion models have demonstrated impressive capability in generating high-quality images for simple prompts. However, when confronted with complex prompts involving multiple objects and hierarchical structures, existing models struggle to accurately follow instructions, leading to issues such as concept omission, confusion, and poor compositionality. To address these limitations, we propose a Hierarchical Compositional Generative framework ( HiCoGen ) built upon a novel Chain of Synthesis (CoS) paradigm. Instead of monolithic generation, HiCoGen first leverages a Large Language Model (LLM) to decompose complex prompts into minimal semantic units. It then synthesizes these units iteratively, where the image generated in each step provides crucial visual context for the next, ensuring all textual concepts are faithfully constructed into the final scene. To further optimize this process, we introduce a reinforcement learning (RL) framework. Crucially, we identify that the limited exploration of standard diffusion samplers hinders effective RL. We theoretically prove that sample diversity is maximized by concentrating stochasticity in the early generation stages and, based on this insight, propose a novel Decaying Stochasticity Schedule to enhance exploration. Our RL algorithm is then guided by a hierarchical reward mechanism that jointly evaluates the image at the global, subject, and relationship levels. We also construct HiCoPrompt , a new text-to-image benchmark with hierarchical prompts for rigorous evaluation. Experiments show our approach significantly outperforms existing methods in both concept coverage and compositional accuracy. 1 Introduction Figure 1 : The motivation of HiCoGen. The semantic gap between text and images widens as the complexity of the text increases, particularly involving the prompts with a hierarchical relationship. While a single T2I model performs well in generating individual objects, it suffers from concept missing and confusion when processing complex prompts. HiCoGen employs a Chain of Synthesis for complex text to preserve the semantic content. Diffusion models 10 have led to significant improvements in text-to-image (T2I) generation. Many T2I models 33 ; 25 ; 31 ; 15 ; 16 ; 5 have been proven to be able to produce high-quality and diverse images. However, existing image diffusion models fail to generate images that cover all concepts within long and complex prompts. Despite being equipped with more powerful text encoders (CLIP 28 , T5-Encoder 30 ), the challenges of image generation with complex prompts remain unresolved. Specifically, complex prompts contain extensive information, resulting in a large semantic gap between text and image domains, as shown in Fig. 1 . This leads to several issues: ① concept missing , where the model overlooks parts of the prompt due to overly long textual descriptions; ② concept confusion , as the text encoder fails to capture hierarchical relationships, causing the model to rely on priors and generate incorrect interactions between objects; ③ overall quality decline , where the visual fidelity of the generated image degrades. In stark contrast, Large Language Models (LLMs) demonstrate powerful inference-time reasoning by scaling not only the amount of computation, but the reasoning process itself: techniques such as Chain-of-Thought 38 and reinforcement-style inference frameworks (e.g., Deepseek-R1 8 ) explicitly decompose a complex problem into a sequence of intermediate steps. This form of content-level inference-time scaling stands apart from recent work on diffusion inference-time scaling 20 , which allocates additional compute to the decoding process (e.g., exploring more noises or sampling trajectories for a fixed holistic prompt) while keeping the textual input and its semantics unchanged. Inspired by the former line of work, we ask a different question: “ Instead of only scaling the sampling procedure of diffusion models, can we introduce a similar step-by-step, content-aware processing paradigm into visual synthesis itself to overcome the limitations of holistic generation? ” To realize this principle in visual synthesis, we propose Chain of Synthesis (CoS) , a new paradigm that replaces monolithic generation with a progressive, part-by-part construction process. Building on this paradigm, our framework, HiCoGen , is the first system to realize this paradigm, engineered specifically for prompts with complex hierarchical and relational structures (e.g., “ one person holding an object standing next to another person who is also holding an object ”). Within HiCoGen, we first leverage an LLM to decompose the complex prompt into a set of minimal semantic units and rewrite the prompt for generation. The framework then synthesizes these units iteratively: the image generated in each step becomes the critical visual context for the next. This process constructs a visual compositional pathway, which progressively replaces entangled textual descriptions with concrete visual information, ensuring a faithful construction of the final complex scene. Moreover, to optimize the fidelity of the CoS pipeline, we introduce a reinforcement learning (RL) framework based on GRPO 8 . However, effectively applying RL to diffusion models is a major challenge, as standard sampling methods offer a limited exploration space, hindering the discovery of optimal generation policies. To overcome this fundamental barrier, we first conduct a rigorous theoretical analysis of the stochastic sampling dynamics. We prove that sample diversity is critically dependent on early-stage stochasticity. Based on this insight, we introduce a Decaying Stochasticity Schedule , a novel sampling mechanism that provably maximizes exploration and is thus essential for enabling effective RL. Building on this enhanced sampler, our RL algorithm is guided by a novel hierarchical reward mechanism , which supplies global-, subject-, and relationship-level rewards for step-by-step supervision. Finally, to facilitate rigorous and standardized evaluation of compositional reasoning in T2I models, a capability central to our work, we construct HiCoPrompt . It is a new, systematically generated benchmark featuring prompts with clearly defined hierarchical and compositional structures. Extensive experiments on HiCoPrompt validate that our HiCoGen significantly outperforms existing models. Our contribution can be summarized as follows: • We introduce a novel Chain of Synthesis (CoS) paradigm for compositional generation, and propose HiCoGen, the first framework to realize this step-by-step process for T2I models, overcoming limitations of monolithic methods. • Combined with our enhanced sampler and a GRPO-based algorithm, we design a novel hierarchical reward mechanism that provides fine-grained supervision on compositional aspects. • We identify the limited exploration of standard diffusion samplers as a key barrier to RL-based optimization. We prove that optimal sample diversity is achieved by concentrating stochasticity in the early generation stages and introduce a Decaying Stochasticity Schedule to realize it. • We construct the HiCoPrompt benchmark with explicit hierarchical structures to facilitate robust evaluation in compositional text-to-image generation. 2 Related Work Text-to-Image Diffusion Models. From UNet-based diffusion models (e.g., LDM 31 , SDXL 26 ) to the more powerful Transformer-based diffusion models (e.g., DiT 25 , SD3 5 , FLUX 15 ), Text-to-image diffusion models have been proven to produce high-fidelity images. However, they often fail to generate images under complex prompts involving multiple entities or subjects. To introduce more controllability into the diffusion model, recent work 47 ; 45 ; 27 ; 17 ; 44 ; 21 has included additional adapters or adjusted the prompt for a more accurate generation. Subject-driven Diffusion Models. While T2I models can generate diverse images of generic concepts, they still struggle to synthesize images of a specific subject provided by users. DreamBooth 32 and textual inversion 7