MoGen: A Unified Collaborative Framework for Controllable Multi-Object Image Generation
Yanfeng Li, Yue Sun, Keren Fu, Sio-Kei Im, Xiaoming Liu, Guangtao Zhai, Xiaohong Liu, Tao Tan · 2026
Read MoGen: A Unified Collaborative Framework for Controllable Multi-Object Image Generation by Yanfeng Li, Yue Sun, Keren Fu, Sio-Kei Im, Xiaoming Liu, Guangtao Zhai, Xiaohong Liu, Tao Tan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-05546.txt.
Introduction
Yanfeng Li and 7 co-authors propose MoGen, a 2026 method for generating images containing several objects with the right counts and attributes. Earlier generators often place the wrong number of objects or mix up which description belongs to which region. Fixes usually depend on rigid external control signals that many users cannot supply in a fixed format.
MoGen anchors each phrase in a description to its image region, enforcing quantity during generation. A second module accepts varied combinations of control signals, parses them into structured intent, and applies selective constraints to layout and attributes. Experiments reported better generation quality, quantity consistency, and fine-grained control than existing methods, with more flexible inputs for users.
Catalog record
| CREATOR | Yanfeng Li, Yue Sun, Keren Fu, Sio-Kei Im, Xiaoming Liu, Guangtao Zhai, Xiaohong Liu, Tao Tan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-05546
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MoGen: A Unified Collaborative Framework for Controllable Multi-Object Image Generation Yanfeng Li, Yue Sun, Keren Fu, Sio-Kei Im, Xiaoming Liu, Guangtao Zhai, Xiaohong Liu, Tao Tan Abstract Existing multi-object image generation methods face difficulties in achieving precise alignment between localized image generation regions and their corresponding semantics based on language descriptions, frequently resulting in inconsistent object quantities and attribute aliasing. To mitigate this limitation, mainstream approaches typically rely on external control signals to explicitly constrain the spatial layout, local semantic and visual attributes of images. However, this strong dependency makes the input format rigid, rendering it incompatible with the heterogeneous resource conditions of users and diverse constraint requirements. To address these challenges, we propose MoGen, a user-friendly multi-object image generation method. First, we design a Regional Semantic Anchor (RSA) module that precisely anchors phrase units in language descriptions to their corresponding image regions during the generation process, enabling text-to-image generation that follows quantity specifications for multiple objects. Building upon this foundation, we further introduce an Adaptive Multi-modal Guidance (AMG) module, which adaptively parses and integrates various combinations of multi-source control signals to formulate corresponding structured intent. This intent subsequently guides selective constraints on scene layouts and object attributes, achieving dynamic fine-grained control. Experimental results demonstrate that MoGen significantly outperforms existing methods in generation quality, quantity consistency, and fine-grained control, while exhibiting superior accessibility and control flexibility. Code is available at: https://github.com/Tear-kitty/MoGen/tree/master . I Introduction Image generation has emerged as one of the most transformative research areas in computer vision, attracting widespread attention from both academia and industry [ 47 , 15 , 19 ] . With significant progress in generative models, researchers can synthesize images with unprecedented quality and diversity. This technological breakthrough not only provides powerful creative tools for professional users but also enables non-professionals to generate high-quality, diverse visual content, substantially lowering the technical barriers to digital art creation [ 51 , 30 , 29 , 4 , 54 ] . \begin{overpic}[fig/homedisplay.pdf] \end{overpic} Fig. 1: High-quality samples from our MoGen . MoGen enables multi-object image generation with fine-grained control via text, optionally augmented with bounding boxes, structure references, and object references. \begin{overpic}[fig/defect.pdf] \put(0.0,-0.5){\footnotesize\parbox{113.81102pt}{(a) Text-to-image method}} \put(23.0,-0.5){\footnotesize\parbox{128.0374pt}{(b) Signal-based method}} \put(44.5,-0.5){\footnotesize\parbox{284.52756pt}{(c) Proposed method MoGen [S: structure, O: object, B: box]}} \end{overpic} Fig. 2: The comparison between previous methods and our MoGen . (a) Text-to-image methods offer high accessibility but often fail to ensure quantity consistency and lack fine-grained control over scene layouts and object attributes. (b) Signal-based methods enable fine-grained control, thereby improving quantity consistency, yet their reliance on the rigid input format limits accessibility and control flexibility. (c) MoGen enables text-to-image generation that follows quantity requirements, and can further integrate various combinations of multi-source control signals. Depending on the activated signals, the model imposes corresponding fine-grained control over scene layouts and object attributes. Early research was pioneered by Stable Diffusion [ 34 ] , which was the first to perform the high-resolution image generation in the latent space. As the technology evolved, subsequent studies progressively expanded the boundaries of its applications [ 12 , 8 , 31 ] . For instance, T-person [ 24 ] and OmniGen [ 47 ] achieved identity consistency in character generation, while CosiStory [ 37 ] further extended this paradigm to the generation of narratively coherent image sequences. However, existing methods still face critical challenges in multi-object composite scene generation, including difficulties in accurately constructing spatial layouts, inter-object relationships, and attribute constraints among multiple objects. These limitations significantly hinder scalability in the number of generable objects, thereby constraining the ability to meet diverse creative demands. In recent years, multi-object image generation has primarily explored text-to-image (T2I) paradigms due to their accessibility and ease of use [ 55 ] . Some works (such as Genartist [ 41 ] and MuLan [ 20 ] ) leverage large language models (LLMs) to enhance global text semantic modeling, improving quantity consistency in single-category or structurally simple scenarios. However, when descriptions contain multi-category objects with multi-attribute constraints, the global semantics provided by LLMs cannot be effectively grounded into phrase-level text semantics that capture individual object names and their modifiers. As a result, localized image generation regions often fail to precisely align with their corresponding text semantics, leading to semantic ambiguity and attribute aliasing, which in turn result in misgeneration. Additionally, textual descriptions lack explicit spatial semantics and visual references, leading to ambiguous representations of key visual elements such as appearance prototypes and relative positions, which inadequately support layout control and attribute restoration. To address these problems, another research introduces human-predefined signals as external controls to explicitly constrain spatial layouts and visual attributes of localized regions. For example, LLM-grounded [ 9 ] and BoxDiff [ 49 ] use bounding boxes to drive generation for precise layouts, significantly improving quantity consistency. Building on this, Emu2 [ 36 ] and CompAgent [ 42 ] further incorporate object references to achieve appearance consistency and attribute controllability. Despite their effectiveness, these methods face two key limitations: first, the high cost of constructing control signals leads to method designs that favor single control paradigms; second, they lack the capability to adaptively parse and integrate structured intent from various combinations of multi-source control signals, resulting in rigid input formats. These factors weaken accessibility and control flexibility, thereby limiting adaptability to heterogeneous resource conditions and diverse control requirements, ultimately compromising user convenience. We propose MoGen, as shown in Fig. 1 , a user-friendly multi-object image generation method. First, MoGen introduces a Regional Semantic Anchor (RSA) module to alleviate the misalignment between localized image generation regions and their corresponding text semantics in multiple categories scenarios with coexisting attributes. This enables the independence of semantics and attributes for each image region, thereby ensuring the text-to-image generation that follows quantity requirements and providing an accessible and extensible generation method for users. Building upon this, MoGen further incorporates the Adaptive Multi-modal Guidance (AMG) module to achieve adaptive parsing and integration of various combinations of multi-source control signals, thereby allowing flexible incorporation of bounding boxes, structure references, and object references. These signals can be utilized individually or in combinations, and according to the activated signals, AMG formulates their corresponding structured intent to dynamically impose fine-grained control over scene layouts and object attributes. This design enables MoGen to enhance generation controllability and control flexibility while maintaining accessibility, effectively adapting to varying user requirements and resource conditions. Fig. 2 demonstrates the comparison between previous methods and MoGen. The specific implementation involves three key components: 1) The RSA module, incorporates two key designs: (i) a task-oriented Semantic Parser is introduced to supersede reliance on LLMs by modeling global text semantics from text embeddings output by text encoder and further decomposing these embeddings into phrase-level text semantics. (ii) a synergistic utilization mechanism is proposed, where global text semantics steer all U-Net blocks for structural stability in image generation, while phrase-level text semantics are integrated into the layout block [ 38 ] to further establish precise alignment between localized generation regions and their corresponding text semantics. These designs effectively ensure quantity consistency. TABLE I: An overview of previous studies. Quantity and Ada Struc. indicate whether quantity consistency and adaptive structured intent parsing mechanisms are incorporated, respectively. w/o LLM indicates whether the method operates without relying on LLMs.