Layer -wise Instance Bind ing for Regional and Occlusion Control in Text-to-Image Diffusion Transformers
Ruidong Chen Yancheng Bai, Xuanpu Zhang, Jianhao Zeng, Lanjun Wang, Dan Song, Lei Sun, Xiangxiang Chu, Anan Liu · 2026
Read Layer -wise Instance Bind ing for Regional and Occlusion Control in Text-to-Image Diffusion Transformers by Ruidong Chen Yancheng Bai, Xuanpu Zhang, Jianhao Zeng, Lanjun Wang, Dan Song, Lei Sun, Xiangxiang Chu, Anan Liu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05769.txt.
Introduction
Chen and team propose LayerBind for area and hiding control. Layout control is handy, yet training methods inherit bias, while tricks struggle with hiding. It models area making as layers. Layout and hiding are set early in cleaning. Single start creates branches watching their own regions while tying to background. At an early step, branches fuse by layer order into a latent. Meaning nursing boosts details and keeps hiding order. A layered focus path runs beside the global path. It is training-free and plug-and-play.
Catalog record
| CREATOR | Ruidong Chen Yancheng Bai, Xuanpu Zhang, Jianhao Zeng, Lanjun Wang, Dan Song, Lei Sun, Xiangxiang Chu, Anan Liu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58339 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05769
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Layer -wise Instance Bind ing for Regional and Occlusion Control in Text-to-Image Diffusion Transformers Ruidong Chen Yancheng Bai, Xuanpu Zhang, Jianhao Zeng, Lanjun Wang, Dan Song, Lei Sun, Xiangxiang Chu, Anan Liu Abstract Region-instructed layout control in text-to-image generation is highly practical, yet existing methods suffer from limitations: (i) training-based approaches inherit data bias and often degrade image quality, and (ii) current techniques struggle with occlusion order, limiting real-world usability. To address these issues, we propose LayerBind . By modeling regional generation as distinct layers and binding them during the generation, our method enables precise regional and occlusion controllability. Our motivation stems from the observation that spatial layout and occlusion are established at a very early denoising stage, suggesting that rearranging the early latent structure is sufficient to modify the final output. Building on this, we structure the scheme into two phases: instance initialization and subsequent semantic nursing. (1) First, leveraging the contextual sharing mechanism in multimodal joint attention, Layer-wise Instance Initialization creates per-instance branches that attend to their own regions while anchoring to the shared background. At a designated early step, these branches are fused according to the layer order to form a unified latent with a pre-established layout. (2) Then, Layer-wise Semantic Nursing reinforces regional details and maintains the occlusion order via a layer-wise attention enhancement. Specifically, a sequential layered attention path operates alongside the standard global path, with updates composited under a layer-transparency scheduler. LayerBind is training-free and plug-and-play, serving as a regional and occlusion controller across Diffusion Transformers. It also supports editable workflows, allowing for flexible modifications like changing instances or rearranging visible orders. Experimental results demonstrate LayerBind’s effectiveness, highlighting its potential for creative applications. Project page: https://littlefatshiba.github.io/layerbind-page Figure 1 : We propose LayerBind, a training-free strategy to empower text-to-image DiT models 3 ; 39 with regional and occlusion controllability. ( Top ) Compared to prior methods 52 ; 53 , LayerBind produces customized images that respect the specified spatial layout and occlusion relations without image quality degradation. ( Bottom ) LayerBind is based on a context-sharing, region-branching strategy. This design inherently enables editable generation, allowing flexible modifications like changing per-region instances or visible orders. † † footnotetext: ∗ Corresponding author; ‡ Project lead. 1 Introduction Text-to-Image (T2I) models have advanced rapidly, with Diffusion Transformers (DiTs) 35 ; 11 emerging as the dominant architecture due to their strong scalability and high-fidelity generation quality. To enhance controllability, region-instructed layout control 60 ; 27 ; 53 ; 8 ; 58 uses regional cues (e.g., boxes or masks with associated instructions) to dictate instance placement and appearance. This approach is valued for executing user-specified or LLM-parsed layout plans 8 ; 49 ; 12 and has garnered widespread research attention. However, most existing methods 27 ; 58 ; 28 ; 60 ; 36 are designed particularly for U-Net pipelines 38 ; 11 , which are transferred poorly to DiTs, given their substantially different attention mechanisms, tokenization schemes, and model size. Consequently, research on DiT-native layout controllers remains limited. Existing DiT-native layout controllers primarily follow two directions. First, training-based methods, which fine-tune the DiT model 53 or adopt layout adapters 16 ; 47 ; 45 . While these methods can achieve precise layout control, they introduce training data bias, causing image quality degradations (e.g. CreatiLayout 53 in Fig. 1 ). Second, training-free methods, such as 8 ; 4 ; 52 , conduct regional prompting to inject semantics into localized regions, often preserving the model’s original generation quality. However, despite both having advantages, they fail to manage object occlusion, and often cause “concept blending”, where semantics from different regions erroneously fuse (Fig. 5 ). This highlights a critical and shared gap: achieving robust control over both regional layout and occlusion with high-fidelity generation in DiTs remains an unsolved problem. To address these limitations, we propose LayerBind , a training-free controller enabling precise regional and occlusion control for DiTs. Our approach is motivated by a key observation regarding the model’s denoising dynamics: the foundational layout is rigidly established at a very early denoising step 37 ; 30 ; 32 ; 24 (Fig. 2 a), and through rearranging this early latent structure, we can directly modify the final layout and occlusion (Fig. 2 b). This leads to our core motivation: effective layout control should align with the model’s intrinsic denoising dynamics , rather than countering them at temporally misaligned stages. LayerBind implements this principle by decoupling the region-instructed layout control into two sequential phases (Fig. 2 c): (1) Layer-wise Instance Initialization, an early-stage process to initialize instances and define layout and occlusion, and (2) Layer-wise Semantic Nursing, a subsequent phase to refine instance details while maintaining occlusion integrity. Figure 2 : (a, b) Observation: simply rearranging the latent structure at an early step directly manipulates the final spatial layout and occlusion order. (c) Our LayerBind scheme: initializing the instance layout first, then conducting semantic nursing for instance detail while maintaining layout and occlusions. LayerBind implements this two-stage design as follows. (1) Layer-wise Instance Initialization first creates per-instance branch tokens from the initial latents, corresponding to each specified region. Leveraging the contextual sharing mechanism of multimodal joint-attention (MM-Attention) 11 , these branches compute attention independently, allowing them to form distinct instances while adapting the shared background context. At a designated early step, these branches are fused into the global latent according to the desired layer order, either via direct latent merging or by using an optional foreground blend to composite the instances, thereby establishing the initial structured latent. (2) Layer-wise Semantic Nursing then takes over after the merge. This stage performs semantic refinement via layer-wise local attention enhancements. In each attention block, a standard global path runs, followed by a sequential layered path that processes each region and its instruction. A layer-transparency scheduler then manages these layered enhancements, ensuring regional semantics and layer relationships are progressively reinforced throughout the subsequent denoising process. In summary, LayerBind is a training-free and plug-and-play controller that enables precise region and occlusion control for DiTs while preserving generation quality. Furthermore, its region-branching scheme inherently enables an editable generation. This design permits flexible modifications, such as changing per-region objects, altering occlusion order (Fig. 1 , Bottom), or even performing composited image editing with any image as background context (Fig. 8 ). Extensive quantitative and qualitative experiments validate LayerBind’s state-of-the-art performance in complex layout and occlusion control. 2 Related Work 2.1 Layout-to-Image Generation Layout-to-Image (L2I) generation 58 ; 36 ; 6 ; 28 ; 7 ; 9 ; 43 aims to control the spatial layouts of synthesized images. A further extension, region-instructed L2I 12 ; 60 ; 53 ; 47 ; 27 ; 43 ; 19 , incorporates detailed regional information for finer-grained controllability. Early approaches diverged into three categories: (1) Training-based methods 58 ; 60 ; 27 that finetune the model on layout inputs; (2) Latent optimization methods 28 ; 36 ; 6 that leverage spatial objectives to guide denoising; and (3) Seed-based methods 32 ; 13 ; 26 that manipulate the initial noise for object placement. While these methods performed well on U-Net models (e.g., Stable Diffusion 1.5 38 ), they struggle to adapt to DiT-based architectures, which feature larger parameter counts and fundamentally different attention mechanisms. Consequently, recent efforts