IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation

Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun · 2026

Read IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation by Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24849.txt.

Introduction

Zixuan Li and twelve colleagues improve how multimodal language models follow the structure of a text prompt when generating images. Models often render appearance well but break the prompt's structure. Object counts, spatial relations, attribute bindings, and rough layouts get lost. The authors blame a single conditioning stream that entangles planning with rendering. Their method, IV-CoT, splits visual queries into two cascades. Structural queries first build a latent visual plan. Semantic queries then render appearance on top of that plan.

Training only sketch supervision guides the structural queries toward real structure. No sketches are needed when the model runs. The whole reasoning happens implicitly in one forward pass. Results on the GenEval and T2I-CompBench benchmarks beat prior approaches. Visualisations show the two query types playing complementary roles. Li and colleagues argue this separation is what structure aware generation was missing.

Catalog record

CREATORZixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24849

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation Zixuan Li, Haokun Lin, Yicheng Xiao, Zhiwei Li, Xinyang Song, Zelong Zheng, Yong He, Heng Yao, Ke Ding, Chao Yu, Chuan Yuan, Qi Li, Zhenan Sun Abstract Unified multi-modal large language models (MLLMs) have achieved strong text-to-image generation quality, but still struggle with structure-aware prompt following, where object counts, spatial relations, attribute bindings, and coarse layouts must be preserved. We attribute this limitation in part to the entanglement of structural planning and appearance rendering within a single conditioning stream. To address this issue, we propose Implicit Visual Chain-of-Thought (IV-CoT) , a latent visual reasoning framework for query-conditioned image generation. IV-CoT decomposes the visual conditioning queries into a structural-to-semantic cascade, where structural queries first form a latent visual plan and semantic queries then render appearance conditioned on this plan. To guide the structural queries, we introduce training-only sketch supervision, which encourages them to capture structure from sketches without requiring sketch extraction or intermediate decoding at inference time. IV-CoT performs implicit CoT reasoning in a single forward pass and achieves superior results on GenEval and T2I-CompBench. Visualizations and analyses demonstrate that the learned structural and semantic queries play complementary roles in structure-aware generation. 1 Introduction Recent unified multi-modal large language models (MLLMs) have shown strong capabilities in generating realistic images from open-ended instructions ( 68 ; 49 ; 34 ; 8 ; 52 ; 56 ) . However, they still struggle with prompts that impose complex structural requirements ( 19 ; 13 ; 65 ; 22 ) . When a prompt specifies multiple objects with distinct shapes, materials, attributes, and spatial arrangements, the model may produce visually plausible images while swapping attributes, omitting objects, or violating the requested layout. We refer to this setting as structure-aware prompt following ; an example is illustrated in Figure 1 . Most unified MLLM-based generators convert the prompt into a visual conditioning stream through an understanding MLLM, where scene structure, object identity, attributes, and appearance details are compressed together ( 36 ; 50 ) . Such entangled conditioning makes it difficult for the generator to distinguish what should determine the scene structure from what should control visual appearance. Recent works therefore explore Chain-of-Thought (CoT) reasoning for image generation, using intermediate reasoning steps to better handle complex scenes ( 16 ; 48 ) . Existing CoT-based generation methods mainly follow two explicit paradigms. Explicit Textual CoT generates intermediate verbal reasoning, scene descriptions, or numerical layouts before image synthesis ( 9 ; 21 ; 45 ) . However, language alone has limited spatial bandwidth for representing continuous 2D geometry, object boundaries, relative scale, and occlusion. Explicit Interleaved CoT incorporates visual intermediate states, such as masks, layouts, or draft images, to provide stronger structural cues ( 15 ; 22 ; 38 ) . Yet this comes at the cost of explicit intermediate decoding, multi-stage pipelines, and potential error accumulation. These complementary limitations suggest that structure-aware generation needs an intermediate planning mechanism that preserves visual-spatial information while avoiding decoded reasoning states at inference time (Figure 1 ). Figure 1: Comparison of reasoning paradigms for text-to-image generation. (Left) : direct generation, explicit textual CoT, explicit interleaved CoT, and IV-CoT differ in where intermediate reasoning states are represented. (Right) : a structure-aware prompt-following example showing that IV-CoT better preserves the requested object layout without explicitly decoding intermediate images. Latent reasoning provides a natural alternative, where intermediate deliberation is carried by hidden states or soft thought tokens rather than explicit outputs ( 17 ; 62 ; 61 ; 39 ) . Although related ideas have begun to appear in multimodal reasoning ( 37 ; 1 ) , latent reasoning for text-to-image generation remains underexplored. The key question is: how should such latent states be organized before image rendering? We argue for a structure-first organization: the latent state should first anchor object boundaries, layout, and coarse shape, and then guide semantic appearance rendering. Without this structural scaffold, the generator may bind attributes to wrong objects, misplace spatial relations, or produce plausible details on incorrect structures. We propose Implicit Visual Chain-of-Thought (IV-CoT) , a structure-first latent reasoning framework in the query space of a unified MLLM-DiT generator. IV-CoT internalizes structural planning into a causal structural-to-semantic query cascade : structural queries are placed before semantic queries and first encode a latent visual plan, including object form, count, layout, and coarse spatial relations; semantic queries then attend to this plan to render appearance and fine-grained details. In this way, the “chain” is realized as an ordered latent dependency from structural planning to semantic rendering, rather than an external sequence of text, layouts, or decoded images. We further use sketches as training-only structural guidance through a two-stage training scheme. Unlike sketch- or layout-conditioned generation methods that require external spatial controls at inference time, IV-CoT uses sketches only to shape latent structural queries during training. In the first stage, sketch supervision encourages structural queries to encode contours, shapes, counts, and layouts while suppressing appearance factors such as color, texture, and lighting. In the second stage, IV-CoT is optimized for image generation with the structural objective retained as a regularizer, keeping structural queries aligned with the latent visual plan while semantic queries learn to render appearance details. At inference time, IV-CoT takes only the text prompt and produces structural and semantic queries in a single forward pass, without extracting sketches, decoding intermediate images, or generating explicit reasoning traces. Our contributions are summarized as follows: • We formulate Implicit Visual Chain-of-Thought, where intermediate visual planning is internalized in latent query representations rather than externalized as explicit textual or visual intermediate states. • We instantiate this formulation with a causal structural-to-semantic query cascade and training-only sketch supervision, which shape structural queries into latent visual plans while preserving text-only, single-pass inference. • Using the same OpenUni-L-1024 backbone, IV-CoT improves GenEval from 0.86 to 0.88 and T2I-CompBench from 0.5448 to 0.5743. Meanwhile, IV-CoT keeps single-pass inference and achieves 9-15 \times lower latency than explicit CoT methods. Visualizations and cross-prompt recombination show that structural queries encode recoverable and manipulable latent visual plans. The relevant code will be released upon acceptance of the paper. 2 Method Figure 2: Illustration of IV-CoT for text-to-image generation. (a) During training, IV-CoT learns latent structural and semantic queries with structural and image-generation supervision. (b) During inference, the MLLM performs latent structural-to-semantic reasoning, and the resulting queries guide the DiT to generate the final image. No intermediate images are explicitly decoded; query visuals are shown only for illustration. We propose Implicit Visual Chain-of-Thought (IV-CoT) , a structure-first latent reasoning framework built upon a query-conditioned MLLM-DiT generator. Motivated by the difficulty of generation models in structure-aware prompt following (Figure 1 ), IV-CoT introduces an ordered structural-to-semantic query cascade that first encodes structural information and then performs semantic refinement. To enable such latent reasoning, we train the structural query inputs \mathbf{Q}_{s}^{0} with a sketch-supervised objective and optimize image generation with structural regularization. This section first reviews the backbone architecture, followed by the proposed components and inference procedure. 2.1 Query-Conditioned MLLM-DiT Generation We build IV-CoT on a unified MLLM-DiT generation architecture ( 36 ; 50 ) . Given a text prompt \mathbf{y} and a set of learnable visual query inputs \mathbf{Q}_{0}\in\mathbb{R}^{N\times d} , the MLLM produces

Read the complete text