Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs
Haochen Zhang, Animesh Sinha, Felix Juefei-Xu, Haoyu Ma, Kunpeng Li, Zhipeng Fan · 2026
Read Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs by Haochen Zhang, Animesh Sinha, Felix Juefei-Xu, Haoyu Ma, Kunpeng Li, Zhipeng Fan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20911.txt.
Introduction
Haochen Zhang and five co-authors tackle image generation that must remember a whole conversation. Their 2026 paper studies multi-round conversational image generation. A model must follow user instructions across many rounds. The chat history mixes text and images that pile up over time. Recent multimodal models can generate and edit images in this setting. Yet most benchmarks and training recipes are effectively Markov.
Markov here means the next output leans mainly on the most recent image. That allows shortcut solutions that ignore long-range history. The authors call their setting non-Markov to stress the full memory demand. A good model must respect choices made many turns ago. The work pushes image tools toward true collaboration. It matters for design tasks where users refine one picture over a long session.
Catalog record
| CREATOR | Haochen Zhang, Animesh Sinha, Felix Juefei-Xu, Haoyu Ma, Kunpeng Li, Zhipeng Fan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Non-Markov Multi-Round Conversational Image Generation with History-Conditioned MLLMs Haochen Zhang, Animesh Sinha, Felix Juefei-Xu, Haoyu Ma, Kunpeng Li, Zhipeng Fan Abstract Conversational image generation requires a model to follow user instructions across multiple rounds of interaction, grounded in interleaved text and images that accumulate as chat history. While recent multimodal large language models (MLLMs) can generate and edit images, most existing multi-turn benchmarks and training recipes are effectively Markov : the next output depends primarily on the most recent image, enabling shortcut solutions that ignore long-range history. In this work we formalize and target the more challenging non-Markov setting, where a user may refer back to earlier states, undo changes, or reference entities introduced several rounds ago. We present (i) non-Markov multi-round data construction strategies, including rollback-style editing that forces retrieval of earlier visual states and name-based multi-round personalization that binds names to appearances across rounds; (ii) a history-conditioned training and inference framework with token-level caching to prevent multi-round identity drift; and (iii) enabling improvements for high-fidelity image reconstruction and editable personalization, including a reconstruction-based DiT detokenizer and a multi-stage fine-tuning curriculum. We demonstrate that explicitly training for non-Markov interactions yields substantial improvements in multi-round consistency and instruction compliance, while maintaining strong single-round editing and personalization. I Introduction Recent progress in diffusion models [ 16 , 31 , 34 ] and multimodal large language models (MLLMs) [ 42 , 14 , 38 ] has made it possible to generate high-quality images that follow detailed natural-language instructions. Beyond single-shot text-to-image synthesis, a growing set of applications—from creative design assistants to personalized visual storytelling and interactive content creation—requires conversational image generation , where a user iteratively refines an image through multiple rounds of interaction. In this setting, the model must not only produce visually compelling outputs, but also track what has been established previously, resolve references in context, and apply new instructions in a way that remains consistent with both the conversation and the evolving image. A key challenge is that multi-round image generation is fundamentally different from single-turn editing [ 6 , 2 ] or personalization [ 8 , 44 ] . Real users interact in ways that are often non-linear : they may ask to revert to an earlier version, undo a sequence of changes, or apply a new edit to a prior state rather than the most recent one. They may also introduce multiple entities early in the conversation and later refer to them by name or attributes that were defined several turns ago. Handling such behavior requires non-Markov multi-round reasoning over interleaved text and images, where the correct next image can depend on information that is not contained in the most recent turn. Fig. 1: Non-Markov vs. Markov multi-round generation. Markov: each turn depends primarily on the latest image. Non-Markov: later turns refer to earlier states (rollback/undo) or to entities introduced multiple rounds ago (name-based references). However, most existing multi-turn datasets [ 49 , 14 ] and training pipelines [ 38 , 14 ] implicitly simplify the problem to an effectively Markov process, where the next output depends primarily on the latest image and the latest instruction. This Markov assumption can lead to shortcut learning: a model can appear strong in multi-turn settings while largely ignoring long-range context. In practice, this manifests as failures in rollback instructions (e.g., “go back to the version before the background change”), inconsistent identity for personalized subjects across turns, and incorrect entity references when multiple people or objects are involved. These failure modes motivate a more faithful formulation of conversational image generation that explicitly targets the non-Markov setting. In this work, we advocate for and develop a solution to non-Markov multi-round conversational image generation . We formalize the distinction between Markov and non-Markov multi-round generation and show why the latter better reflects realistic user behavior. We then introduce practical data construction strategies to create non-Markov supervision at scale. First, we construct rollback-style multi-round editing examples by augmenting Markov editing chains with edits originating from earlier states and composing them into dialogs that explicitly refer back to those earlier versions. These dialogs require the model to retrieve the correct historical visual state before applying a new instruction, making them inherently non-Markov. Second, we introduce name-based multi-round personalization examples in which a subject is introduced and named in an early round, and later rounds request portraits or refinements using only the name. We construct this data from videos, using captioning and name assignment to define entities and face detection/segmentation to extract supervised targets, thereby enforcing persistent name-to-appearance binding across rounds. Solving non-Markov multi-round generation also raises systems-level and modeling challenges. A practical obstacle is multi-round drift : repeatedly encoding a generated image into tokens and decoding back to pixels can introduce small reconstruction errors that accumulate over turns, leading to identity degradation and compounding artifacts. To address this, we propose token-level history caching , where the model caches and reuses previously generated image tokens directly as dialog history, rather than re-encoding reconstructed images at each round. This design reduces compounding reconstruction noise and stabilizes long-horizon interactions. Finally, robust non-Markov multi-round performance depends on strong underlying capabilities in reconstruction fidelity and personalization. We therefore incorporate two enabling improvements. We upgrade the diffusion-based detokenizer with a reconstruction-based DiT detokenizer [ 29 ] that preserves fine details (especially faces) while maintaining compatibility with the tokenizer interface, minimizing the need for upstream retraining. In addition, we adopt a multi-stage fine-tuning curriculum for editable personalization that transitions from identity-preserving reconstruction to prompt-following edits while maintaining subject consistency. Together, these components form a coherent framework: high-fidelity reconstruction and editable personalization provide a reliable substrate, while non-Markov data construction and history-aware training/inference unlock true conversational reasoning beyond the Markov assumption. Our contributions are summarized as follows: • Non-Markov formulation for multi-round conversational image generation. We formalize the limitation of Markov-style multi-turn settings and motivate non-Markov multi-round reasoning as the appropriate target for realistic conversational behavior. • Non-Markov multi-round data construction. We propose rollback-style editing construction from existing Markov dataset, and name-based multi-round personalization from videos, producing supervision that requires retrieving earlier visual states and long-range entity bindings. • History-conditioned training and token-level caching. We train with full dialog history and introduce token-level caching to mitigate multi-round drift caused by repeated encode/decode cycles. • Enabling improvements. We introduce a reconstruction-based DiT detokenizer and a multi-stage personalization curriculum that together improve fidelity and editable identity preservation, which are critical for stable multi-round interactions. This manuscript is an extended version of our published conference paper [ 48 ] , with a primary focus on non-Markov multi-round conversational image generation, including additional datasets, detailed methods, and more evaluations. The rest of the paper is organized as follows. Section III formalizes Markov and non-Markov multi-round generation and presents our non-Markov data construction pipelines. Section IV describes history-conditioned training, token-level caching, and key enabling components. Section V reports experimental results, emphasizing non-Markov rollback editing and name-based multi-round personalization, followed by ablations and analysis. II Related Work II-A Image Generation, Editing, and Personalization Diffusion models, beginning with DDPM [ 16 ] , have rapidly advanced text-to-image generation, with systems such as Stable Diffusion [ 31 , 34 ] , DALL-E [ 33 ] , and Imagen [ 4 ]