Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation
Zekai Zhang, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang · 2026
Read Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation by Zekai Zhang, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26907.txt.
Introduction
Zekai Zhang and five colleagues present Qwen-Image-Agent, an agentic framework for real-world image generation. Text-to-image models draw well from detailed prompts. Real requests are vague, implicit, or need fresh facts. The authors call that mismatch the Context Gap. Their agent treats the user's words as partial context and builds the rest before drawing.
Planning first spots what context is missing and how to get it. Grounding then gathers it by reasoning, web search, memory, and feedback. The enriched context drives the image model. A new benchmark, IA-Bench, tests the four core agent skills: planning, reasoning, search, and memory. Across three benchmarks, the system beat strong baselines and reached state-of-the-art results, the authors report.
Catalog record
| CREATOR | Zekai Zhang, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.26907
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Qwen-Image-Agent: Bridging the Context Gap in Real-World Image Generation Thanks: Corresponding Author. Zekai Zhang, Jiahao Li, Jie Zhang, Kaiyuan Gao, Kun Yan, Lihan Jiang Abstract While text-to-image (T2I) models have achieved remarkable progress, they struggle with real-world requests that are often underspecified, implicit, or dependent on up-to-date knowledge. We identify this challenge as the Context Gap: the mismatch between the user context and the sufficient generation context for T2I models. To bridge this gap, we propose Qwen-Image-Agent, a unified agentic framework that integrates plan, reason, search, memory and feedback in a context-centric manner. Qwen-Image-Agent treats user input as partial context and progressively constructs the generation context through Context-Aware Planning and Context Grounding. Specifically, Context-Aware Planning identifies missing context and plans how it should be acquired and used, while Context Grounding gathers this context from reason, search, memory, and feedback. To evaluate agentic image generation, we further introduce Image Agent Bench (IA-Bench), a benchmark covering four core image agent capabilities: Plan, Reason, Search, and Memory. Experiments on IA-Bench, Mindbench and WISE-Verified show that Qwen-Image-Agent outperforms strong baselines and achieves state-of-the-art performance. Figure 1: Qwen-Image-Agent examples, generated without providing visual references. Introduction Text-to-image (T2I) models have achieved impressive progress in generating high-quality images from natural language prompts ( 17 ; 28 ; 31 ) . As these systems move into real-world applications such as marketing, product design, and slide creation, they are increasingly expected to solve practical visual tasks rather than merely render prompts. Despite their generative ability, current T2I models remain limited on real-world tasks ( 11 ) . A key reason is a structural mismatch between training and deployment: models are optimized for fully specified prompts ( 31 ) , while real-world requests are often underspecified. In practice, successful generation may require inferring implicit user intent, retrieving up-to-date knowledge or visual references from web, and incorporating interaction history. We refer to this mismatch as the Context Gap : the gap between the provided user context and the generation context required for T2I models. This gap motivates a paradigm shift from traditional direct image generation to agentic image generation , where the system must identify missing context, acquire it, and use it effectively during generation. Recent work has explored components such as plan ( 33 ) , reason ( 11 ) , search and tool use ( 35 ; 7 ; 11 ) , memory ( 12 ) , and self feedback ( 15 ; 29 ) , but these efforts remain fragmented and do not provide a unified framework for context-centered generation. To this end, we propose Qwen-Image-Agent , a unified agentic framework that integrates plan, reason, search, memory and feedback in a context-centric manner. Rather than treating user context as the final generation condition, our pipeline progressively constructs the full generation context through Context-Aware Planning and Context Grounding. Specifically, Context-Aware Planning operates at three levels: Information-level Planning identifies missing information and routes it to appropriate grounding strategies, Content-level Planning assembles grounded context into a detailed generation specification, and Generation-level Planning allocates context in multi-image and multi-turn scenarios. Context Grounding collects missing context from multiple sources, including reasoning for implicit intent inference, search for factual knowledge and visual references, memory for historical and personalized context, and feedback for iterative refinement. Overall, Qwen-Image-Agent is training-free, compatible with existing image generators, and supports both multi-image and multi-turn interaction. Existing evaluations mainly emphasize rendering abilities ( 8 ; 13 ) or isolated knowledge and reasoning abilities ( 20 ; 37 ) , but fail to systematically assess the capabilities required for agentic image generation. To fill this gap, we introduce Image-Agent-Bench (IA-Bench) , a benchmark that evaluates four core agentic capabilities: Plan, Reason, Search, and Memory, over 17 real-world tasks, 730 test instances, and 1801 fine-grained binary checklist items. Each task is paired with a structured VLM-based evaluation protocol for reliable assessment. Experiments on IA-Bench and prior benchmarks, including WISE-Verified ( 20 ) and MindBench ( 11 ) , show that Qwen-Image-Agent substantially outperforms strong agentic baselines and achieves state-of-the-art results. Ablation studies further verify the complementary benefits of different grounded contexts. Our contributions are summarized as follows: — We identify the Context Gap , i.e., the mismatch between user context and generation context as a fundamental challenge in real-world image generation. This provides a unified lens for understanding why current T2I systems fail in practical settings. — We propose Qwen-Image-Agent , a unified and context-centric framework for agentic image generation that addresses the context gap through plan, reason, search, memory and feedback. — We introduce IA-Bench , a benchmark for systematically evaluating agentic image generation along four capabilities: Plan, Reason, Search, and Memory. — Experiments show that Qwen-Image-Agent substantially outperforms strong agentic baselines, and achieve state-of-the-art performance on IA-Bench, Mindbench and WISE-Verified. Related Work 2.1 Agentic Image Generation Recent work extends image generation and editing with agent capabilities such as planning, reasoning, memory, search, and self feedback. Planning-based methods decompose complex intents into intermediate steps ( 33 ) ; Reasoning-based methods handle implicit user intent for more intelligent generation and editing ( 11 ) ; Search-based methods incorporate web search and image search to improve grounding in open-world scenarios ( 7 ; 11 ) ; Memory-based methods support long-horizon interactions through persistent memory ( 12 ) ; and Feedback-based methods study test-time scaling for image generation ( 15 ; 29 ) . However, from the perspective of generation context, existing methods remain fragmented in how they identify, acquire, and use the context required for real-world image generation. In contrast, Qwen-Image-Agent unifies plan, reason, memory, search, and feedback within a single framework, bridging the context gap in real-world image generation. 2.2 Benchmarks for Image Generation Early image generation benchmarks mainly evaluate instruction following and text–image alignment, such as GenEval ( 8 ) for compositional attribute binding and DPGBench ( 13 ) for dense prompt following. More recent benchmarks target harder settings that are either knowledge-driven or reasoning-driven. Knowledge-driven benchmarks, such as WISE ( 20 ) and PhyBench ( 19 ) , evaluate grounding in domain knowledge and physical commonsense. Reasoning-driven benchmarks, such as RISEBench ( 37 ) , test whether models can translate logical, causal, and spatio-temporal reasoning into visual outputs. Mind-Bench ( 11 ) covers both aspects. However, existing benchmarks mainly evaluate partial agent abilities, especially reasoning or search, while largely overlooking planning and memory. To support holistic evaluation of agentic image generation, we introduce IA-Bench, which covers the full spectrum of agent capabilities with fine-grained, checklist-based evaluation. Qwen-Image-Agent Framework 3.1 Formulation of Image Agents We formalize image generation and edit as a conditional rendering problem. Given a user context c_{u}=(P,I_{\mathrm{ref}}) with prompt P and optional reference images I_{\mathrm{ref}} , Direct image generation renders output image y in a single forward pass, where p_{\mathrm{gen}} is the image generator: y\sim p_{\mathrm{gen}}(\cdot\mid c_{u}). (1) In real-world scenarios, however, the provided user context is often incomplete for the desired visual task. We therefore distinguish user context c_{u} from the generation context c_{g} , which denotes the complete context needed for successful rendering. The earlier mentioned context gap is thus defined as the discrepancy between c_{u} and c_{g} . Agentic image generation addresses this gap by treating p_{\mathrm{gen}} as a renderer and introducing a context-construction process