Endogenous Reprompting: Self-Evolving Cognitive Alignment for Unified Multimodal Models
Zhenchen Tang, Songlin Yang, Zichuan Wang, Bo Peng, Yang Li, Beibei Dong, Jing Dong · 2026
Read Endogenous Reprompting: Self-Evolving Cognitive Alignment for Unified Multimodal Models by Zhenchen Tang, Songlin Yang, Zichuan Wang, Bo Peng, Yang Li, Beibei Dong, Jing Dong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-20305.txt.
Introduction
Zhenchen Tang and six colleagues address a gap in unified multimodal models, which both understand and generate images and text. Their understanding is strong, yet it often fails to guide their own generation. The authors call this the cognitive gap: the model does not know how to use what it understands to improve what it makes.
They propose endogenous reprompting, where the model writes self-aligned descriptors during generation as an explicit reasoning step. Understanding then becomes an active part of making, not a passive encoding. The idea could raise the quality of generated images without a bigger model.
Catalog record
| CREATOR | Zhenchen Tang, Songlin Yang, Zichuan Wang, Bo Peng, Yang Li, Beibei Dong, Jing Dong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-20305
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Endogenous Reprompting: Self-Evolving Cognitive Alignment for Unified Multimodal Models Zhenchen Tang, Songlin Yang, Zichuan Wang, Bo Peng, Yang Li, Beibei Dong, Jing Dong Abstract Unified Multimodal Models (UMMs) exhibit strong understanding, yet this capability often fails to effectively guide generation. We identify this as a Cognitive Gap: the model lacks the understanding of how to enhance its own generation process. To bridge this gap, we propose Endogenous Reprompting, a mechanism that transforms the model’s understanding from a passive encoding process into an explicit generative reasoning step by generating self-aligned descriptors during generation. To achieve this, we introduce SEER (Self-Evolving Evaluator and Reprompter), a training framework that establishes a two-stage endogenous loop using only 300 samples from a compact proxy task, Visual Instruction Elaboration. First, Reinforcement Learning with Verifiable Rewards (RLVR) activates the model’s latent evaluation ability via curriculum learning, producing a high-fidelity endogenous reward signal. Second, Reinforcement Learning with Model-rewarded Thinking (RLMT) leverages this signal to optimize the generative reasoning policy. Experiments show that SEER consistently outperforms state-of-the-art baselines in evaluation accuracy, reprompting efficiency, and generation quality, without sacrificing general multimodal capabilities. Code and project page are available at https://2kxx.github.io/SEER.github.io/. 1 New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences 2 School of Artificial Intelligence, University of Chinese Academy of Sciences 3 MMLab@HKUST, The Hong Kong University of Science and Technology 1 Introduction Recent Unified Multimodal Models (UMMs) have successfully integrated understanding and generation capabilities into a single framework ( 39 ; 10 ) . By sharing a common backbone, UMMs possess the potential for seamless cross-modal interaction, offering a promising path to resolving the semantic disconnect often found in traditional separate-encoder T2I models. However, a significant imbalance persists: despite a shared architecture, the model’s ability to generate lags substantially behind its ability to understand ( 8 ; 25 ; 40 ; 30 ; 17 ) . As illustrated in Figure 1 , while the model accurately comprehends visual instructions, it struggles to translate this understanding into generative guidance, resulting in misalignment in the final output. We identify this as a cognitive gap: the model lacks the specific understanding of how to enhance its own generation process (details in the Appendix A.4.3 ). Figure 1 : Bridging the cognitive gap. While direct generation fails to reflect the model’s understanding (Left), and external reprompters often cause misalignment by generating descriptors that mismatch the generator’s priors (Right), SEER (Middle) leverages Endogenous Reprompting. It produces concrete, self-aligned descriptors that strictly match the generator’s generative priors, successfully bridging understanding and generation. To bridge this gap, we propose Endogenous Reprompting, a mechanism that transforms the model’s understanding from a passive encoding process into an explicit generative reasoning step by translating user intention into self-aligned descriptors (aligned with the model’s own capabilities) during generation. Prior approaches ( 2 ; 29 ) rely on disjoint models (e.g., an LLM and a generator), which leads to representation mismatch, causing the generated descriptions to mismatch the generator’s priors in Figure 1 . In contrast, our approach exploits the intrinsic representational consistency of UMMs. Being endogenous, all reasoning operates within the model’s shared representation space, enabling knowledge to be seamlessly transferred between understanding and generation to generate reprompts. This ensures Model-Specific Alignment: the model translates user intention into specific linguistic descriptors that are aligned with the generator’s generative priors. We argue that this constitutes the missing “understanding-to-generation” pathway. To achieve this, we propose SEER (Self-Evolving Evaluator and Reprompter), a training framework that establishes a two-stage endogenous loop using only 300 samples from a compact proxy task, Visual Instruction Elaboration. We introduce this task to specifically bootstrap generative reasoning: it targets the explicit elaboration of implicit user intention, forcing the model to reason about visual instructions to enhance generation rather than simply mapping text to pixels. Using this compact signal, SEER unfolds in two stages. First, we employ Reinforcement Learning with Verifiable Rewards (RLVR) ( 26 ; 33 ) through a curriculum learning strategy to activate the internal evaluator. By adopting a pairwise optimization strategy that learns from relative comparisons, we transform the model’s understanding into a high-fidelity reward signal. Second, we employ Reinforcement Learning with Model-rewarded Thinking (RLMT) ( 3 ) to harness this signal, enabling the model to autonomously refine its reasoning policy. This distinguishes our work from traditional Reinforcement learning from human feedback (RLHF) for visual generation ( 45 ; 41 ; 22 ; 13 ) : whereas RLHF targets the generation execution phase (optimizing pixels), our approach optimizes the generative reasoning process (optimizing prompts), effectively teaching the model to think before it generates. Empirically, SEER enhances instruction compliance and reasoning capabilities, utilizing the model’s own understanding to resolve the cognitive gap. The entire self-evolution process relies on minimal data, highlighting the efficiency of unlocking intrinsic potential rather than learning from scratch. Our key contributions are summarized as follows: • We pioneer Endogenous Reprompting, establishing the first generative reasoning framework for UMMs to bridge the cognitive gap via a compact proxy task, Visual Instruction Elaboration. • We propose SEER, a self-evolving framework that establishes a two-stage endogenous loop to achieve Endogenous Reprompting, activating an internal evaluator with RLVR and optimizing the reasoning policy with RLMT. • We demonstrate that SEER consistently outperforms state-of-the-art baselines in instruction compliance, reprompting efficiency, and generation quality, using only 300 samples, without sacrificing general multimodal capabilities. Figure 2 : Overview of SEER. The framework bridges the cognitive gap via a two-stage endogenous loop. RLVR: (Step 1) collecting endogenous image pairs to (Step 2) optimize the internal evaluator. RLMT: (Step 3) optimizing the Reprompting Policy using (Step 4) relative rewards derived from the activated evaluator. 2 Related Work Unified Multimodal Models. UMMs integrate understanding and generation within a single end-to-end framework to promote cross-modal interaction. Recent research explores a broad architectural design space, primarily categorized into three paradigms: (1) Autoregressive Models, such as Janus ( 34 ) , and Emu3 ( 30 ) , which tokenize inputs for sequential next-token prediction, with variants like Show-o ( 39 ) incorporating discrete diffusion schedules; (2) Hybrid Architectures, which either attach diffusion heads to a shared transformer backbone ( 43 ; 6 ) or route features from frozen MLLMs to external generators via learnable queries ( 7 ) ; and (3) Emerging Designs, including Masked-Autoregressive (MAR) generation ( 20 ; 36 ) and scalable Mixture-of-Transformers (MoT) frameworks ( 44 ; 9 ) . Despite these architectural strides, a common bottleneck persists: the alignment between strong understanding and generation remains insufficiently activated, creating the cognitive gap that SEER aims to bridge. Prompt Optimization and Reprompting. To bridge the gap between user intent and generative execution, researchers have increasingly prioritized Prompt Optimization. Foundational approaches like DALL-E 3 ( 2 ) utilize recaptioning to generate dense captions for model fine-tuning. For inference-time enhancement, on-the-fly rewriting has become a dominant paradigm, employing LLMs for iterative critique and refinement ( 37 ) , automated prompt engineering ( 5 ) , or complex agent-based planning ( 32 ) . Recent works like PromptEnhancer ( 29 ) further advance this by training dedicated rewriters against fine-grained reward models. However, these methods typically couple disjoint models, leading to a fundamental representation mismatch. In contrast, by leveraging the intrinsic representational consistency of UMMs,