Reinforcement Learning in Vision: A Survey
Weijia Wu, Chen Gao, Joya Chen, Kevin Qinghong Lin, Qingwei Meng, Yiming Zhang, Yuke Qiu, Hong Zhou, Mike Zheng Shou · 2025
Read Reinforcement Learning in Vision: A Survey by Weijia Wu, Chen Gao, Joya Chen, Kevin Qinghong Lin, Qingwei Meng, Yiming Zhang, Yuke Qiu, Hong Zhou, Mike Zheng Shou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08189.txt.
Introduction
Weijia Wu, Chen Gao, Joya Chen, and a team of co-authors survey how reinforcement learning, training by reward, has merged with visual AI. The combination now powers systems that not only see scenes but reason about them, generate images and video, and act in the world.
The survey maps more than two hundred works into four areas: multimodal language models, visual generation, unified frameworks, and models that join vision, language, and action. It traces how the training recipes evolved, from learning from human preferences to rewards that can be checked automatically. Recurring trends include staged curricula and reward designs shared across tasks. The authors also review how the field measures success and flag open problems, from sample hunger to safe deployment.
Catalog record
| CREATOR | Weijia Wu, Chen Gao, Joya Chen, Kevin Qinghong Lin, Qingwei Meng, Yiming Zhang, Yuke Qiu, Hong Zhou, Mike Zheng Shou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08189
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reinforcement Learning in Vision: A Survey Weijia Wu, Chen Gao, Joya Chen, Kevin Qinghong Lin, Qingwei Meng, Yiming Zhang, Yuke Qiu, Hong Zhou, Mike Zheng Shou Abstract Recent advances at the intersection of reinforcement learning (RL) and visual intelligence have enabled agents that not only perceive complex visual scenes but also reason, generate, and act within them. This survey offers a critical and up-to-date synthesis of the field. We first formalize visual RL problems and trace the evolution of policy-optimization strategies from RLHF to verifiable reward paradigms, and from Proximal Policy Optimization to Group Relative Policy Optimization. We then organize more than 200 200 representative works into four thematic pillars: multi-modal large language models, visual generation, unified model frameworks, and vision-language-action models. For each pillar we examine algorithmic design, reward engineering, benchmark progress, and we distill trends such as curriculum-driven training, preference-aligned diffusion, and unified reward modeling. Finally, we review evaluation protocols spanning set-level fidelity, sample-level preference, and state-level stability, and we identify open challenges that include sample efficiency, generalization, and safe deployment. Our goal is to provide researchers and practitioners with a coherent map of the rapidly expanding landscape of visual RL and to highlight promising directions for future inquiry. Resources are available at: https://github.com/weijiawu/Awesome-Visual-Reinforcement-Learning . 1 Introduction Reinforcement Learning (RL) has achieved remarkable successes in the field of Large Language Models (LLMs) ( Jaech et al. 2024 ; Rafailov et al. 2023 ) , most notably through paradigms like Reinforcement Learning from Human Feedback (RLHF) ( Ouyang et al. 2022 ) and innovative frameworks such as DeepSeek-R1 ( Guo et al. 2025a ) . These methodologies have significantly enhanced the capabilities of LLMs, aligning generated outputs closely with human preferences and enabling nuanced, complex reasoning and interaction capabilities previously unattainable through supervised learning alone. In recent years, inspired by these remarkable achievements from LLM, there has been an explosive interest in extending the RL methodologies that proved successful for LLMs to multimodal large models, including Vision-Language Models (VLM) ( Zhou et al. 2025a ; Zheng et al. 2025 ; Zhang et al. 2025g ) , Vision-Language-Action models (VLA) ( Lu et al. 2025b ; Tan et al. 2025b ; Luo et al. 2025 ; Yuan et al. 2025b ) , diffusion-based visual generation models ( Fan et al. 2023b ; Black et al. 2023 ; Zhou et al. 2025c ) , and unified multimodal frameworks ( Mao et al. 2025 ; Wang et al. 2024b ; Wang et al. 2025a ) , as shown in Figure 1 . Multimodal models such as Gemini 2.5 ( DeepMind 2025 ) have leveraged RL to align visual-textual reasoning processes and produce outputs with higher semantic coherence and alignment with human judgments. Concurrently, VLA models integrating vision and language with action-oriented outputs have adopted RL to optimize complex sequential decision-making processes in interactive environments, significantly improving task-specific performance in GUI automation ( Yuan et al. 2025b ; Shi et al. 2025 ) , robotic manipulation ( Lu et al. 2025b ) , and embodied navigation ( Kim et al. 2025 ) . The rapid advancement of diffusion-based generative models has further spurred this RL-driven innovation wave. Works like ImageReward ( Xu et al. 2023 ) have introduced reinforcement learning to enhance the semantic alignment and visual quality of generative outputs, refining diffusion-based generation through iterative feedback mechanisms derived from human preferences or automated reward critics. Moreover, unified models that blend multiple tasks, i.e., understanding, and generation, into single architectures ( Mao et al. 2025 ; Jiang et al. 2025b ) have increasingly relied on RL-driven fine-tuning, achieving generalization and task transfer previously considered challenging. Despite the substantial progress in integrating reinforcement learning with multimodal large language models, several core challenges remain. These include stabilizing policy optimization under complex reward signals, managing high-dimensional and diverse visual inputs, and designing scalable reward functions that support long-horizon decision-making. Addressing these challenges necessitates methodological innovations in both algorithmic design and evaluation protocols. In this survey, we present a comprehensive synthesis of recent advances in visual reinforcement learning within the context of multimodal large models, with a focus on the surge of research activity since 2024. We begin by revisiting foundational RL successes in language models, such as RLHF ( Ouyang et al. 2022 ) and DeepSeek-R1 ( Guo et al. 2025a ) , which have laid the groundwork for multimodal adaptation. Subsequently, we discuss how these strategies have evolved in the visual domain, categorizing over 200 representative works into four key domains: (i) multimodal large language models, (ii) visual generation, (iii) unified RL frameworks, and (iv) vision-language-action agents, as shown in Figure 1 . Within each category, we analyze key developments in algorithmic formulations, reward modeling, and benchmarking methodologies. Finally, we identify open challenges and future directions, highlighting the need for more efficient multimodal reasoning, robust long-horizon learning strategies for VLA tasks, and scalable, high-fidelity reward signals tailored to visual generation. Through this comprehensive overview, we offer a structured overview of visual reinforcement learning to support future research and practical deployment in this rapidly evolving field. Figure 1 : Timeline of Representative Visual Reinforcement Learning Models. The figure presents a chronological overview of key Visual RL models from 2023 to 2025, organized into four tracks: Multimodal LLM, Visual Generation, Unified Models, and VLA Models. Our key contributions are as follows: • We provide a systematic and up-to-date survey of over 200 visual reinforcement learning studies, encompassing MLLMs, visual generation, unified models, and vision-language-action agents. • We analyze advances in policy optimization, reward modeling, and benchmarking across subfields, revealing key challenges and future directions: such as reward design in visual generation and the lack of intermediate supervision in reasoning and VLA tasks. • We introduce a principled taxonomy of Visual RL methods based on metric granularity and reward supervision, including three reward paradigms for image generation. This framework clarifies the design trade-offs across domains and offers actionable insights for selecting and developing RL strategies. 2 Preliminary: Reinforcement Learning in LLM Table 1 : Glossary of Symbols for Visual Reinforcement Learning. It consolidates the notation that recurs across Sections § 2.1 – § 2.3 , with the rightmost column pointing to each appearance of the symbol. Symbol Alias Meaning Appears in p prompt User prompt (initial state) § 2.1 , § 2.2.1 , § 2.3.1 , § 2.3.2 a_{t} action Token / pixel patch / diffusion noise at step t § 2.1 , Eq. ( 8 ), § 2.3.2 y traj Full continuation (a_{1},\dots,a_{T}) § 2.1 , § 2.2.1 , § 2.2.3 y_{i} continuation i -th continuation in a GRPO group § 2.1 , § 2.3.2 s_{t} state Prompt plus previously generated actions § 2.1 , § 2.3.1 , § 2.3.2 \pi_{\theta} policy Trainable model (current parameters) § 2.1 , § 2.2.1 , § 2.2.2 , § 2.3.1 , § 2.3.2 \pi_{\theta_{\mathrm{old}}} behaviour policy Frozen policy that produced current batch Eq. ( 10 ), § 2.3.2 \pi_{\text{SFT}} SFT baseline Supervised-fine-tuned checkpoint § 2.2.1 , § 2.3.1 \pi_{\text{ref}} reference Policy used in KL regulariser § 2.1 , § 2.3.1 , § 2.3.2 \rho_{t} ratio Importance weight \pi_{\theta}/\pi_{\theta_{\mathrm{old}}} Eq. ( 10 ), § 2.3.2 V_{\psi} critic Value network predicting future return § 2.3.1 \hat{A}_{t} advantage GAE advantage (token-level) Eq. (