Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play

Qinsi Wang 1 , Bo Liu 2 , Tianyi Zhou 3 , Jing Shi 4 , Yueqian Lin 1 , Yiran Chen 1 , Hai Helen Li 1 , Kun Wan 4 , Wentian Zhao 4* · 2025

Read Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play by Qinsi Wang 1 , Bo Liu 2 , Tianyi Zhou 3 , Jing Shi 4 , Yueqian Lin 1 , Yiran Chen 1 , Hai Helen Li 1 , Kun Wan 4 , Wentian Zhao 4* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25541.txt.

Introduction

Qinsi Wang and eight co-authors present Vision-Zero, a way for vision language models to improve by playing games against each other. Current training leans on hand built data sets and costly human checks, which limits growth. Their work came out in twenty twenty-five.

In Vision-Zero, models play a Who Is the Spy style game that calls for planning and role based thought. Through play, the models make their own training data with no human labels. Games can start from any image, not just a fixed set. The team shows this range with made up scenes, charts, and real world photos, and reports steady gains that carry across tasks.

Catalog record

CREATORQinsi Wang 1 , Bo Liu 2 , Tianyi Zhou 3 , Jing Shi 4 , Yueqian Lin 1 , Yiran Chen 1 , Hai Helen Li 1 , Kun Wan 4 , Wentian Zhao 4*
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-25541

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play Qinsi Wang 1 , Bo Liu 2 , Tianyi Zhou 3 , Jing Shi 4 , Yueqian Lin 1 , Yiran Chen 1 , Hai Helen Li 1 , Kun Wan 4 , Wentian Zhao 4* Abstract Although reinforcement learning (RL) can effectively enhance the reasoning capabilities of vision–language models (VLMs), current methods remain heavily dependent on labor-intensive datasets that require extensive manual construction and verification, leading to extremely high training costs and consequently constraining the practical deployment of VLMs. To address this challenge, we propose Vision-Zero , a domain-agnostic framework enabling VLM self-improvement through competitive visual games generated from arbitrary image pairs. Specifically, Vision-Zero encompasses three main attributes: (1) Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the Spy"-style games, where the models engage in strategic reasoning and actions across multiple roles. Through interactive gameplay, models autonomously generate their training data without human annotation. (2) Gameplay from Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate games from arbitrary images, thereby enhancing the model’s reasoning ability across diverse domains and showing strong generalization to different tasks. We demonstrate this versatility using three distinct types of image datasets: CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable Performance Gain: We introduce Iterative Self-Play Policy Optimization (Iterative-SPO), a novel training algorithm that alternates between Self-Play and reinforcement learning with verifiable rewards (RLVR), mitigating the performance plateau often seen in self-play-only training and achieving sustained long-term improvements. Despite using label-free data, Vision-Zero achieves state-of-the-art performance on reasoning, chart question answering, and vision-centric understanding tasks, surpassing other annotation-based methods. Models and code has been released at here . 1 Introduction Recent breakthroughs in vision-language models (VLMs) have demonstrated remarkable capabilities across diverse multimodal tasks ( Achiam et al. 2023 ; Team et al. 2023 ) . However, current training paradigms face fundamental scalability constraints: they depend heavily on human-curated data through supervised fine-tuning (SFT) ( Liu et al. 2023 ) , reinforcement learning from human feedback (RLHF) ( Ouyang et al. 2022 ; Sun et al. 2023 ) , and carefully engineered reward functions for reinforcement learning with verifiable rewards (RLVR) ( Guo et al. 2025 ) . This dependency creates two critical bottlenecks. First, a data scarcity problem —the extraordinary cost of multimodal annotation limits both scale and diversity of training data, with datasets like COCO Attributes requiring $60,480 for 200,000 objects ( Patterson & Hays 2016 ) , Ego4D consuming over 250,000 annotation hours ( Grauman et al. 2022 ) , and Visual Genome mobilizing 33,000 annotators ( Krishna et al. 2017 ) . Second, a knowledge ceiling —model capabilities remain fundamentally bounded by human-generated supervision, preventing VLMs from discovering strategies beyond human expertise. Self-Play offers a solution by eliminating human supervision through competitive dynamics ( Silver et al. 2017 ; Tesauro 1995 ) . In self-play, models learn by engaging in competitive interactions with copies of themselves, receiving automatic feedback based on the outcomes of each interaction. As the model improves, its opponents correspondingly advance, thus maintaining a consistently challenging learning environment and driving continuous improvement. By removing the need for human supervision during data generation, self-play has already surpassed the knowledge ceiling across many domains: from TD-Gammon’s backgammon supremacy ( Tesauro 1995 ) to AlphaGo’s conquest of Go ( Silver et al. 2016 ; Silver et al. 2017 ) to OpenAI Five’s mastery of complex team coordination ( Berner et al. 2019 ) . With the growing capabilities of LLMs, recent work has begun to import Self-Play into LLMs training to reduce dependence on human intervention. These approaches construct Language Gamification frameworks wherein LLMs compete under clearly defined game rules, incrementally enhancing their competencies. For example, SPIRAL enhances LLM reasoning by having models play games such as Tic-Tac-Toe and Kuhn Poker ( Liu et al. 2025 ) ; Absolute Zero frames self-play between proposer and solver ( Zhao et al. 2025 ) , achieving state-of-the-art results on mathematics and coding tasks. However, extending self-play to VLMs remains largely unexplored, despite the prohibitive costs of multimodal data that make such an approach particularly urgent. An ideal self-play game environment should satisfy the following four conditions: (1) The skills acquired by agents in order to win the game should closely align with those required by the target tasks. (2) Skill growth should be scalable: as self-play progresses, the environment should continually escalate difficulty so that ever stronger agents can emerge rather than converging to a fixed upper bound. (3) The environment should be sufficiently diverse and complex to enable a wide range of target tasks can satisfy conditions (1). (4) The environment should require no external data or only a small amount of low-cost data, such as label-free data. To the best of our knowledge, existing visual reasoning games fail to satisfy all of the above criteria simultaneously. For instance, Sudoku satisfies conditions (2) and (4), but fails to meet (1) and (3). Due to the multimodal nature of VLMs, designing a self-play environment that fulfills all four conditions requires joint consideration of both vision and language modalities, which is non-trivial. Inspired by language-based social deduction games, particularly those involving alternating rounds of statements and voting such as “Who Is the Spy?” , we propose a novel visual reasoning game that addresses these four requirements. (a) Supervised Learning (b) Reinforcement Learning (c) Vision-Zero Figure 1: Vision-Zero Paradigm. (a) Supervised learning depends on human-curated reasoning trajectories; (b) Reinforcement Learning, although enabling models to autonomously learn reasoning processes via validated rewards, still relies heavily on expert-designed question-answer pairs. (c) In contrast, Vision-Zero is a novel self-improvement paradigm entirely independent of human experience. It constructs self-play games by leveraging image pairs that exhibit visual differences. Through the interactive and strategic game, Vision-Zero continuously generates training data for VLMs, enabling the model to achieve scalable self-improvement. We present Vision-Zero, the first gamified self-play framework that enables scalable self-improvement of VLMs without requiring human annotations. We design a visual “Who Is the Spy?” game based on subtly differing image pairs, which are generated either by an automated image editing tool or rendered procedurally. By reasoning over and hypothesizing about these subtle differences, agents gradually acquire stronger visual reasoning capabilities. This setup compels models to engage in strategic reasoning across multiple roles while handling diverse visual inputs such as CLEVR scenes ( Johnson et al. 2017 ) , charts, and natural images. We further propose Iterative Self-Play Policy Optimization (Iterative-SPO), which alternates between Self-Play and RLVR. By incorporating verifiable supervision into self-play, Iterative-SPO stabilizes training and prevents premature convergence to equilibrium states, thereby ensuring consistent performance gains within the Vision-Zero framework. Vision-Zero provides a domain-agnostic framework that effectively leverages diverse image inputs, enabling continuous improvement without reliance on task-specific datasets. Through a carefully designed strategic visual gameplay, it strengthens reasoning, spatial understanding, and visual comprehension while reducing shortcut bias from text and negative capability transfer that are prevalent in conventional VLM training methods. Moreover, its reliance on automated image editing supports highly cost-efficient dataset construction. As shown in Fig. 2 , Vision-Zero simultaneously enhances performance across tasks including reasoning, chart/OCR, and vision-centric tasks, surpassing state-of-the-art baselines trained on expensive human-labeled datasets. These results underscore Vision-Zero’s substantial potential and

Read the complete text