EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards

Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman M Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Khan, Mohamed bin Zayed University of AI, Aalto University · 2025

Read EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards by Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman M Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Khan, Mohamed bin Zayed University of AI, Aalto University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-16672.txt.

Introduction

Omkar Thawakar and seven co-authors present EvoLMM, a way for a large multimodal model to improve its own reasoning with no labeled data and no external reward model. Two agents are instantiated from one backbone: a Proposer that invents diverse, image-grounded questions, and a Solver that answers them, with learning driven by internal consistency as a continuous self-reward.

The feedback loop rewards informative questions and cleaner reasoning structure at the same time. Built on Qwen2.5-VL, EvoLMM gained up to about three percent on multimodal math benchmarks including ChartQA, MathVista, and MathVision, using only raw training images. The authors offer it as a simple baseline for fully unsupervised self-improvement of multimodal models, with code and models released.

Catalog record

CREATOROmkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman M Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Khan, Mohamed bin Zayed University of AI, Aalto University
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-16672

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards Omkar Thawakar, Shravan Venkatraman, Ritesh Thawkar, Abdelrahman M Shaker, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Fahad Khan, Mohamed bin Zayed University of AI, Aalto University Abstract Recent advances in large multimodal models (LMMs) have enabled impressive reasoning and perception abilities, yet most existing training pipelines still depend on human-curated data or externally verified reward models, limiting their autonomy and scalability. In this work, we strive to improve LMM reasoning capabilities in a purely unsupervised fashion (without any annotated data or reward distillation). To this end, we propose a self-evolving framework, named EvoLMM, that instantiates two cooperative agents from a single backbone model: a Proposer, which generates diverse, image-grounded questions, and a Solver, which solves them through internal consistency, where learning proceeds through a continuous self-rewarding process. This dynamic feedback encourages both the generation of informative queries and the refinement of structured reasoning without relying on ground-truth or human judgments. When using the popular Qwen2.5-VL as the base model, our EvoLMM yields consistent gains upto ∼ \sim 3% on multimodal math-reasoning benchmarks, including ChartQA, MathVista, and MathVision, using only raw training images. We hope our simple yet effective approach will serve as a solid baseline easing future research in self-improving LMMs in a fully-unsupervised fashion. Our code and models are available at https://github.com/mbzuai-oryx/EvoLMM . $\dagger$ $\dagger$ footnotetext: Equal contribution. 1 Introduction Figure 1 : Illustration of our fully unsupervised self-evolving LMM framework (EvoLMM). Our EvoLMM enables a base LMM to improve its reasoning ability without any human labels, metadata, or external reward models . Given only raw images, a proposer first generates visually grounded questions, and a solver attempts to answer them multiple times. The degree of agreement among solver responses produces a continuous self-consistency reward , forming a closed-loop training signal that drives both modules to co-evolve. Large multimodal models (LMMs) that jointly handle vision and language have made substantial progress in recent years. They excel at tasks like image captioning 22 ; 14 ; 29 ; 16 , visual question answering 42 ; 13 ; 51 ; 28 , and multimodal reasoning 43 ; 46 ; 37 ; 30 ; 34 by leveraging large-scale pre-training and fine-tuning on annotated datasets. However, despite these advances, two fundamental dependencies remain: (i) the dependence on human-curated annotations or metadata 11 ; 50 ; 1 ; 8 ; 19 , and (ii) the dependency on externally defined reward models or evaluators (e.g., human-labeled correctness, knowledge-distilled reward networks) 36 ; 4 ; 32 ; 12 ; 17 . These dependencies pose scalability limitations, hinder domain generalization, and restrict deployment in settings where annotations or reward models are unavailable. Recent research in large language models (LLMs) explores the idea of self-evolving models that not only solve tasks but also generate the tasks they learn from. For instance, few works 5 ; 6 aim to improve base LLM by generating its own questions and attempting to answer them, using the agreement among multiple sampled answers as a self-reward signal. Such a technique can be further improved 6 by splitting the model into interacting roles such as Proposer , Solver , and Judge that co-exist and train themselves through reinforcement learning. This research direction is pushed further 45 by incorporating open-ended scheme where the system incorporates an open-ended evolution to continually refine internal behaviors. However, these research efforts are largely constrained to the language-only domain, where reasoning is explicit and correctness can be inferred through textual consistency or symbolic execution. Moreover, these approaches still either rely on implicit evaluators (e.g., external Judge module, execution feedback) or explicit evaluators, implying that the learning signal is not yet fully free of external priors. In the multimodal (LMM) domain, few recent works aim to improve LMMs through self-rewarding training signals, thereby reducing human supervision. For instance, the work of 18 proposes a self-rewarding reasoning strategy where the model decomposes each task into a perception step and a reasoning step, and then evaluates with internal bootstrapping. Similarly, ViPER 47 introduces a closed-loop framework that gradually strengthens visual perception ability through image-level and instance-level reconstruction, combined with reinforcement style updates driven by internally synthesized data. While these approaches represent progress in self-improving LMMs, they still rely on structured intermediate signals such as reconstruction objectives, heuristic quality filtering, or semantic similarity-based scoring. Consequently, their self-evolution remains partially guided by external priors rather than emerging purely from the model’s own reasoning dynamics. In this work, we investigate a more fundamental form of autonomy: Can a large multimodal model (LMM) enhance its own reasoning capability without relying on human-annotated supervision or any external reward signals? To this end, we propose a purely unsupervised self-evolving framework, named EvoLMM, that learns entirely through internal consistency (see Figure. 1 ). Our approach instantiates two cooperative roles from a single backbone model: a Proposer , which generates diverse and visually grounded mathematical questions from unlabeled images, and a Solver , which attempts to solve these questions. The framework evolves through a continuous self-reward mechanism, where the Solver is rewarded by the fraction of consistent or correct answers within its own generations. It is worth mentioning that no ground-truth labels, metadata, or external reward models are used at any stage. This simple yet effective design enables the model to construct, evaluate, and refine its own training signal directly from raw multimodal inputs. The framework naturally scales to large models and domains where ground-truth supervision is unavailable or expensive. Despite training exclusively on raw images, we show consistent improvements across multiple multimodal math reasoning benchmarks over the base model. Further, our qualitative analysis reveals that the Proposer learns to generate increasingly complex visual problems, while the Solver develops more structured reasoning chains over time, suggesting emergent self-evolution. In summary, our main contributions are: • We introduce a self-evolving multimodal framework, named EvoLMM, that enables a base LMM to improve without human labels, metadata, or external reward models. The framework decomposes the model into two internal roles, Proposer and Solver , forming a closed-loop propose-solve cycle trained solely through internal consistency feedback. • We develop a continuous self-rewarding mechanism based on multi-sample answer consistency, which replaces both learned discrete reward models and semantic similarity scoring used in prior LMM self-evolution approaches. This continuous internal reward signal provides smooth gradients and stable optimization, enabling consistent improvement in performance. • We empirically validate EvoLMM on mathematical visual reasoning benchmarks, with absolute gains of \sim 2–3\% over the Qwen-2.5-VL-7B baseline using only raw images during training. We further analyze the evolution of our propose-solve mechanism where the difficulty level gradually progresses and maintains stable learning, showing that the model naturally develops more structured and grounded reasoning behaviors over time. Furthermore, we show that internal consistency can serve as a viable supervision signal for open-ended multimodal learning. 2 Related Work In the context of large language models (LLMs), several recent works 5 ; 31 ; 6 aim to self-improve the model capabilities (e.g., scientific reasoning) without significant human annotations. The work of 5 proposes an approach to improve the implicit reasoning capabilities of the base model through self-play and majority-vote reward signals. Sheikh 31 explores self-consistency as a training reward within an reinforcement learning (RL) framework, demonstrating

Read the complete text