Ariadne: A Controllable Framework for Probing and Extending VLM Reasoning Boundaries

Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu · 2025

Read Ariadne: A Controllable Framework for Probing and Extending VLM Reasoning Boundaries by Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00710.txt.

Introduction

Minghe Shen and five co-authors ask whether reinforcement learning with verifiable rewards truly expands what vision-language models can do, or merely amplifies what pretraining already made likely. In text-only studies the pessimistic reading has support, but visual spatial reasoning stayed unexplored. Their controlled framework, Ariadne, generates synthetic mazes whose difficulty is set precisely by path length and number of turns.

After reward-based training, the model solves mazes where the base policy scores zero accuracy even with generous repeated sampling, evidence of navigating search spaces the base distribution could not reach. Trained only on synthetic mazes, the model also improves zero-shot on two real-world navigation benchmarks, MapBench and ReasonMap. Genuine capability expansion, not just sampling efficiency, is the authors' verdict.

Catalog record

CREATORMinghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00710

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Ariadne: A Controllable Framework for Probing and Extending VLM Reasoning Boundaries Minghe Shen, Zhuo Zhi, Chonghan Liu, Shuo Xing, Zhengzhong Tu, Che Liu Abstract While Vision-Language Models (VLMs) post-trained with Reinforcement Learning (RL) show impressive general reasoning, their evaluation is often confined to language-dominant tasks (e.g., math). This raises a critical question: can RL post-training truly extend the inherent capability boundary of a base VLM, particularly for visual-centric spatial tasks where it initially fails? To investigate this, we introduce Ariadne , a framework utilizing synthetic mazes for multi-step spatial reasoning with precisely controlled task difficulty (e.g., path length, turns). We employ this controllable setup to train VLMs using Reinforcement Learning with Verified Rewards (RLVR) under a difficulty-aware curriculum. Surprisingly, after RLVR training, the VLM attains over 50% accuracy on tasks where the base model scored 0% , indicating that RLVR can extend the base policy’s capability boundary within, but not fully beyond, the training data scope. Specifically, it generalizes to unseen numbers of turns but fails when the number of movement steps exceeds training exposure, revealing a divergent phenomenon in spatial reasoning. Furthermore, we observe that in real-world settings, this boundary shifts: the model handles longer-step reasoning, likely due to noise-induced redundancy in natural scenarios. Despite being trained solely on synthetic mazes, Ariadne still yields notable zero-shot gains, averaging 16% on MapBench (e.g., museum navigation) and 24% on ReasonMap (subway transfer tasks). These results suggest that our method not only broadens the model’s intrinsic limits but also enhances its generalization to realistic spatial reasoning tasks. We note that our study is confined to the post-training phase due to the opacity of pre-training data and hope this work inspires further exploration into specialized, capability-extending alignment. Codes, models, and resources are available at https://mingheshen.github.io/Ariadne. \project https://mingheshen.github.io/Ariadne † † affiliation: University College London † † affiliation: University of California, Los Angeles † † affiliation: Texas A&M University † † affiliation: Imperial College London † † contribution: Corresponding author † † date: August 10, 2026 † † correspondence: [email protected] 1 Introduction Reinforcement Learning (RL) has played a pivotal role in advancing the reasoning capabilities of large language models (LLMs) 10.5555/3600270.3602281 ; Schulman2017ProximalPO . Recent work, such as DeepSeek-R1 deepseekai2025 , demonstrated that Group Relative Policy Optimization (GRPO), trained with Reinforcement Learning with Verified Rewards (RLVR) and without a separate reward model, can effectively enhance complex reasoning abilities deepseekai2025 ; shao2024deepseekmath . Extending this paradigm to the vision-language domain huang2025vision ; wang2025vl ; wan2025srpo , Vision-Language Models (VLMs) show strong reasoning potential after RL post-training. However, their evaluation remains largely confined to language-dominant tasks such as mathematics. A key question thus arises: Can RLVR extend the inherent capability boundary of VLMs, particularly in visual-centric spatial reasoning? To create a controllable scenario for evaluation, we select visual-spatial reasoning tasks such as path-finding mazes, which are well-suited for this study because they provide deterministic ground-truth paths for verifiable rewards dao2025alphamaze ; mirowski2017learning . Moreover, maze complexity can be precisely adjusted through parameters such as path length and number of turns, enabling systematic probing of model reasoning behavior. To this end, we introduce Ariadne , a fully controllable framework designed to probe and extend VLM reasoning boundaries. Ariadne enables fine-grained control over maze complexity while maintaining clear difficulty definitions and verifiable rewards. This design allows transparent analysis of a base VLM’s capability boundary and how RLVR extends it. Furthermore, we transfer RLVR-trained VLMs to real-world benchmarks for zero-shot evaluation, exploring whether reasoning learned from synthetic mazes generalizes to practical tasks. The main contributions of this work are summarized as follows: • We propose Ariadne , a controllable framework for probing and extending VLM reasoning boundaries via RLVR on synthetic maze tasks with precisely tunable difficulty. • Ariadne achieves over 50% accuracy on maze problems where the base model scored 0%, indicating that RLVR can extend the base policy’s capability boundary within but not fully beyond the training domain , generalizing to unseen numbers of turns but failing on longer step sequences. • We identify a divergent phenomenon where real-world reasoning boundaries deviate from synthetic ones, as models generalize better to longer-step reasoning in noisy, natural environments. • Despite being trained solely on synthetic mazes, Ariadne delivers consistent zero-shot gains on real-world benchmarks, MapBench xing2025can (16%) and ReasonMap feng2025can (24%), highlighting its effectiveness in enhancing spatial reasoning beyond training conditions. 2 Related Work 2.1 Reasoning in VLMs VLMs have achieved substantial progress on tasks such as Visual Question Answering (VQA) and image captioning alayrac2022flamingo ; liu2023visual ; zhu2025internvl3 . To strengthen reasoning capabilities, prior work has explored Chain of Thought (CoT) prompting wei2022chain and the construction of Supervised Fine-Tuning (SFT) datasets enriched with step-level rationales zhu2025internvl3 . While these strategies enhance reasoning to some extent, they fall short of capturing human-like cognitive processes such as questioning and self-verification, and their effectiveness diminishes on complex reasoning tasks. Recent models such as DeepSeek-R1 shao2024deepseekmath attempts to address this gap by combining cold-start initialization with RL, thereby acquiring higher-quality multimodal CoT reasoning and achieving state-of-the-art results on challenging visual reasoning benchmarks. 2.2 Reasoning with RL Despite the advances in VLMs, spatial and multi-step reasoning, particularly in navigation and spatial understanding, remains a key challenge. To overcome this limitation, researchers have increasingly adopted RL-based approaches. Ji et al. applied GRPO with structured CoT supervision to spatial VQA and navigation tasks, showing that verifiable, rule-based rewards can significantly improve reasoning robustness ji2025enhancing . Similarly, CoT-VLA extended CoT reasoning to vision–language–action models, where explicit intermediate reasoning steps yielded strong gains in robotic navigation zhao2025cot . In synthetic navigation domains, Mirowski et al. enhanced RL training with auxiliary tasks such as depth prediction and loop-closure detection, enabling efficient traversal of complex 3D mazes mirowski2017learning . While these approaches have shown effectiveness, most rely on large-scale synthetic data or are narrowly tailored to specific architectures. Open questions remain regarding how VLMs adapt their reasoning strategies when systematically exposed to increasing task complexity and to what extent such improvements generalize beyond synthetic environments feng2025can ; xing2025can . In this work, we introduce a controllable training framework, Ariadne, for exploring the visual reasoning boundary of VLMs. Ariadne enables precise manipulation of problem complexity while maintaining consistent logical structures and verifiable rewards, allowing both targeted capability enhancement and systematic analysis of generalization and failure modes. Finally, through evaluation on real-world map navigation datasets, we demonstrate that reasoning gains achieved in synthetic maze environments can effectively transfer to practical out-of-domain scenarios. 3 Methodology 3.1 Preliminaries of GRPO Recent advancements in enhancing the reasoning capabilities of vision-language models (VLMs) have demonstrated that RL is a powerful training strategy shao2024deepseekmath . In this study, we adopt GRPO as our learning framework. GRPO operates by directly comparing groups of candidate responses, thereby eliminating the need for a separate critic model. During training, GRPO generates a set of candidate outputs {o_{1},o_{2},\dots,o_{G}} for a given question q drawn from the dataset \mathcal{D} using the old policy \pi_{\theta_{\text{old}}} . Each candidate response o_{i} is then evaluated using a reward function R(o_{i},q) , yielding a reward r_{i} . To assess the relative quality of the responses within the group, GRPO normalizes the rewards by computing their mean and standard deviation. The advantage A_{i} for each response is then

Read the complete text