Generalizable Geometric Image Caption Synthesis
Yue Xin, Wenyuan Wang, Rui Pan, Ruida Wang, Howard Meng, Renjie Pi, Shizhe Diao, Tong Zhang · 2025
Read Generalizable Geometric Image Caption Synthesis by Yue Xin, Wenyuan Wang, Rui Pan, Ruida Wang, Howard Meng, Renjie Pi, Shizhe Diao, Tong Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15217.txt.
Introduction
Yue Xin and seven co-authors synthesize training data for geometry problem solving with reinforcement-checked captions, in a 2025 machine learning paper. Multimodal models still stumble on geometry, partly because good image-text training pairs for diagrams are scarce. Template-based synthesis pipelines generate captions that rarely generalize beyond their templates.
The authors insert reinforcement learning with verifiable rewards into the data pipeline. Diagrams are synthesized from fifty basic geometric relations, captions are drafted, and a refinement step rewards captions that actually help solve mathematical problems. The reward signal comes from downstream problem-solving itself, so captions keep the features a solver needs. The resulting data generalizes better to fresh questions and yields clear improvements on geometry tasks, showing that data quality can be trained rather than merely filtered.
Catalog record
| CREATOR | Yue Xin, Wenyuan Wang, Rui Pan, Ruida Wang, Howard Meng, Renjie Pi, Shizhe Diao, Tong Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-15217
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalizable Geometric Image Caption Synthesis Yue Xin, Wenyuan Wang, Rui Pan, Ruida Wang, Howard Meng, Renjie Pi, Shizhe Diao, Tong Zhang Abstract Multimodal large language models have various practical applications that demand strong reasoning abilities. Despite recent advancements, these models still struggle to solve complex geometric problems. A key challenge stems from the lack of high-quality image-text pair datasets for understanding geometric images. Furthermore, most template-based data synthesis pipelines typically fail to generalize to questions beyond their predefined templates. In this paper, we bridge this gap by introducing a complementary process of Reinforcement Learning with Verifiable Rewards (RLVR) into the data generation pipeline. By adopting RLVR to refine captions for geometric images synthesized from 50 basic geometric relations and using reward signals derived from mathematical problem-solving tasks, our pipeline successfully captures the key features of geometry problem-solving. This enables better task generalization and yields non-trivial improvements. Furthermore, even in out-of-distribution scenarios, the generated dataset enhances the general reasoning capabilities of multimodal large language models, yielding accuracy improvements of 2.8%–4.8% in statistics, arithmetic, algebraic, and numerical tasks with non-geometric input images of MathVista and MathVerse, along with 2.4%-3.9% improvements in Art & Design and Tech & Engineering tasks in MMMU. * * footnotetext: Core contributors. $\dagger$ $\dagger$ footnotetext: Work done during a research internship at UIUC. 1 Introduction Multimodal Large Language Models (MLLMs) have exhibited impressive capabilities across a variety of vision-related tasks, including Visual Question Answering (VQA), visual grounding, and image captioning. Recent MLLMs, such as Qwen2.5-VL, Intern2.5-VL, and LLaVA-Next ( bai2025qwen2-5vl ; chen2024internvl ; liu2024llavanext ) , have shown superior performance compared to specialized vision models across a wide range of visual tasks, highlighting the potential of unified multimodal architectures. As the field advances, there has been increasing interest in enhancing the reasoning capabilities of MLLMs ( jaech2024openai ; shao2024visual ) , which is regarded as a crucial factor in extending the performance boundaries of these models. Among various reasoning tasks, mathematical reasoning ( zhang2024mathverse ) has attributed particular attention due to its structured problem-solving nature, offering a clear pathway for MLLMs to develop and improve their reasoning skills. Research from MathVerse ( zhang2024mathverse ) indicates that MLLMs perform best when the input is purely textual, while their performance declines significantly with visual-only inputs. This highlights the urgent need for MLLMs to develop strong cross-modal reasoning capabilities, which involves accurately and comprehensively transferring information from the image to the text. Although numerous geometry and math datasets have been introduced ( lu2023mathvista ; zhang2024mavis ; wang2024measuring ) to boost various aspects of model performance, high - quality datasets explicitly designed for cross - modal reasoning remain scarce. That is primarily because in existing datasets, the alignment between images and captions is often asymmetrical. For instance, in geometric problems, two lines of equal length can be easily described textually but may not be correspondingly annotated or visually distinct in the image. Such discrepancies hinder the model’s ability to learn robust cross-modal reasoning. Meanwhile, Reinforcement Learning (RL) has been demonstrated to significantly improve model reasoning and generalization capabilities ( guo2025deepseek ) . Its reward-driven framework is particularly effective for cross-modal reasoning, allowing models to optimize decision-making through interactive feedback ( deng2025openvlthinker ; peng2025lmm ; huang2025vision ) . Building on these insights, we employ the RAFT method ( Dong2023RAFT ) and design a reward function that incorporates both reasoning and caption rewards. This facilitates the alternating optimization of dataset quality and model reasoning abilities, leading to improved results on complex multimodal tasks. To bridge the gap between visual and linguistic modalities, we propose an RL-based data refinement engine that iteratively enhances data quality. Utilizing this pipeline, we introduce a novel geometry dataset comprising 10,000 image-caption pairs. To the best of our knowledge, this is the first high-quality dataset in which visual and textual information are fully aligned and generalize well to out-of-distribution tasks, making it a valuable resource for improving cross-modal reasoning. Experimental results demonstrate that our dataset significantly enhances models’ cross-modal reasoning abilities and their performance on geometric image textualization tasks. Furthermore, models trained on our dataset exhibit strong generalization capabilities on other mathematics-focused benchmarks, including MathVerse and MathVista. In summary, our main contributions are summarized as follows: • We introduce GeoReasoning-10K , a new dataset containing 10,000 carefully constructed image-caption pairs where visual and textual information are fully equivalent. This dataset serves as a high-quality resource for training cross-modal reasoning models. • We propose Geo-Image-Textualization , a scalable RL-based framework for generating and refining high-quality synthetic image-caption pairs in geometry. Our iterative RL-driven optimization significantly enhances data alignment and semantic accuracy, and demonstrates generalization to out-of-domain geometric tasks. • Extensive experiments show that the improvements facilitated by GeoReasoning extend beyond geometric tasks to non-geometric mathematical tasks and even to non-mathematical domains such as art and engineering. Figure 1: Examples of generalization . MLLMs learn from our synthetic geometric mathematical problems and generalize to algebraic cases with even non-geometric input images. 2 Related Works 2.1 Data Generation Recent studies have highlighted the scarcity of high-quality geometry image–caption datasets, which limits fine - grained cross - modal reasoning in geometric tasks. Following AlphaGeometry ( trinh2024solving ) , Autogeo ( huang2025autogeo ) proposed an automatic generation engine to produce image-caption pairs, constructing a 100K dataset named AutoGeo-100k. MATHGLANCE ( sun2025mathglance ) introduced GeoPeP, a perception-oriented dataset of 200K structured geometry image-text pairs explicitly annotated with geometric primitives and spatial relationships. MagicGeo ( wang2025magicgeo ) formulates diagram synthesis as a coordinate optimization problem, ensuring formal geometric correctness via solvers before coordinate - aware text generation. Despite the advances, existing pipelines still struggle to guarantee full modality alignment, i.e., captions frequently omit visual details, while images lack exhaustively aligned textual descriptions. 2.2 Image Captioning Image captioning aims to generate comprehensive descriptions of visual content and has been widely studied for natural images. While general-purpose multimodal large language models (MLLMs) such as mPLUG-Owl2 ( ye2024mplug ) , MiniGPT-4 ( zhu2023minigpt ) , and BLIP-3 ( xue2024xgen ) can perform captioning to some extent, their effectiveness is often limited by insufficient fine-grained cross-modal alignment. Image-Textualization ( pi2024image ) mitigates this issue by integrating multiple vision experts to produce more detailed and accurate captions. However, the potential of utilizing image captioning to enhance geometric reasoning capacity remains underexplored. OmniCaptioner ( lu2025omnicaptioner ) proposes a unified visual captioning framework that converts diverse images into fine - grained textual descriptions. Nonetheless, its geometric annotations are derived from AutoGeo and MAVIS, largely relying on synthetic or loosely aligned pairs rather than fully equivalent visual - textual representations. Moreover, the scarcity of high-quality geometric image-caption pairs makes it difficult to accurately extract and align geometric information. As a result, current models underperform on geometric image textualization compared to natural image captioning and general visual reasoning benchmarks. 3 Methods In this section, we introduce our Geo-Image-Textualization data generation pipeline first, followed by our RAFT method used for data refinement. 3.1 Geo-Image-Textualization Data Generation Engine The proposed data generation pipeline mainly contains three parts: the relation sampling, image-caption pair generation, and question-answer generation procedure, as shown in Figure. 3 . Figure 2: Symbolically synthesized geometric images . These geometry problems are symbolically composed from our relation library, corresponding to easy, medium, and hard difficulty levels,