GeoFM: Enhancing Geometric Reasoning of MLLMs via Synthetic Data Generation through Formal Language

Yuhao Zhang , Dingxin Hu , Tinghao Yu , Hao Liu , Yiting Liu Tencent Hunyuan Team {yuhaozzhang,dingxinhu,maxwellyu,paulhliu,timytliu}@tencent.com · 2025

Read GeoFM: Enhancing Geometric Reasoning of MLLMs via Synthetic Data Generation through Formal Language by Yuhao Zhang , Dingxin Hu , Tinghao Yu , Hao Liu , Yiting Liu Tencent Hunyuan Team {yuhaozzhang,dingxinhu,maxwellyu,paulhliu,timytliu}@tencent.com (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27448.txt.

Introduction

Yuhao Zhang, Dingxin Hu, and three other co-authors make fresh geometry problems for AI training in this 2025 paper. Models that see images and text still stumble on geometry, mostly for lack of good practice data. Older ways to fake data reword old problems or follow fixed templates. The results are narrow, noisy, or unlike real diagrams.

Their method, GeoFM, writes problems in a formal language, a strict notation the computer can check. It mixes and matches conditions in metric space, the space of lengths and angles. A symbolic engine, a rule-based checker, verifies each new problem as it is made. Models trained on this data did far better than those trained on older fakes. On one geometry test, their model beat the commercial GPT-4o by a wide margin.

Catalog record

CREATORYuhao Zhang , Dingxin Hu , Tinghao Yu , Hao Liu , Yiting Liu Tencent Hunyuan Team {yuhaozzhang,dingxinhu,maxwellyu,paulhliu,timytliu}@tencent.com
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-27448

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GeoFM: Enhancing Geometric Reasoning of MLLMs via Synthetic Data Generation through Formal Language Yuhao Zhang , Dingxin Hu , Tinghao Yu , Hao Liu , Yiting Liu Tencent Hunyuan Team {yuhaozzhang,dingxinhu,maxwellyu,paulhliu,timytliu}@tencent.com Abstract Multi-modal Large Language Models (MLLMs) have gained significant attention in both academia and industry for their capabilities in handling multi-modal tasks. However, these models face challenges in mathematical geometric reasoning due to the scarcity of high-quality geometric data. To address this issue, synthetic geometric data has become an essential strategy. Current methods for generating synthetic geometric data involve rephrasing or expanding existing problems and utilizing predefined rules and templates to create geometric images and problems. However, these approaches often produce data that lacks diversity or is prone to noise. Additionally, the geometric images synthesized by existing methods tend to exhibit limited variation and deviate significantly from authentic geometric diagrams. To overcome these limitations, we propose GeoFM, a novel method for synthesizing geometric data. GeoFM uses formal languages to explore combinations of conditions within metric space, generating high-fidelity geometric problems that differ from the originals while ensuring correctness through a symbolic engine. Experimental results show that our synthetic data significantly outperforms existing methods. The model trained with our data surpass the proprietary GPT-4o model by 18.7% on geometry problem-solving tasks in MathVista and by 16.5% on GeoQA. Additionally, it exceeds the performance of a leading open-source model by 5.7% on MathVista and by 2.7% on GeoQA. 1 Introduction Figure 1: Comparison of different methods for synthesizing geometric data. (a) Generate geometric Q&A data by using MLLMs to rephrase existing problems or create new Q&A from collected geometric images. (b) Utilize a rule-based data engine to generate template-based Q&A and low-fidelity images. (c) Employ formal language to explore the combinations of geometric metric conditions and synthesize new problems, ensuring solution accuracy through symbolic reasoning, and generate high-fidelity geometric images. Large language models (LLMs) exhibit excellent reasoning capabilities. There has been a significant amount of research dedicated to applying large language models to solve text-based mathematical problems, resulting in substantial progress ( OpenAI 2024 ; Luo et al. 2023 ; Shao et al. 2024 ; Yang et al. 2024 ) . Recently, there has also been a growing focus on using Multi-modal Large Language Models (MLLMs) to address multi-modal mathematical problems that include images ( Gao et al. 2023 ; Shi et al. 2024 ; Zhang et al. 2024a ; Li et al. 2024a ) . Although MLLMs perform well in general tasks such as Visual Question Answering (VQA), their performance often falls short when tackling multi-modal mathematical problems ( Lu et al. 2024 ; Wang et al. 2024a ) . In particular, geometry problems, which are a typical example of multi-modal mathematical problems with wide-ranging applications, require the integration of both visual and textual information for reasoning and solution. However, MLLMs struggle with these problems. One of the primary reasons for this difficulty is the lack of high-quality geometric data for training MLLMs. Compared to natural scene tasks like VQA, the sources and quantity of geometric data are relatively limited, which hinders the advancement of MLLMs’ abilities in geometry. To address the shortage of geometric data, some approaches have employed synthetic data generation. A straightforward method involves rewriting the problem statements and answers ( Gao et al. 2023 ) . However, simple rewrites do not alter the underlying meaning of the problems. Although this increases the quantity of problems, it does not enhance the diversity. Other approaches have attempted to use MLLMs to modify original geometric problems and generate answers ( Gao et al. 2023 ) , or to directly create new problems and corresponding responses based on collected geometric images ( Shi et al. 2024 ) , as shown in Figure 1 (a). Nevertheless, these methods rely on the geometric reasoning capabilities of MLLMs. Given the current limitations of MLLMs in solving geometric problems, these approaches are prone to introducing noise into the synthetic data. Recently, there have been attempts to synthesize geometric problems using predefined rules and templates ( Kazemi et al. 2023 ; Zhang et al. 2024a ) . For example, new shapes are generated by continuously extending basic geometric figures such as triangles and quadrilaterals outward along their edges. The reasoning paths and final answers are obtained through programming, as illustrated in Figure 1 (b). While this method ensures the correctness of the reasoning and answers, the low fidelity of the synthesized images and the restricted variety of problems resulting in a significant disparity from real geometric problems. This discrepancy limits the progress of MLLMs in developing geometric capabilities. To address the challenges present in current approaches, we propose a novel method for synthesizing geometric data. We have observed that existing geometric datasets often associate a single geometric diagram with only one or two problems, despite the fact that geometric diagrams often contain rich metric information that are not fully covered by the existing problems. Therefore, we propose GeoFM, a method that employs formal languages to explore the combinations of conditions within metric spaces of geometric diagrams, thereby generating high-fidelity geometric problems differ from the original ones but whose correctness is guaranteed using a symbolic engine. Existing work on geometric formal languages is scattered across different fields, such as geometric problem solving ( Lu et al. 2021 ; Peng et al. 2023 ; Zhang et al. 2024b ) , theorems proving ( Trinh et al. 2024 ; Chervonyi et al. 2025 ) and geometric drawing ( Krueger et al. 2021a ) . Furthermore, these studies frequently necessitate human intervention, such as manual formalization, to accomplish the associated tasks ( Zhang et al. 2024b ; Krueger et al. 2021a ) , which prevents their application for large-scale automatic synthesis of geometric data. To address this issue, we propose a comprehensive framework for geometric data synthesis that automates the formalization of seed problems, the synthesis of new problems, and the generation of images. Utilizing this approach, we have developed a highly accurate and realistic geometric synthetic dataset GeoFM80K. Experimental results demonstrate our synthetic data can effectively enhance the geometric reasoning capabilities of MLLMs. Our contributions are summarized as follows: 1. We propose GeoFM, a geometric data synthesis method using formal languages and symbolic reasoning to generate accurate solutions and new geometric diagrams, addressing data noise and discrepancies in existing data synthesis methods. 2. We introduce a strategy for synthesizing new geometric problems through the combination of geometric metric conditions, resulting in the GeoFM80K dataset. Models trained on GeoFM80K outperform those trained on representative synthetic data by 8.2% on MathVista-GPS ( Lu et al. 2024 ) and 11.1% on GeoQA ( Chen et al. 2021 ) . 3. Experimental results show our method enhances the geometric reasoning of MLLMs. The GeoFM-8B model surpasses GPT-4o by 18.7% on MathVista-GPS and 16.5% on GeoQA, and exceeds the best open-source model by 5.7% on MathVista-GPS and 2.7% on GeoQA. Figure 2: The Framework of Geometric Data Synthesis GeoFM 2 Method 2.1 Overview In this section, we present our method for generating synthetic geometric problems. We start by converting seed problems into a formal language for problem-solving. New problems are created by combining metric conditions from the seed problems and solved using symbolic

Read the complete text