Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration

Yue Zhou, Litong Feng, Mengcheng Lan, Xue Yang, Qingyun Li, Yiping Ke, Xue Jiang, Wayne Zhang · 2025

Read Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration by Yue Zhou, Litong Feng, Mengcheng Lan, Xue Yang, Qingyun Li, Yiping Ke, Xue Jiang, Wayne Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-10059.txt.

Introduction

Yue Zhou and 7 co-authors build the first benchmark for mathematical reasoning over drone imagery in a 2025 vision-language paper. Remote sensing from small aircraft constantly needs distances, areas, and trajectory estimates, yet today's image-aware models had never been seriously examined on maths grounded in aerial vehicle views. Their AVI-Math collection holds nearly four thousand questions tied to vehicle scenes shot from varied heights and angles.

Questions span six school subjects and twenty topics, from geometry through logic and algebra, well past simple counting. Fourteen prominent vision-language models were run through the set, and even systems that shine on older benchmarks stumbled here. Careful error analysis maps exactly where the reasoning breaks, and the authors point to the gaps as the most useful agenda for the next round of model building.

Catalog record

CREATORYue Zhou, Litong Feng, Mengcheng Lan, Xue Yang, Qingyun Li, Yiping Ke, Xue Jiang, Wayne Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-10059

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multimodal Mathematical Reasoning Embedded in Aerial Vehicle Imagery: Benchmarking, Analysis, and Exploration Yue Zhou, Litong Feng, Mengcheng Lan, Xue Yang, Qingyun Li, Yiping Ke, Xue Jiang, Wayne Zhang Abstract Mathematical reasoning is critical for tasks such as precise distance and area computations, trajectory estimations, and spatial analysis in unmanned aerial vehicle (UAV) based remote sensing, yet current vision-language models (VLMs) have not been adequately tested in this domain. To address this gap, we introduce AVI-Math , the first benchmark to rigorously evaluate multimodal mathematical reasoning in aerial vehicle imagery, moving beyond simple counting tasks to include domain-specific knowledge in areas such as geometry, logic, and algebra. The dataset comprises 3,773 high-quality vehicle-related questions captured from UAV views, covering 6 mathematical subjects and 20 topics. The data, collected at varying altitudes and from multiple UAV angles, reflects real-world UAV scenarios, ensuring the diversity and complexity of the constructed mathematical problems. In this paper, we benchmark 14 prominent VLMs through a comprehensive evaluation and demonstrate that, despite their success on previous multimodal benchmarks, these models struggle with the reasoning tasks in AVI-Math . Our detailed analysis highlights significant limitations in the mathematical reasoning capabilities of current VLMs and suggests avenues for future research. Furthermore, we explore the use of Chain-of-Thought prompting and fine-tuning techniques, which show promise in addressing the reasoning challenges in AVI-Math . Our findings not only expose the limitations of VLMs in mathematical reasoning but also offer valuable insights for advancing UAV-based trustworthy VLMs in real-world applications. The code, and datasets will be released at https://github.com/VisionXLab/avi-math . [orcid=0000-0002-3080-6721] keywords Remote sensing (RS) ,Vision language model (VLM) ,Multi-modal large language model (MLLM) † † corresponding: Corresponding author. † † This work was partially conducted at Nanyang Technological University. This work was supported by National Natural Science Foundation of China (62506229), and Natural Science Foundation of Shanghai (25ZR1402268). 1 Introduction Mathematical reasoning plays a pivotal role in unmanned aerial vehicle (UAV) based remote sensing tasks such as precise distance measurement, area computation, trajectory estimation, and spatial analysis ( Alvarez-Vanhard et al. 2021 ; Banerjee and Corbetta 2020 ) . These tasks often require accurate and interpretable reasoning. While deep learning has achieved significant success in remote sensing (RS), its black-box nature raises concerns regarding transparency and reliability, particularly in high-stakes applications ( Höhl et al. 2024 ) . The advent of vision-language models (VLM) ( Yin et al. 2023 ) , which exhibit strong mathematical reasoning capabilities, offers a new approach to developing reliable autonomous UAV systems ( Wang et al. 2024d ) . Unlike traditional deep learning models, VLMs can offer a transparent reasoning process. To ensure the development of trustworthy UAV-based VLMs, it is crucial to rigorously assess their multimodal mathematical reasoning abilities. Several RS Visual Question Answering (VQA) datasets ( Lobry et al. 2020 ; Zheng et al. 2021 ; Zhang et al. 2023a ) have been developed to evaluate multimodal question answering systems. However, these datasets predominantly focus on visual perception tasks, with only a limited number of math-related questions. Furthermore, these math questions tend to be overly simplistic, often restricted to tasks like object counting or basic 2D spatial reasoning, which do not reflect the complexity of real-world RS applications that require domain-specific knowledge in areas like geometry, logic, and algebra. This lack of comprehensive evaluation for mathematical reasoning leaves a critical gap in assessing the true capabilities of current VLMs. To address this gap, we introduce AVI-Math , the first comprehensive multimodal benchmark specifically designed to evaluate mathematical reasoning in Aerial Vehicle Imagery (AVI). Unlike existing datasets, AVI-Math moves beyond simple counting tasks and incorporates domain-specific knowledge across six mathematical subjects — geometry, logic, statistics, arithmetic, counting, and algebra — ensuring a more rigorous and realistic assessment of VLMs’ capabilities. The benchmark comprises 3,773 high-quality vehicle-related questions, captured from UAV imagery across 11 distinct 4K resolution scenes. These scenes vary in terms of above-ground level (AGL) and pitch angles, simulating real-world UAV flight conditions. This diversity ensures that the data reflects the complexity of real-world UAV operations, challenging models to reason about mathematical problems under diverse and dynamic conditions. Given the complexity of the mathematical reasoning tasks in AVI-Math , we explore methods to enhance VLM performance. We introduce a new instruction fine-tuning dataset with 215k samples and employ LoRA ( Hu et al. 2021 ) to enable models to better integrate domain-specific knowledge. Furthermore, we investigate several Chain-of-Thought prompting techniques ( Wei et al. 2022 ; Wang et al. 2023 ; Wu et al. 2023 ; Mitra et al. 2024 ) , which encourage models to adopt a multi-step reasoning approach, more closely mimicking human cognitive processes. This work selects vehicles as the main subject and provides a preliminary exploration of mathematical problems in UAV imagery, without involving multisource RS images or complex sensor characteristics. As illustrated in Tab. 7 , GPT-4o demonstrates superior performance in five subjects. However, even the highest overall accuracy achieved is only 34.6%. We highlight the challenges that high-resolution UAV images pose to VLMs. We hope that AVI-Math will serve as a valuable resource, providing a benchmark for the future development of trustworthy multimodal UAV interpretation systems. In summary, our key contributions are as follows: • We introduce AVI-Math , the first multimodal benchmark for mathematical reasoning in UAV imagery, covering six subjects and real-world UAV scenarios. • We provide a comprehensive analysis, uncovering the limitations of current VLMs in mathematical reasoning and offering insights for future improvements. • We explore the potential of Chain-of-Thought prompting and fine-tuning techniques to enhance VLM performance, providing a 215k-sample instruction set for VLMs to learn domain-specific knowledge in UAV scenarios. 2 Related Work Table 1: Comparison between existing RS VQA datasets and our fine-tuning instruction set, AVI-Math-215k . Dataset Image Width #Questions #Subjects Rationale RSVQA-LR ( Lobry et al. 2020 ) 512 77,232 2 RSVQA-HR ( Lobry et al. 2020 ) 512 1,066,316 2 RSVQAxBEN ( Lobry et al. 2021 ) 20 \sim 120 14,758,150 0 FloodNet ( Rahnemoonfar et al. 2021 ) 4,000 11,000 1 RSIVQA ( Zheng et al. 2021 ) 256 \sim 4,000 111,134 2 CDVQA ( Yuan et al. 2022 ) 512 122,000 1 VQA-TextRS ( Al Rahhal et al. 2022 ) 256 \sim 600 6,245 1 CRSVQA ( Zhang et al. 2023a ) 600 4,644 2 RSIEval ( Hu et al. 2023 ) 512 936 2 EarthVQA ( Wang et al. 2024b ) 1,024 208,593 2 VRSBench ( Li et al. 2024 ) 512 123,221 2 AVI-Math-215k 4,000 215,658 6 Figure 1: Question types covered by AVI-Math . ARI: arithmetic, CNT: counting, ALG: algebra, STA: statistics, LOG: logic, GEO: geometry. Several benchmarks ( Lu et al. 2024b ; Wang et al. 2024c ; Liu et al. 2024b ) have been proposed to evaluate the multimodal mathematical reasoning capabilities of VLMs. However, these benchmarks primarily focus on abstract mathematical theory and computation, often relying on simple visual contexts like small figures, charts, or a few natural images. This limits their ability to test models in real-world, domain-specific settings such as RS, where understanding complex spatial relationships and geometric transformations is critical. Unlike these existing benchmarks, our work introduces a domain-specific multimodal mathematical reasoning benchmark that leverages high-resolution UAV

Read the complete text