A Survey on Large Language Models for Mathematical Reasoning
Peng-Yuan Wang 1,2* , Tian-Shuo Liu 1,2* , Chenyang Wang 1,2 , Yi-Di Wang 1,2 , Shu Yan 1 , Cheng-Xing Jia 1,2 , Xu-Hui Liu 1,2 , Xin-Wei Chen 3 , Jia-Cheng Xu 4,5 , Ziniu Li 6 , Yang Yu 1,2, ⋄ \diamond 1 National Key Laboratory for Novel Software Technology, Nanjing University, China 2 School of Ar · 2025
Read A Survey on Large Language Models for Mathematical Reasoning by Peng-Yuan Wang 1,2* , Tian-Shuo Liu 1,2* , Chenyang Wang 1,2 , Yi-Di Wang 1,2 , Shu Yan 1 , Cheng-Xing Jia 1,2 , Xu-Hui Liu 1,2 , Xin-Wei Chen 3 , Jia-Cheng Xu 4,5 , Ziniu Li 6 , Yang Yu 1,2, ⋄ \diamond 1 National Key Laboratory for Novel Software Technology, Nanjing University, China 2 School of Ar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-08446.txt.
Introduction
Peng-Yuan Wang and a large team survey how large language models handle math reasoning in a 2025 paper. Math has long been a hard test for artificial intelligence. Recent models have made clear gains, and the survey maps how those gains came about.
The authors split the work into two stages: first gaining math understanding in training, then producing answers. Answer methods have moved from direct guesses to step-by-step reasoning and longer test-time work. They review prompting, fine-tuning, and learning from rewards. Gaps remain in capacity, cost, and how well skills transfer. The survey points to formal reasoning and better training as next steps.
Catalog record
| CREATOR | Peng-Yuan Wang 1,2* , Tian-Shuo Liu 1,2* , Chenyang Wang 1,2 , Yi-Di Wang 1,2 , Shu Yan 1 , Cheng-Xing Jia 1,2 , Xu-Hui Liu 1,2 , Xin-Wei Chen 3 , Jia-Cheng Xu 4,5 , Ziniu Li 6 , Yang Yu 1,2, ⋄ \diamond 1 National Key Laboratory for Novel Software Technology, Nanjing University, China 2 School of Ar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-08446
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Survey on Large Language Models for Mathematical Reasoning Peng-Yuan Wang 1,2* , Tian-Shuo Liu 1,2* , Chenyang Wang 1,2 , Yi-Di Wang 1,2 , Shu Yan 1 , Cheng-Xing Jia 1,2 , Xu-Hui Liu 1,2 , Xin-Wei Chen 3 , Jia-Cheng Xu 4,5 , Ziniu Li 6 , Yang Yu 1,2, ⋄ \diamond 1 National Key Laboratory for Novel Software Technology, Nanjing University, China 2 School of Artificial Intelligence, Nanjing University, China 3 Polixir.ai 4 Nanyang Technological University, Singapore 5 Skywork AI, Singapore 6 The Chinese University of Hong Kong, Shenzhen * Equal contribution ⋄ \diamond Corresponding: [email protected] Abstract Mathematical reasoning has long represented one of the most fundamental and challenging frontiers in artificial intelligence research. In recent years, large language models (LLMs) have achieved significant advances in this area. This survey examines the development of mathematical reasoning abilities in LLMs through two high-level cognitive phases: comprehension, where models gain mathematical understanding via diverse pretraining strategies, and answer generation, which has progressed from direct prediction to step-by-step Chain-of-Thought (CoT) reasoning. We review methods for enhancing mathematical reasoning, ranging from training-free prompting to fine-tuning approaches such as supervised fine-tuning and reinforcement learning, and discuss recent work on extended CoT and “test-time scaling”. Despite notable progress, fundamental challenges remain in terms of capacity, efficiency, and generalization. To address these issues, we highlight promising research directions, including advanced pretraining and knowledge augmentation techniques, formal reasoning frameworks, and meta-generalization through principled learning paradigms. This survey tries to provide some insights for researchers interested in enhancing reasoning capabilities of LLMs and for those seeking to apply these techniques to other domains. 1 Introduction “Can machines think?” This profound question, posed by Alan Turing in the 1950s [ Turing 1950 ] , established the philosophical foundation for modern artificial intelligence. Since then, enabling machines to reason has remained a central and enduring objective in AI research. Among the most rigorous and illuminating assessments of machine reasoning is mathematical problem-solving, a domain that demands not only the manipulation of symbols, but also the representation and understanding of abstract concepts, the construction of formal arguments, and the transfer of principles to new and varied contexts. As such, mathematical reasoning provides a precise and demanding lens through which to evaluate and advance machine intelligence. Beginning in the 1960s, AI researchers sought to endow machines with mathematical reasoning abilities by developing systems capable of representing and manipulating formal knowledge. Early work centered on symbolic rule-based systems [ Feigenbaum et al. 1963 , Bobrow et al. 1964 ] , which depended on handcrafted rules and pattern matching [ Slagle 1965 , Fletcher 1985 ] . While pioneering, these approaches were restricted to narrow domains and lacked generalization. Subsequent efforts, such as semantic parsing methods [ Kwiatkowski et al. 2013 , Goldwasser and Roth 2014 ] , focused on mapping problem text to structured logical forms, but continued to rely heavily on human engineering and struggled to scale to the full diversity of mathematical tasks. Recent advances in large language models (LLMs) have profoundly reshaped the field of natural language representation and understanding. The introduction of transformer-based architectures and instruction-tuned models such as ChatGPT has led to remarkable progress in natural language problem-solving. In this context, mathematical reasoning, a long-standing and rigorous benchmark for AI systems, has become a focal point for evaluating LLM capabilities. As illustrated in Figure 1 , state-of-the-art models now exhibit significantly enhanced performance across a wide range of complex mathematical benchmarks. Figure 1: The rapid advancements in mathematical capabilities in recent years. For example, Grok 3 Beta achieved an impressive score of 83.9\% on the AIME 2024 [ aim 2024 ] , a prestigious Olympiad-level contest. This result places the model within the top 2.5\% of all participants nationwide 1 1 1 Human performance is based on data from MAA: https://maa.edvistas.com/eduview . , as shown in Figure 3 . As LLMs continue to excel in mathematical problem-solving, their impact is becoming increasingly significant across various domains. In Finance, they can leverage complex mathematical reasoning capabilities to process both structured tables and unstructured text, enabling Financial Document Question Answering, leading to handling complex mathematical scenarios [ Srivastava et al. 2024 ] . In the Medical field, advanced LLMs can work as a medical diagnostic assistant, achieving high accuracy in answering specialized healthcare-related questions [ Wu et al. 2024a ] . Driven by the rapid advancements and increasing applicability of mathematical reasoning in LLMs, this work seeks to consolidate the fragmented developments into a coherent framework. The ability of LLMs to perform mathematical reasoning can be grounded roughly in two elements: comprehension of mathematical concepts and the generation of solutions through step-by-step deduction. Comprehension involves acquiring a broad and flexible understanding of diverse mathematical concepts, problem formats, and difficulty levels to enable effective reasoning and problem-solving. Recent research has demonstrated that large-scale pre-training on mathematical corpora, such as textbooks, academic publications, and problem datasets, enables LLMs to internalize domain-specific knowledge, terminology, and the contextual reasoning patterns characteristic of mathematical discourse [ Shao et al. 2024 , Yang et al. 2024b ] . This data-driven paradigm marks a significant shift from earlier approaches based on rule-matching [ Slagle 1965 , Fletcher 1985 ] or semantic parsing [ Kwiatkowski et al. 2013 , Goldwasser and Roth 2014 ] , moving toward a more holistic and contextualized understanding of mathematics. The second element, solution generation, focuses on producing logically coherent intermediate steps during problem-solving. A key technique for achieving this in LLMs is chain-of-thought (CoT) prompting [ Wei et al. 2022b ] , which encourages models to emulate deductive reasoning by generating step-by-step explanations. This approach has proven effective in enabling LLMs to tackle more complex, abstract, and real-world mathematical challenges, as it structures the reasoning process into interpretable and verifiable steps. Despite their impressive language understanding abilities, pre-trained LLMs often struggle to produce contextually appropriate and relevant responses when applied directly to downstream tasks, frequently resulting in repetitive or irrelevant output. To mitigate these shortcomings, prompt engineering, particularly approaches like In-Context Learning (ICL) [ Jie and Lu 2023 , Zhou et al. 2022 ] , has emerged as a vital strategy for enhancing the reasoning capabilities of LLMs through careful design of input prompts. In addition to prompt engineering, fine-tuning 2 2 2 Here, we refer supervised fine-tuning, reinforcement learning, and any other stages of guided learning after pre-training of LLMs as “fine-tuning” instead of “post-training”, for exactness of verbal expression. strategies have been shown to further boost model performance. For example, supervised fine-tuning (SFT) on high-quality demonstrations adapts pre-trained models to specific tasks or domains [ Yang et al. 2024b , Ho et al. 2023 , Magister et al. 2023 ] . This process helps align models with instruction-following objectives, although it introduces challenges such as overfitting and limitations on exploration [ Li et al. 2025b , Wang et al. 2024a , Zeng et al. 2025 ] . Reinforcement learning (RL) methods [ Guo et al. 2025 ] further empower models to improve their problem-solving abilities through trial-and-error exploration. Recent research has also shown that extending CoT reasoning by generating longer token sequences during inference [ OpenAI 2024 , Snell et al. 2024 ] can significantly enhance the reasoning