Solving Math Word Problems Using Estimation Verification and Equation Generation

Mitchell Piehl, Dillon Wilson, Ananya Kalita, Jugal Kalita · 2025

Read Solving Math Word Problems Using Estimation Verification and Equation Generation by Mitchell Piehl, Dillon Wilson, Ananya Kalita, Jugal Kalita (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-18565.txt.

Introduction

Word problems trip up language models because they mix reading, reasoning, and arithmetic. Mitchell Piehl, Dillon Wilson, Ananya Kalita, and Jugal Kalita present a checking method that lifts model scores in this 2025 paper. Their approach first splits the problem into parts. The model then writes equations from those parts, and a symbolic solver computes the answer.

A second pass asks the model to estimate the answer, the way teachers urge students to sanity-check. If the estimate and the computed answer clash, a repair loop runs until they agree. The method set new best scores on standard numeric and algebraic word problem sets, gaining about two points on average. It also handled trigonometry problems, a type not tried before in this line of work. The team releases two new test sets to push the area further.

Catalog record

CREATORMitchell Piehl, Dillon Wilson, Ananya Kalita, Jugal Kalita
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-18565

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Solving Math Word Problems Using Estimation Verification and Equation Generation Mitchell Piehl, Dillon Wilson, Ananya Kalita, Jugal Kalita Abstract Large Language Models (LLMs) excel at various tasks, including problem-solving and question-answering. However, LLMs often find Math Word Problems (MWPs) challenging because solving them requires a range of reasoning and mathematical abilities with which LLMs seem to struggle. Recent efforts have helped LLMs solve more complex MWPs with improved prompts. This study proposes a novel method that initially prompts an LLM to create equations from a decomposition of the question, followed by using an external symbolic equation solver to produce an answer. To ensure the accuracy of the obtained answer, inspired by an established recommendation of math teachers, the LLM is instructed to solve the MWP a second time, but this time with the objective of estimating the correct answer instead of solving it exactly. The estimation is then compared to the generated answer to verify. If verification fails, an iterative rectification process is employed to ensure the correct answer is eventually found. This approach achieves new state-of-the-art results on datasets used by prior published research on numeric and algebraic MWPs, improving the previous best results by nearly two percent on average. In addition, the approach obtains satisfactory results on trigonometric MWPs, a task not previously attempted to the authors’ best knowledge. This study also introduces two new datasets, SVAMPClean and Trig300, to further advance the testing of LLMs’ reasoning abilities. 1 1 1 The code, results, and datasets can be found at: https://github.com/mitchellpiehl/EVoSS I Introduction Solving MWPs has been an area of interest in natural language processing since Daniel Bobrow in 1964 [ 1 ] developed a computing system to understand natural language and solve algebraic word problems. In recent years, researchers have found many different ways of solving MWPs, including using Seq2Seq methods [ 2 ] , Seq2Tree methods [ 3 ] , [ 4 ] , Graph2Tree methods [ 5 ] or using transformers to directly translate from text to algebraic equations [ 6 ] , [ 7 ] and many more. Recently, there has been a noticed surge in the use of LLMs to solve MWPs to achieve high accuracy [ 8 ] , which has been difficult to obtain despite the perceived simplicity of MWPs. LLMs often get confused when solving MWPs unless assisted in reasoning steps via appropriate prompts [ 9 ] . To address this challenge, researchers have aided plain LLMs with Chain-of-Thought prompting, which allows the LLM to solve an MWP as a multi-step reasoning task [ 10 ] , [ 11 ] . Among the various state-of-the-art prompt engineering techniques, Progressive Rectification Prompting combined with Chain-of-Thought [ 12 ] stands out. This method allows an LLM to rectify its answer by generating responses until the answer passes verification. Another notable technique, BRIDGE, erases the interrogative parts of an MWP, decomposes the rest of the MWP into simpler statements, translates them into equations, and finally solves to get the answer [ 13 ] . Fig. 1: Simplified diagram example of the EVoSS method from the Algebra dataset Current methods for solving MWPs, in spite of being aided by designed prompts, sometimes solve problems incorrectly due to various reasons. This paper proposes a novel approach to solve MWPs called Equation Verification of Symbolic Solvers (EVoSS). Figure 1 illustrates EVoSS with an example from the Algebra dataset [ 14 ] . The method starts by instructing an LLM to decompose the problem statement and generate algebraic equations. Then, a symbolic solver, external to the LLM, is used to solve the problem precisely. The proposed method does not commit the generated answer as final right away, but uses a verification technique that asks the LLM to estimate the answer instead of solving exactly. By comparing the rough estimate with the precise answer, the approach increases the chances of discovering mistakes during equation generation. The introduction of the answer-estimation step during the solving process is inspired by a commonsense practice among teachers of elementary math, supported by publications, which recommend checking the obtained answer against an estimate when the work is finished [ 15 ] , [ 16 ] , [ 17 ] , [ 18 ] . There are many YouTube videos 2 2 2 This video at the well-respected site Khan Academy discusses a two-step estimation of answers to word problems: https://www.khanacademy.org/math/cc-third-grade-math/arithmetic-patterns-and-problem-solving/estimation-word-problems/v/2-step-estimation-word-problems. and websites geared toward elementary school children that discuss how to estimate an answer to math word problems and check the reasonableness of the estimate. The approach is as follows. The first step involves asking the LLM to decompose the question into more straightforward statements and removing unnecessary textual details that may hinder correct translation into equations. Once the equations have been generated, a symbolic solver attempts to solve them. Once the symbolic solver has computed a numerical value, the LLM is presented with the question again. However, the model is now asked to estimate the correct answer instead of solving it precisely. This estimate serves as a point of comparison with the numerical value derived earlier. If the estimation is aligned with the answer, the verification process confirms that the correct answer has been found. If verification fails due to the estimate significantly diverging from the answer obtained by the symbolic solver, a rectification process finds an answer again. During this process, the LLM is presented with the original question and the estimation as a hint to guide the LLM to the correct answer. This paper makes several key contributions. First, the paper introduces a novel MWP answer verification scheme that uses a prompting-based method, assisted by a symbolic solver to verify answers generated by MWPs. This method achieves new state-of-the-art results. Secondly, the paper creates a new trigonometry dataset that is utilized to stress-test the symbolic solver and estimation verification. Finally, this paper finds inadmissible errors in a commonly used dataset and fixes them so that future solvers will be tested fairly. The rest of the paper starts by discussing related works on math word problem-solving. The following sections present the problem statement, and introduces the approach that uses answer estimation to verify MWP answers. Finally, the paper elaborates upon the experiments performed, the results obtained, and the new datasets that are published for use by future researchers. II Related Work A number of recent methods use LLMs to solve MWPs. Here are a variety of state-of-the-art methods that use diverse methodologies. II-A CoT Prompting Methods Chain-of-thought prompting methods ask LLMs to generate reasoning paths while solving MWPs to guide the process [ 10 ] . The reasoning path helps the LLM solve the MWP because the process of explicitly generating the steps in the path and articulating them in natural language, often causes the LLM to correct any mistakes it may have committed when solving a problem [ 19 ] . Two CoT prompting methods have been proposed recently: zero-shot prompting [ 11 ] and few-shot prompting [ 10 ] . Zero-shot CoT prompting instructs the LLM to find the answer without any examples. In contrast, few-shot prompting gives the LLM a few examples illustrating the instructions before asking the LLM to solve. Auto-CoT automatically generates reasoning chains by sampling

Read the complete text