Neuro-Symbolic Data Generation for Math Reasoning
Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang, Xian Zhang, Xiaoxing Ma · 2024
Read Neuro-Symbolic Data Generation for Math Reasoning by Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang, Xian Zhang, Xiaoxing Ma (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04857.txt.
Introduction
Zenan Li and co-authors present a 2024 paper on making fresh math problems to train language models. Whether such models are truly weak at math, or just lack good training data, is an open question. Their method starts from existing problems and changes them in careful ways, so the new problems stay varied and valid.
The method joins a language model's loose, intuitive rewriting with a math solver's exact checking. A sampling step then explores the space of possible changes. Models retrained on the new data beat their peers on math tests. The work suggests data quality, not just model size, drives math skill.
Catalog record
| CREATOR | Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang, Xian Zhang, Xiaoxing Ma |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-04857
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Neuro-Symbolic Data Generation for Math Reasoning Zenan Li, Zhi Zhou, Yuan Yao, Yu-Feng Li, Chun Cao, Fan Yang, Xian Zhang, Xiaoxing Ma Abstract A critical question about Large Language Models (LLMs) is whether their apparent deficiency in mathematical reasoning is inherent, or merely a result of insufficient exposure to high-quality mathematical data. To explore this, we developed an automated method for generating high-quality, supervised mathematical datasets. The method carefully mutates existing math problems, ensuring both diversity and validity of the newly generated problems. This is achieved by a neuro-symbolic data generation framework combining the intuitive informalization strengths of LLMs, and the precise symbolic reasoning of math solvers along with projected Markov chain Monte Carlo sampling in the highly-irregular symbolic space. Empirical experiments demonstrate the high quality of data generated by the proposed method, and that the LLMs, specifically LLaMA-2 and Mistral, when realigned with the generated data, surpass their state-of-the-art counterparts. 1 Introduction Despite recent progress [ 1 , 2 , 3 , 4 , 5 , 6 ] , both proprietary and open-source LLMs are still far from satisfactory in mathematical reasoning [ 7 , 8 , 9 ] . It is an open question whether LLM’s subpar reasoning capability is inherent or due to the the extreme scarcity of high-quality mathematical datasets [ 10 , 11 , 12 , 13 ] . As an initial step towards answer this question, a data generation framework that could create high-quality math datasets is required. To this end, current two lines of research struggle in the diversity-validity dilemma: (1) to produce diverse math data, the prompt-based method effectively rephrases math problems using LLMs, but may induce errors thus ruining the validity , especially considering the rigor of maths; (2) to ensure the validity, template-based methods are often used by rewriting math problems with certain rules, sacrificing the diversity and thus confining data scale. To address this dilemma, we propose a novel neuro-symbolic framework that automatically generates high-quality, supervised mathematical data. The merit of this paradigm lies in leveraging both neural and symbolic strengths: (1) the math problem is generated in the symbolic space, achieving diversity through systematic sampling, while maintaining validity through symbolic solvers; (2) the translation from the symbolic space back to the natural language space can be effectively supported by LLMs, ensuring the consistency between newly generated formal problems and their corresponding natural language versions. Figure 1: The overview of our neuro-symbolic data generation framework. The framework comprises three steps: (1) Formalize the seed problem into its symbolic version. (2) Mutate the symbolic problem to create new variants. (3) Translate the variants in symbolic form back to the natural language version. Additionally, we prompt GPT-4 to generate reasoning paths, which are verified by symbolic solvers, as part of the supervision. Our framework, as illustrated in Figure 1 , initiates with the formalization of the original problem expressed via the math symbolic tools. Next, it mutates the formal problem into an evolved version, and then derives a new natural language problem by informalization . Specifically, we design a mutation mechanism, including various simplification and complication strategies, such that the new problems can be generated with a controllable complexity. As shown in Figure 2 , our mutation mechanism can properly adjust the complexity of generated problems, and the exposure to more complex math problems can improve the LLM’s reasoning capability. Moreover, to ensure the data validity and achieve higher generation diversity, we combine the symbolic solving with the random sampling through the projected Markov chain Monte Carlo technique [ 14 , 15 ] . Figure 2: The performance of our proposed mutation mechanism. The first figure illustrates that the generated problems with higher difficulty levels lead to more reasoning steps of GPT-4. The second figure shows that the gradual incorporation of more difficult problems consistently improves the LLM’s reasoning capability. Empirical evaluation on GSM8K [ 10 ] and MATH [ 11 ] demonstrates the effectiveness of the proposed method. Particularly, we use the proposed framework to generate a mathematical dataset of 620K examples for supervised fine-tuning. The experimental results show that, the fine-tuned models on LLaMA-2 [ 16 ] and Mistral-7B [ 17 ] significantly outperform the existing open-source counterparts on both GSM8K and MATH datasets, as well as two out-of-domain datasets SVAMP [ 12 ] and ASDiv [ 18 ] . On the GSM8K dataset, the model fine-tuned on Mistral-7B even outperforms GPT-3.5-Turbo (by 2.4%), a proprietary model with an order of magnitude larger parameters. Additionally, we evaluate the scalability of our method, and observe consistent performance improvements, as the size of training data increases. This upward trajectory suggests a promising avenue for further enhancing LLMs’ mathematical capabilities. 2 Mutation Compared to existing data generation methods, the key feature of our framework lies in the mutation of math problems within the symbolic space, enabling systematic sampling and symbolic solving. Technically, our mutation mechanism includes several simplification and complication strategies, to control the complexity of the generated math problems. The overall framework of our problem mutation method is summarized in Algorithm 1 . Algorithm 1 The overall framework of problem mutation 1: A seed problem expressed by goal g and constraints h_{1},\dots,h_{n} . 2: A new problem expressed by goal g^{\prime} and constraint h_{1}^{\prime},\dots,h_{n}^{\prime} . 3: for i=1,\dots, do \triangleright Complicate the problem using Projected MCMC 4: Initalize random operations \oplus_{i} and interpreted functions e_{i}^{\prime} . 5: Mutate g^{\prime}=g\oplus_{0}e^{\prime}(z_{0}) , h_{i}^{\prime}=h_{i}\oplus_{i}e^{\prime}(z_{i}) with parameter z_{i} . 6: Randomly perturb a subset \{z_{1},\dots,z_{j}\} for j<n . 7: Solve the rest subset \{z_{j+1},\dots,z_{n}\} . 8: if \{z_{1},\dots,z_{n}\} is solvable then 9: Instantiate the new problem by \{z_{1},\dots,z_{n}\} . 10: Stop the complication process 11: end if 12: end for 13: for i=1,\dots do \triangleright Simplicate the problem using SMT tactics 14: Initalize a random tactic from { simplify , qe , simplify , \dots }. 15: Apply the tactic to the problem g^{\prime} and constraints h_{1}^{\prime},\dots,h_{n}^{\prime} . 16: end for 2.1 Formalization We first provide the formalization of math problems, based on which the mutation mechanism is operated. Specifically, we adopt the SMT-LIB language [ 19 ] , a standard notation compatible with prevalent SMT solvers (e.g., Z3 [ 20 ] , CVC5 [ 21 ] , and MathSAT [ 22 ] ). It can also be easily extended for symbolic calculators (e.g., SymPy [ 23 ] ) and numerical solvers (e.g., SciPy [ 24 ] ). With SMT-LIB language, the math problem in the following structure is enabled: \displaystyle\emph{Goal} \displaystyle g:=\min\mid\max\mid\mathrm{solve}~~f({\mathbf{x}}) \displaystyle\emph{Constraints} \displaystyle h:=h_{1}\wedge h_{2}\mid h_{1}\vee h_{2}\mid\mathrm{ite}(h_{1},h_{2},h_{3})\mid \displaystyle\forall{\mathbf{x}}.\,e_{1}({\mathbf{x}})\bowtie e_{2}({\mathbf{x}})\mid\exists{\mathbf{x}}.\,e_{1}({\mathbf{x}})\bowtie e_{2}({\mathbf{x}})\mid \displaystyle e_{1}\bowtie e_{2},~~\bowtie\,\in\{\geq,\leq,>,<,=,\neq\} \displaystyle\emph{Expressions} \displaystyle e:=c\mid{\mathbf{x}}:=(x_{1},\dots,x_{n})\mid\mathrm{foo}({\mathbf{x}})\mid \displaystyle e_{1}\oplus e_{2},~~\oplus\in\{+,-,\times,\div\} \displaystyle\emph{Domains} \displaystyle\mathcal{D}:=\mathbb{N}\mid\mathbb{N}^{+}\mid\mathbb{R}\mid\mathbb{C} where c denotes a constant, {\mathbf{x}} denotes an n -dimensional variable, \mathrm{ite} denotes the if-then-else structure, \mathrm{foo} refers to an interpreted function (e.g., trigonometric, logarithmic or user-defined ones) on the domain, and g and h represent any function of interest (can include quantifiers). In particular, we pre-defined a series of interpreted functions, such as \mathtt{summation},\mathtt{binomial},\mathtt{gcd} , \mathtt{lcm} , \mathtt{derivate} , and \mathtt{integral} , which facilitate the formalization of most high-school level mathematical problems (excluding geometry) within the above SMT-LIB language. 2.2 Simplification We perform simplification by systematically considering expression reduction and constraint reduction , which can be attained through heuristic tactics provided