Evolutionary Pre-Prompt Optimization for Mathematical Reasoning

Mathurin Videau, Alessandro Leite, Marc Schoenauer, Olivier Teytaud · 2024

Read Evolutionary Pre-Prompt Optimization for Mathematical Reasoning by Mathurin Videau, Alessandro Leite, Marc Schoenauer, Olivier Teytaud (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04291.txt.

Introduction

Mathurin Videau and co-authors tune, in a 2024 paper, the worked sums that teach a talk model to reason in steps. Few-shot prompts with chain-of-thought sums steer models to sound ends. Which sums to show is the live ask. Their path, EPPO, breeds prompt sets by evolution: keep, mix, and shift the sums, grade by score, and breed the best on. Match-based breeds beat plain few-shot picks by over 10 points of exact match, on GSM8k and MathQA sets. A tight, fit-to-task search is the trick: slight overfit to the ask pays off here.

Gains held in many frames. Self-agree votes, where the model polls its own runs, stacked on top. The breed path is cheap next to retrain.

Catalog record

CREATORMathurin Videau, Alessandro Leite, Marc Schoenauer, Olivier Teytaud
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evolutionary Pre-Prompt Optimization for Mathematical Reasoning Mathurin Videau, Alessandro Leite, Marc Schoenauer, Olivier Teytaud Abstract . Recent advancements have highlighted that large language models (LLMs), when given a small set of task-specific examples, demonstrate remarkable proficiency, a capability that extends to complex reasoning tasks. In particular, the combination of few-shot learning with the chain-of-thought (CoT) approach has been pivotal in steering models towards more logically consistent conclusions ( Wei et al., 2022b ) . This paper explores the optimization of example selection for designing effective CoT pre-prompts and shows that the choice of the optimization algorithm, typically in favor of comparison-based methods such as evolutionary computation, significantly enhances efficacy and feasibility. Specifically, thanks to a limited exploitative and overfitted optimization, Evolutionary Pre-Prompt Optimization (EPPO) brings an improvement over the naive few-shot approach, exceeding 10 absolute points in exact match scores on benchmark datasets such as GSM8k and MathQA. These gains are consistent across various contexts and are further amplified when integrated with self-consistency (SC). CCS: Computing methodologies Reasoning about belief and knowledge 1. Introduction Large language models (LLMs) ( Brown et al., 2020 ; Hoffmann et al., 2022 ; Touvron et al., 2023 ; Anil et al., 2023 ; Team et al., 2023 ) have emerged as a transformative force, demonstrating exceptional capabilities across a spectrum of tasks. However, despite their size and complexity, these models still face challenges in multi-step reasoning, particularly in tasks that require arithmetic, logic, and/or mathematical reasoning ( Cobbe et al., 2021 ; Rae et al., 2021 ) . To address this limitation, recent works have focused on enhancing the reasoning abilities of LLMs. A significant advancement in this direction is the chain-of-thought (CoT) prompting method ( Wei et al., 2022b ) . This approach involves guiding LLMs to articulate intermediate reasoning steps in a manner akin to human thought processes, leading to more accurate and interpretable solutions. This method has shown substantial improvements on complex tasks, including mathematics and commonsense reasoning ( Wei et al., 2022b ; Suzgun et al., 2022 ; Lu et al., 2022 ) . The advancement of the CoT prompting has opened new pathways in the design of effective CoT prompts ( Zhou et al., 2022 ; Jiang et al., 2023 ; Fu et al., 2022 ; Kojima et al., 2022 ) . A crucial aspect of this research is the strategic use of complex examples in prompts for a solution. Fu et al. (2022) demonstrate that using such intricate examples in few-shot prompts could improve LLM performance in reasoning tasks, highlighting the importance of not only the content but also the structure of the prompts in enhancing the reasoning process. In the context of LLMs, few-shot pre-prompting based on CoT consists of adding several examples, typically made of several steps as in standard CoT, as a pre-prompt before each prompt under consideration. Another effective technique in CoT prompting is the inclusion of directive phrases, such as “let us think step by step,” which leads to more organized reasoning generation ( Kojima et al., 2022 ) . Overall, these developments highlight the need to carefully select and structure prompts, as they directly affect LLM efficiency in tackling complex reasoning tasks. In line with previous works, we investigate the effect of few-shot prompting through automated example selection. However, we use these examples as a pre-prompt for the whole downstream task at hand. Once constructed, these few examples remain fixed for the given task (here, mathematical modeling ( Feigenbaum et al., 1963 ; Bundy, 1983 ) ). This differs from various in-context learning methods (see Section 6 ) in which prompt selection depends on each specific instance. Furthermore, since most reasoning benchmarks provide a training set that is too small for proper training ( Ni et al., 2023 ; Liu et al., 2023 ) , we focus on evolutionary algorithms, which can be comparison-based and therefore only need a few bits of feedback, paving the way for information-theoretic generalization guarantees. We analyze mathematically the risk of overfitting or exploitative behavior of such evolutionary approaches, and prove generalization bounds that do not exist for other approaches accessing fine-grained data with gradient-based methods. Such an Evolutionary Pre-Prompt Optimization strategy (termed EPPO in the following) effectively identifies a concise set of examples (called for short few-shot in the following) that notably enhance performance when used together as a pre-prompt. Additionally, considering the current limited understanding of few-shot mechanisms ( Min et al., 2022 ) , the insights gained from the selected examples enable us to identify new ways to improve the performance of large language models, not only helping in their practical use for complex tasks, but also contributing to a better understanding of the nuances of few-shot prompting at large. Contributions. The main contributions of this work are as follows: • We introduce Evolutionary Pre-Prompt Optimization (EPPO), a comparison-based evolutionary framework for optimizing fixed few-shot chain-of-thought (CoT) pre-prompts at the task level, framing prompt design as a principled combinatorial optimization problem. • We empirically demonstrate that controlled, exploitative evolutionary search reduces overfitting in low-data regimes while outperforming brute-force example selection and classical fine-tuning. • We demonstrate strong and robust empirical gains on mathematical reasoning benchmarks, achieving improvements exceeding 10 absolute exact-match points on GSM8k and MathQA, with performance that generalizes across decoding strategies and combines additively with self-consistency. • We show that optimized pre-prompts induce more consistent and structured reasoning traces, revealing that prompt quality rather than prompt length is the key factor in effective and interpretable CoT reasoning. • We enforce these results by an information-theoretic analysis of evolutionary few-shot optimization, deriving generalization bounds that characterize the effects of data size, population parallelism, and optimization budget, and that are unavailable for gradient-based fine-tuning methods. Roadmap . The paper is organized as follows. Section 2 introduces the different components of EPPO and details the overall optimization procedure. It also briefly summarizes the theoretical results that bound the generalization risks of EPPO. Section 3 presents the experimental evaluation of EPPO, with a particular emphasis on overfitting behavior and generalization. Section 4 further analyzes the internal working mechanisms of the method. Section 5 discusses the implications and limitations of the results, while Section 6 positions our contribution with respect to related work. Finally, Section 7 concludes the paper. For completeness, the theoretical analysis of the generalization risk induced by the selection process is provided in Appendix A , with detailed proofs of the bounds relating the optimization budget and feedback granularity to the probability of overfitting. 2. Methods This work focuses on optimizing the performance of a given LLM on a given complex downstream task (here, mathematical reasoning) through few-shot CoT pre-prompting. 2.1. Datasets Our primary goal is to identify the most impactful small pre-prompt that can significantly boost the LLM performance on the task at hand. We will be looking for a pre-prompt composed of several prompts chosen from a much larger set, the demonstration set . We want this set of selected prompts, when used together as a pre-prompt for all further queries, to improve the efficiency and effectiveness of the LLM on the task at hand. Downstream tasks for LLMs (e.g., GSM8k, SVAMP, MathQA, among others, more in Section 3 ) are usually

Read the complete text