Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models

Yingqian Cui Pengfei He Jingying Zeng Hui Liu Xianfeng Tang, Zhenwei Dai Yan Han Chen Luo Jing Huang, Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He · 2025

Read Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models by Yingqian Cui Pengfei He Jingying Zeng Hui Liu Xianfeng Tang, Zhenwei Dai Yan Han Chen Luo Jing Huang, Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13260.txt.

Introduction

Yingqian Cui and co-authors wrote this 2025 paper on trimming the reasoning chains of large language models. Step-by-step reasoning lifts accuracy on hard problems, but the chains run long and burn compute on steps that add nothing. The team needed a cheap way to tell the load-bearing steps from the filler.

Their gauge is perplexity, a measure of how surprised the model is: delete a step, and if surprise jumps, the step mattered. Models can then be shown trimmed examples, or fine-tuned on them, so they produce only the critical steps. Wide experiments show the balance improving, with accuracy held while chains shrink. The method offers a practical dial between smarts and speed.

Catalog record

CREATORYingqian Cui Pengfei He Jingying Zeng Hui Liu Xianfeng Tang, Zhenwei Dai Yan Han Chen Luo Jing Huang, Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13260

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models Yingqian Cui Pengfei He Jingying Zeng Hui Liu Xianfeng Tang, Zhenwei Dai Yan Han Chen Luo Jing Huang, Zhen Li Suhang Wang Yue Xing Jiliang Tang Qi He Abstract Chain-of-Thought (CoT) reasoning, which breaks down complex tasks into intermediate reasoning steps, has significantly enhanced the performance of large language models (LLMs) on challenging tasks. However, the detailed reasoning process in CoT often incurs long generation times and high computational costs, partly due to the inclusion of unnecessary steps. To address this, we propose a method to identify critical reasoning steps using perplexity as a measure of their importance: a step is deemed critical if its removal causes a significant increase in perplexity. Our method enables models to focus solely on generating these critical steps. This can be achieved through two approaches: refining demonstration examples in few-shot CoT or fine-tuning the model using selected examples that include only critical steps. Comprehensive experiments validate the effectiveness of our method, which achieves a better balance between the reasoning accuracy and efficiency of CoT. 1 Introduction Large language models (LLMs) are powerful generative models capable of performing diverse tasks in different domains Gramopadhye et al. (2024) ; Karabacak and Margetis (2023) ; Ling et al. (2024) and demonstrating strong reasoning capabilities Jaech et al. (2024) . Recent advancements, such as few-shot/zero-shot Chain-of-Thought (CoT) Wei et al. (2022) ; Kojima et al. (2022) , as well as fine-tuning Liu et al. (2023) , have significantly enhanced the LLMs’ reasoning capabilities by leveraging intermediate reasoning steps. In particular, through few-shot CoT, LLMs can learn from the reasoning steps in the demonstration examples and apply similar reasoning patterns to target tasks. In the case of zero-shot CoT, LLMs are prompted to"think step by step" to generate reasoning steps. In fine-tuning, LLMs can also learn from the reasoning steps in the fine-tuning samples, further enhancing their reasoning abilities. While many existing reasoning methods rely on available data (e.g., few-shot examples or fine-tuning datasets), there is limited understanding of which reasoning steps are truly essential and how their impact varies across different models. This gap hinders progress in two key areas: (1) how to effectively identify and remove unimportant reasoning steps from the data to reduce computational costs, and (2) whether the important reasoning steps for one model are also important to another. Figure 1: Prediction accuracy of few-shot CoT using all/selected steps in the demonstration examples. For example, we observe that removing certain reasoning steps from the demonstrations in few-shot CoT can have varying effects: some models follow the modified examples and generate much fewer tokens while maintaining reasoning accuracy, whereas others experience a decline in performance. Specifically, we consider a math problem of function solving Saxton et al. (2019) . We compare two versions of demonstrations when conducting few-shot CoT: one with full manually crafted reasoning paths and another containing only intuitively important steps, as shown in Figure 1 . For most models, removing certain steps significantly reduces the number of generated tokens with minimal impact on accuracy, suggesting that the removed steps contribute limited meaningful information. However, LLaMA3-8B shows a noticeable decline in accuracy, indicating that the importance of reasoning steps can vary across different LLMs. Similar to the few-shot CoT scenario, when given a set of fine-tuning samples with reasoning steps, some LLMs may find some steps redundant, and the fine-tuning cannot improve the prediction accuracy. However, these LLMs will follow the fine-tuning samples to generate the additional tokens, raising the computation cost. Other LLMs may struggle to develop reasoning capabilities when given too few reasoning steps during fine-tuning. This observation will be further demonstrated in Section 4 . Therefore, in this work, we focus on identifying unimportant reasoning steps from few-shot examples or fine-tuning data given a specific LLM. To achieve this, we propose a method leveraging perplexity, a metric commonly used to measure the confidence or fluency of model-generated text Jelinek et al. (1977) , to quantify the impact of each reasoning step. Our contributions are as follows: First, since perplexity reflects an LLM’s confidence in processing inputs and generating outputs Jelinek et al. (1977) , we hypothesize that perplexity can serve as an indicator of reasoning step importance. Specifically, if the perplexity changes significantly after removing a reasoning step, we conjecture that the removed step plays a crucial role in the model’s decision-making process. To validate this hypothesis, we conduct empirical analyses (Section 2.2 ) and observe a strong correlation between changes in perplexity (with and without a reasoning step) and the prediction performance. This finding reveals that perplexity effectively quantifies the significance of individual reasoning steps. Second, inspired by this insight, we develop an algorithm, S tepwise P erplexity-Gu I ded R ef I nemen T (SPIRIT), to remove or merge unimportant reasoning steps. To effectively apply this approach across different scenarios of CoT, we tailor our approach for two different use cases, (1) few-shot CoT, where the full reasoning steps in the examples are known (SPIRIT-FS), and (2) fine-tuning, where the samples only have input and the final answer at the beginning (SPIRIT-FT). When developing the algorithms, a common technical challenge is that some steps, though considered unimportant by the selection criteria, may still contain partial usefulness. Removing such steps could disrupt the coherence of the remaining reasoning process. To address this, we further refine the algorithm by incorporating a merging mechanism to ensure the overall coherence of the whole reasoning process. Finally, we conduct comprehensive experiments to examine the effectiveness of the proposed algorithms. In few-shot CoT, our method successfully provides demonstrations that guide the model to generate a more efficient reasoning process without greatly sacrificing performance. For fine-tuning, our approach achieves a better effectiveness-efficiency trade-off than randomly select steps to be removed. 2 Preliminary In this section, we first present the essentials of perplexity, and then introduce our exploration on how to use perplexity to analyze the reasoning steps. 2.1 Perplexity (PPL) Perplexity was developed in Jelinek et al. (1977) and is a common metric for LLMs. It is defined as \displaystyle\text{PPL}(x,\{w_{k}\}_{k=1}^{N})\hskip 180.67499pt \displaystyle=\quad\exp\left(-\frac{1}{N}\sum_{i=1}^{N}\log p(w_{i}\mid x,w_{1},\dots,w_{i-1})\right), (1) where x represents the prompt, \{w_{k}\}_{k=1}^{N} denotes sequence of tokens with total length N which are conditioned on x . The probability p(w_{i}\mid x,w_{1},w_{2},\dots,w_{i-1}) is the likelihood assigned by the model to the i -th token given the prompt and the preceding tokens. In literature, many studies utilize perplexity, e.g., for reference model pruning Ankner et al. () , attack detection Alon and Kamfonas (2023) , misinformation detection Lee et al. (2020) , and uncertainty quantification Cooper and Scholak (2024) . 2.2 Relationship between Perplexity and CoT Prediction Accuracy We conduct preliminary evaluation to investigate the relationship between PPL and CoT prediction accuracy when changing the steps used in the reasoning procedure. Intuitively, a higher likelihood indicates that the LLM is more confident to the context, and from Eq.( 1 ), a higher likelihood results is a lower PPL. Thus, we hypothesize that the PPL is negatively correlated with the prediction accuracy. In the experiments summarized in Table 1 , we apply few-shot demonstrations to perform CoT reasoning across three tasks from the DeepMind Mathematics Dataset Saxton et al. (2019)

Read the complete text