Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models

Sohyun An, Ruochen Wang, Tianyi Zhou, Cho-Jui Hsieh · 2025

Read Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models by Sohyun An, Ruochen Wang, Tianyi Zhou, Cho-Jui Hsieh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21765.txt.

Introduction

Sohyun An, Ruochen Wang, Tianyi Zhou, and Cho-Jui Hsieh tackled wasted effort in two thousand twenty-five in large reasoning models. Such models often think too long, spinning out complex paths that waste computing power and can even hurt accuracy. The team guessed the fault lies in picking the wrong thinking pattern at the wrong spot.

They split reasoning paths into patterns, kept the helpful ones, and cut the harmful ones. Cleaned paths cut computing cost by up to half while keeping right answers right. Some wrong answers turned right, with a solid gain in accuracy at shorter length. Training on pairs of good and bad paths spread the gain to new problems.

Catalog record

CREATORSohyun An, Ruochen Wang, Tianyi Zhou, Cho-Jui Hsieh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21765

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models Sohyun An, Ruochen Wang, Tianyi Zhou, Cho-Jui Hsieh Abstract While recent success of large reasoning models (LRMs) significantly advanced LLMs’ reasoning capability by optimizing the final answer accuracy using reinforcement learning, they may also drastically increase the output length due to overthinking —characterized by unnecessarily complex reasoning paths that waste computation and potentially degrade the performance. We hypothesize that such inefficiencies stem from LRMs’ limited capability to dynamically select the proper modular reasoning strategies, termed thinking patterns at the right position. To investigate this hypothesis, we propose a dynamic optimization framework that segments model-generated reasoning paths into distinct thinking patterns, systematically identifying and promoting beneficial patterns that improve the answer while removing detrimental ones. Empirical analysis confirms that our optimized thinking paths yield more concise yet sufficiently informative trajectories, enhancing reasoning efficiency by reducing attention FLOPs by up to 47% while maintaining accuracy for originally correct responses. Moreover, a non-trivial portion of originally incorrect responses are transformed into correct ones, achieving a 15.6% accuracy improvement with reduced length. Motivated by the improvement brought by the optimized thinking paths, we apply a preference optimization technique supported by a pairwise dataset contrasting suboptimal and optimal reasoning paths. Experimental evaluations across multiple mathematical reasoning benchmarks reveal that our method notably reduces computational overhead while simultaneously improving reasoning accuracy, achieving up to a 12% accuracy improvement and reducing token usage from approximately 5,000 to 3,000 tokens. 1 Introduction Recent advancements in Large Reasoning Models (LRMs) [ 13 , 44 , 9 ] have substantially enhanced their capabilities across various complex reasoning-intensive tasks. These advancements have largely been driven by outcome-only-based reinforcement learning (RL) [ 23 , 25 , 12 , 16 ] , which train models to maximize the final answer accuracy. Models trained under this framework prioritize producing the correct answer over generating an optimal reasoning trajectory, and thus can sometimes produce long reasoning paths, i.e. , test-time scaling , achieving strong performance even in complex scenarios [ 28 , 36 , 15 ] . While these methods effectively increase the correctness of final predictions, they also reveal inherent limitations [ 19 ] . One notable issue frequently observed in current LRMs is the tendency toward overthinking [ 4 , 29 , 31 ] , characterized by excessively prolonged reasoning paths or unnecessarily intricate inference steps. This behavior typically results in significant computational inefficiencies, increasing resource usage without improvement in performance, and can even degrade overall performance due to excessive exploration [ 30 , 2 , 27 , 22 ] . We hypothesize that this inefficiency largely stems from current LRMs’ limited capability to dynamically identify and select optimal reasoning strategies—referred to hereafter as thinking patterns —at critical junctures within the reasoning process. A thinking pattern is defined as a modular reasoning segment that performs a distinct cognitive function, such as hypothesis generation, self-verification, intermediate summarization, or exploring alternative scenarios [ 7 ] . An ideal LRM should dynamically utilize beneficial thinking patterns while minimizing or discarding unnecessary or detrimental ones, thus improving computational efficiency without compromising accuracy. To investigate this hypothesis, we formulate the enhancement of reasoning efficiency as a constrained optimization problem aimed at minimizing the expected computational cost of reasoning trajectories while preserving or improving task performance. Subsequently, we propose a framework for D ynamic T hinking pattern O ptimization ( DTO ), specifically designed to refine the selection of modular reasoning strategies. Our proposed method operates by first segmenting model-generated reasoning trajectories into identifiable thinking patterns. We then systematically evaluate each segment’s contribution, classifying it as positive or negative based on its impact on reasoning efficiency. By identifying appropriate finalization points and selectively pruning negative segments while reinforcing positive ones, our approach not only compresses the original reasoning paths into concise, effective trajectories, but also enables the transformation of flawed reasoning traces associated with incorrect outcomes into more coherent and accurate alternatives. In contrast to prior methods, which rely heavily on heuristic truncation [ 4 ] or aggregate metrics like token length [ 27 , 17 ] , our framework explicitly models and optimizes the individual contributions of reasoning segments, enabling a more precise enhancement of reasoning efficiency. To explicitly guide LRMs toward more optimal reasoning behaviors, we integrate a preference optimization approach [ 20 ] , leveraging a specialized pairwise dataset that contrasts suboptimal and optimal reasoning trajectories based on our framework. This encourages models to preferentially adopt thinking patterns demonstrated to yield superior efficiency and effectiveness. We validate our framework through extensive experiments conducted across multiple established mathematical reasoning benchmarks. The results consistently demonstrate that our approach reduces computational requirements while simultaneously enhancing reasoning performance. Our contributions and findings are summarized as follows: • We formulate the enhancement of LRM reasoning efficiency as a constrained optimization problem, and introduce a dynamic optimization framework called DTO, for this purpose. • We empirically validate that dynamically selecting optimal thinking patterns significantly improves reasoning efficiency. • Leveraging a preference optimization strategy based on pairwise trajectory comparisons, we achieve high reasoning efficiency across various mathematical reasoning benchmarks, attaining up to 12% higher accuracy over the original LRM, while reducing token usage from around 5,000 to 3,000. 2 Related Work Figure 1 : Illustration of DTO. We construct a truncated reasoning trajectory \Delta_{x}^{f} by identifying the point where the probability score p_{i} in Equation 6 exceeds a threshold T=1.0 , and then applying the binary selection function f(\cdot) from Equation 7 . We then append the finalization pattern \delta_{\text{finalize}} and sampled answer s^{*} ( Equation 9 ) to form \tilde{\Delta}_{x}^{f} . Finally, the pruning function g(\cdot) ( Equation 11 ) refines the trajectory into the optimized version \Delta_{x}^{g} , as illustrated in the orange box. Large Reasoning Models. LLMs have traditionally exhibited rapid, intuitive decision-making aligned with System 1 thinking. Recent efforts aim to equip them with more deliberate, System 2 reasoning abilities. Early studies primarily adopted prompting-based methods, such as chain-of-thought, to elicit explicit step-by-step reasoning [ 34 , 32 , 39 , 14 ] . While effective for structured reasoning, these methods often struggled with complex, multi-step problems [ 36 ] . To address this, later work introduced test-time scaling techniques [ 28 , 3 , 35 , 24 ] , incorporating search and verification during inference to boost accuracy. More recently, Large Reasoning Models (LRMs) have been developed to internalize sophisticated reasoning capabilities within the model itself [ 9 , 44 , 13 ] . They are typically trained via outcome-based RL that focuses on maximizing final answer accuracy [ 23 , 25 , 12 ] . While effective for improving correctness, it may inadvertently lead to overthinking —unnecessarily long or complex reasoning paths that reduce computational efficiency [ 4 , 29 , 31 ] . Reasoning Efficiency of LRMs. Several recent studies have addressed the issue of reasoning efficiency in LRMs. For example, some studies [ 22 , 41 , 2 , 30 , 1 , 38 , 26 , 11 ] have introduced various length-based objectives integrated with RL approaches, while other efforts [ 40 , 42 ] have employed SFT supplemented by the collection of effective datasets. Complementary to these strategies, another line of

Read the complete text