From Long to Short: LLMs Excel at Trimming Own Reasoning Chains
Wei Han, Geng Zhan, Sicheng Yu, Chenyu Wang, Bryan Hooi · 2025
Read From Long to Short: LLMs Excel at Trimming Own Reasoning Chains by Wei Han, Geng Zhan, Sicheng Yu, Chenyu Wang, Bryan Hooi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06174.txt.
Introduction
Reasoning models that think too long waste effort on easy problems, and Wei Han and four co-authors teach them to trim their own chains in a 2025 paper. Large reasoning models gain power from long internal monologues but tend to overthink, switching strategies and bloating traces. The authors first document a common dilemma: models struggle to balance correctness against brevity.
Their method, EDIT, works at test time, guiding generation under length constraints while tracking answer distributions, then selecting the shortest correct path found. Across models and datasets, EDIT substantially improves reasoning efficiency, producing compact, readable outputs. The study frames brevity as a controllable objective rather than a fixed model trait.
Catalog record
| CREATOR | Wei Han, Geng Zhan, Sicheng Yu, Chenyu Wang, Bryan Hooi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06174
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Long to Short: LLMs Excel at Trimming Own Reasoning Chains Wei Han, Geng Zhan, Sicheng Yu, Chenyu Wang, Bryan Hooi Abstract O1/R1-style large reasoning models (LRMs) signal a substantial leap forward over conventional instruction-following LLMs. By applying test-time scaling to generate extended reasoning paths, they establish many SOTAs across a wide range of complex reasoning tasks. However, recent studies show that LRMs are prone to suffer from overthinking —the tendency to overcomplicate simple problems, leading to excessive strategy switching and long, convoluted reasoning traces that hinder their interpretability. To mitigate this issue, we conduct a systematic investigation into the reasoning efficiency of a broad set of LRMs and uncover a common dilemma: the difficulty in balancing multiple generation objectives such as correctness and brevity. Based on this discovery, we propose a test-time scaling method, EDIT ( E fficient D ynamic I nference T rimming), which efficiently guides LRMs to identify the shortest correct reasoning paths at test time. EDIT employs constraint-guided generation while jointly tracking length and answer distributions under varying constraints, allowing it to select responses that strike an optimal balance between conciseness and correctness. Extensive experiments across diverse models and datasets show that EDIT substantially enhance the reasoning efficiency, producing compact yet informative outputs that improve readability and user experience. 1 Introduction The rapid emergence of large language models (LLMs) has revolutionized the development of artificial intelligence xu2025towards . These models possess the remarkable capability to autonomously complete a wide range of tasks across diverse scenarios following human instructions grok2025grok3 ; yang2024qwen2 ; anil2023palm ; grattafiori2024llama ; touvron2023llama ; achiam2023gpt . Recently, researchers have begun to push beyond traditional instruction-following or few-shot chain-of-thought. They explore more sophisticated application scenarios, where LLM has to trigger deliberate thinking and reasoning to produce the correct answer fan2023nphardeval ; wei2025browsecomp ; yuan2025naturalreasoning . It has been observed that LLMs can naturally acquire advanced reasoning abilities through a specialized training procedure that combines supervised fine-tuning (SFT) as cold-start plus a subsequent reinforcement learning (RL) stage guo2025deepseek . Building on this insight, an increasing number of large reasoning models (LRMs) have been introduced, marking key milestones in this area. However, the enhanced reasoning abilities of LRMs always come at the cost of increased output redundancy chen2024not , leading to substantial computational overhead that grows quadratically with output length. In addition, the excessively long reasoning chains generated by these models make it difficult for users to interpret and quickly grasp the underlying logic during interactions. Recent models like Qwen-3 qwenteam2025qwen3 and Kimi team2025kimi ; team2025kimik2 combine mixed thinking/non-thinking data or length penalty into training, but still fail to make LRMs wisely select the mode during the inference time. To alleviate this issue, we propose a test-time scaling method— EDIT ( E fficient D ynamic I nference T rimming), which leverages the variation trends of solution lengths and answer confidence under parametric constraints. Building on this statistical discovery, we design a search algorithm that automatically identifies the critical point at which LRMs achieve the pareto optimality between reasoning cost and accuracy. In summary, our main contributions are as follows: • We explore the innovative task named reasoning path compression, with a particular focus on constraint-based generation. Our study includes a systematic analysis on the characteristics of the independent and joint distribution of reasoning length and correctness under various restrictions fed to the LLMs. • Based on the previous analysis, we propose EDIT ( E fficient D ynamic I nference T rimming), a fully test-time reasoning path pruning framework. At the core of this framework is a dual-goal search algorithm that effectively guides the model toward near-optimal solutions in terms of both brevity and correctness. To address the limitations of conventional length measurements, we introduce two evaluation approaches—one metric and one comparison paradigm that better aligns with the observed trade-offs between length and correctness. • Extensive experiments on seven advanced reasoning and instruction-following models as well as three typical datasets validate the effectiveness of EDIT , showcasing its ability to generate concise and accurate reasoning paths. 2 Related Work Large Reasoning Models. Reasoning is a desirable capability for LLMs. The first trial to elicit LLM’s reasoning ability was on mathematical problems hendrycks2measuring , where outcome- or process-based verifiers are trained to supervise LLM learning lightman2023let . More recently, researchers believe that LLMs can simulate human’s System 1 (fast, instinct) and System 2 (slow, delibrate) yu2024distilling to efficiently handle logical tasks. Reward-guided decoding khanovargs ; liao2025reward ; xie2023self and Monte Carlo tree search (MCTS) hao2023reasoning ; zhang2024rest improve these base methods by introducing external or future feedback. CoT wei2022chain and several subsequent enhancement work wang2022self ; yao2023tree ; besta2024graph managed to elicit the reasoning paths directed to the correct answer. The release of OpenAI-o1 jaech2024openai heightened the engagement from the research community in test-time scaling. DeepSeek-R1 guo2025deepseek demonstrates the great role of reinforcement learning in LLM’s grasping of reasoning capabilities liu2025understanding , and there are a line of follow-up works to develop skilled reasoning models shen2025vlm ; liu2025visual . Regards foundation LRM training, most efforts are put at the RL algorithm yu2025dapo and self-evolving pipelines zelikman2022star ; shao2024deepseekmath . Underthinking and Overthinking Issue in LRMs. While O1/R1-like LRMs exhibit stunning capabilities in solving complex problems, they expose undesired weakness termed overthinking when encountering simple questions like “ what is the result of 2+3? ” chen2024not ; team2025kimi ; chiang2024over . Meanwhile, although LRMs are encouraged to generate complete reasoning paths based on the SFT versions, underthinking wang2025thoughts is ubiquitous among LRMs, which could stagate at shallow thinking. To alleviate this issue, the model should learn to avoid both underthinking (incorrection) and overthinking (inefficiency). Several routines have been proposed to alleviate reasoning inefficiency while maintaining an acceptable error rate: direct tuning on pruned generation xia2025tokenskip ; han2024token ; ma2025reasoning , latent-space reasoning geiping2025scaling ; hao2024training and input-aware test-time searching damani2024learning ; wang2025sampling . While Most of these approaches focus on system-level improvement, such as reducing memory usage or latency, this work falls under the category of test-time scaling. However, unlike prior trials, EDIT aims at generating concise and interpretable solution. 3 Methods In this section, we first formalize the task definition of efficient reasoning mathematically. Next, we present key observations derived from our benchmark results. Finally, based on these observations, we propose our efficient search algorithm that navigates toward the optimal solution. 3.1 Problem Formulation EDIT is a test-time adaptation method, and therefore we hold a target model \mathcal{M} with constant parameters during the entire inference period. Given a reasoning dataset \mathcal{D} , for each question q_{i}\in\mathcal{D} with the ground truth a_{i} , we prompt \mathcal{M} multiple times to sample a collection \mathcal{R}=\{\mathbf{r}_{1},\mathbf{r}_{2},...,\mathbf{r}_{N}\} , where The final answer \hat{a}_{i}=\phi(\mathbf{r}_{i}) can be further extracted by a format matching function \phi . Our goal is to maximize the accuracy, which is based on an identity-checking function: \mathrm{acc}=\frac{\sum_{\mathbf{r}_{i}\in\mathcal{R}}\mathbf{1}(\phi(\mathbf{r}_{i})=a_{i})}{|D|} (1) where \mathbf{1(\cdot)} is the indicator function and |D| is the dataset size. Meanwhile, we aim to minimize the average reasoning lengths \bar{l} calculated as the average number of tokens from the correct solution path for each question (at most one for each question) across the datasets. \bar{l}=\frac{\sum_{i,\phi(\vec{r}_{i}=a_{i})}l_{i}}{|D_{corr}|} (2) The goal is to seek a generation strategy \Gamma , which can produce accurate