ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning
Jingyang Yi *, Jiazheng Wang *, Sida Li · 2025
Read ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning by Jingyang Yi *, Jiazheng Wang *, Sida Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21370.txt.
Introduction
Jingyang Yi, Jiazheng Wang, and Sida Li tackle a waste problem in reasoning models: they think for too long, in a paper from 2025. Models such as OpenAI o1 and DeepSeek-R1 write long chains of thought. Long chains help on hard problems, but they also add repeat checks and dead ends that cost time.
Their method, ShorterBetter, teaches a model to find its own best length with no hand labels. The target is the shortest correct answer among several tries, used as a reward signal. Applied to two DeepSeek student models, the method cut output length by half or more on test tasks. Scores held steady, and the traces showed less repeat and less stray searching.
Catalog record
| CREATOR | Jingyang Yi *, Jiazheng Wang *, Sida Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-21370
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning Jingyang Yi *, Jiazheng Wang *, Sida Li Abstract Recent models such as OpenAI o1 and DeepSeek-R1 have demonstrated strong performance on reasoning-intensive tasks by generating extended Chain-of-Thought (CoT) traces. While longer reasoning helps with thorough exploration of solution paths for complex problems, it also often leads to inefficient and redundant outputs—a phenomenon commonly described as overthinking . In this paper, we propose ShorterBetter , a simple yet effective reinforcement learning method that enables reasoning models to learn their own optimal CoT lengths without manual supervision. We define the Sample Optimal Length (SOL) as the length of the shortest correct response among multiple generations, which serves as a dynamic reward signal to guide the model toward efficient reasoning. Applied to DeepSeek-Distill-Qwen-1.5B/7B as base models, ShorterBetter achieves 50%-80% reduction in output lengths in both in-domain and out-of-domain reasoning tasks while maintaining accuracy. Our reasoning trace analysis shows that ShorterBetter refines the structure of the reasoning traces by reducing unnecessary repetition, excessive self-verification, and over-exploration of alternatives. 1 1 1 The source code, model weights, and analysis pipelines are available at: https://jiazheng-wang-yes.github.io/ShorterBetter_ProjectPage/ * * footnotetext: Equal contribution Figure 1 : ShorterBetter exhibits strong length reduction performance (1.5B models). We report accuracy and output length of five sets of models on various tasks. Models include DeepSeek-R1-Distill-Qwen-1.5B, Qwen2.5-1.5B-Instruct (non-reasoning model), ShorterBetter-1.5B ( \alpha =1 and \alpha =2) and Training Efficent-1.5B ( Arora and Zanette, 2025 ) . Our method achieves significant length reduction compared to all baseline reasoning models while maintaining accuracy. 1 Introduction Improving the reasoning capabilities of Large Language Models (LLMs) has recently led to substantial performance gains in highly structured and complex tasks, such as mathematical problem-solving ( Shao et al., 2024 ) and competitive programming ( Jiang et al., 2025 ) . Exemplified by OpenAI o1 ( OpenAI, 2024 ) and DeepSeek-R1 ( Guo et al., 2025 ) , these reasoning models are incentivized to generate extended Chain-of-Thought (CoT) steps during inference. Such extended reasoning often includes detailed intermediate steps, reflective self-verifications, and exploration of alternative problem-solving strategies. These advancements are widely believed to enable LLMs to exhibit human-like System-2 slow thinking ( Evans, 2003 ; Qu et al., 2025 ; Li et al., 2025 ) , wherein significant test-time computational resources are deliberately allocated in exchange for improved accuracy and robustness on challenging reasoning benchmarks ( Jain et al., 2025 ; Rein et al., 2024 ) . However, recent studies have shown that excessively long reasoning processes can be counterproductive, leading not only to computational inefficiencies but also to the so-called overthinking phenomenon ( Chen et al., 2025 ; Luo et al., 2025a ) , where additional reasoning steps yield diminishing returns or even negatively impact model performance ( Su et al., 2025 ) . 2 2 2 We provide an experiment in Section B.3 demonstrating that incorrect reasoning traces often have longer output lengths. For example, we identify several common problematic patterns in lengthy reasoning traces: (1) verbose repetitions without new insights, (2) early stopping of promising paths, (3) continued exploration of alternative solutions despite having arrived at correct answers, and (4) collapse of reasoning coherence following very long outputs. 3 3 3 We include concrete examples for each of the aforementioned patterns in Appendix D . These observations indicate that lengthy reasoning traces are highly compressible, motivating techniques producing more concise reasoning chains that improve computational efficiency while preserving overall performance. Existing approaches targeting efficient LLM reasoning primarily focus on explicitly controlling reasoning length , for instance, by imposing a length budget based on user specifications (L1; Aggarwal and Welleck (2025) ), or by incentivizing shorter reasoning paths through comparisons with reference models (O1-Pruner; Luo et al. (2025a) ). Further discussions on related approaches are provided in Section 2 . In contrast to these explicit control strategies, our work is based on the hypothesis that reasoning models implicitly possess an optimal reasoning length (OL), which is dependent on model capacity and problem difficulty. OL thus represents an in-between "sweet spot": reasoning that is shorter might contain insufficient analysis, while reasoning that is longer risks redundancy and overthinking. This hypothesis has recently undergone theoretical formulation ( Wu et al., 2025 ) and empirical support ( Su et al., 2025 ) . Since OL is intrinsic and not directly observable, we approximate it with the Sample Optimal Length (SOL): for each problem, we generate multiple outputs and define its SOL as the length of the shortest correct response. Leveraging SOL as a practical proxy for OL, we propose ShorterBetter , a reinforcement learning method that teaches the model to dynamically identify and steer towards efficient reasoning lengths during inference: In summary, our contributions are as follows. 1. We propose ShorterBetter , a novel reinforcement learning method that guides reasoning model to autonomously learn its optimal reasoning lengths by defining and using Sample Optimal Length as a dynamic reward signal. 2. We empirically demonstrate that ShorterBetter , when applied to 1.5B/7B base models, achieves substantial (50%-80%) reductions in output token length on both in-domain mathematical reasoning tasks and diverse out-of-domain benchmarks, while maintaining accuracy. 3. We introduce novel reasoning trace analysis frameworks to demonstrate how ShorterBetter enhances reasoning efficiency. Specifically, the model reduces verbosity after finding the correct answer and refactors the overall reasoning traces to adopt more confident and targeted reasoning strategies. 2 Related Work Reasoning in LLMs Since Wei et al. (2022) demonstrated that CoT prompting enhances LLM reasoning, following research has explored ways to scale test-time computation—such as parallel sampling ( Wang et al., 2023 ) and iterative refinement ( Madaan et al., 2023 ) —to improve performance. This led to Large Reasoning Models (LRMs) that autonomously generate extended CoTs through reinforcement learning-based post-training ( Zhang et al., 2025 ) . Examples include OpenAI’s o1 ( OpenAI, 2024 ) , DeepSeek-R1 ( Guo et al., 2025 ) , and Qwen-QwQ ( Yang et al., 2024 ) . These models internalize the ability to deliberate over multiple steps before producing final answers, achieving significant performance gains on reasoning-intensive benchmarks ( Jimenez et al., 2024 ; Jain et al., 2025 ; Gao et al., 2024 ) . Our work addresses a complementary challenge : how to dynamically optimize reasoning efficiency without sacrificing accuracy? The Overthinking Phenomenon This challenge, which was initially observed as verbosity and increased computational costs in CoT prompting ( Nayab et al., 2024 ; Han et al., 2024 ) , has become more pronounced with the advent of LRMs—following the identification and understanding of the overthinking phenomenon ( Chen et al., 2025 ; Su et al., 2025 ) . This phenomenon is typically understood through two primary lenses. Firstly, overthinking is recognized as a significant source of computational inefficiency. In this view, LRMs expend excessive resources generating overly lengthy and often redundant reasoning traces, especially for simple problems ( Lee et al., 2025 ) . This inefficiency manifests as increased token usage, higher latency, and greater computational expenditure ( Aytes et al., 2025 ) , and might be exploited adversarially to induce slowdowns ( Kumar et al., 2025 ) . Secondly, overthinking is increasingly understood as a factor that can directly impair practical performance.