CoT-Valve: Length-Compressible Chain-of-Thought Tuning
Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, Xinchao Wang · 2025
Read CoT-Valve: Length-Compressible Chain-of-Thought Tuning by Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, Xinchao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09601.txt.
Introduction
Xinyin Ma and co-authors present CoT-Valve, a way to tune how long a language model's written reasoning runs, in a 2025 paper. Long chains of thought sharpen reasoning but cost time and compute, while easy tasks need far fewer steps than hard ones. The method finds a direction in the model's settings that, when shifted, stretches or shrinks the chain it writes.
Training data pairs the same questions with chains from long to short, and two added strategies give finer control or shrink chains step by step. On the QwQ model, math chains fell from 741 to 225 tokens with accuracy nearly unchanged, and harder contest chains shrank by about a third. The control beat prompt-based steering in the reported tests.
Catalog record
| CREATOR | Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, Xinchao Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09601
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CoT-Valve: Length-Compressible Chain-of-Thought Tuning Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, Xinchao Wang Abstract Chain-of-Thought significantly enhances a model’s reasoning capability, but it also comes with a considerable increase in inference costs due to long chains. With the observation that the reasoning path can be easily compressed under easy tasks but struggle on hard tasks, we explore the feasibility of elastically controlling the length of reasoning paths with only one model, thereby reducing the inference overhead of reasoning models dynamically based on task difficulty. We introduce a new tuning and inference strategy named CoT-Valve, designed to allow models to generate reasoning chains of varying lengths. To achieve this, we propose to identify a direction in the parameter space that, when manipulated, can effectively control the length of generated CoT. Moreover, we show that this property is valuable for compressing the reasoning chain. We construct datasets with chains from long to short for the same questions and explore two enhanced strategies for CoT-Valve: (1) a precise length-compressible CoT tuning method, and (2) a progressive chain length compression approach. Our experiments show that CoT-Valve successfully enables controllability and compressibility of the chain and shows better performance than the prompt-based control. We applied this method to QwQ-32B-Preview, reducing reasoning chains on GSM8K from 741 to 225 tokens with a minor performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with only one additional incorrect answer. 1 Introduction Figure 1: The reasoning model, after the length-compressible CoT tuning, can generate reasoning paths from long to short, leveraging LoRA as a ‘Valve’. We show one example from our constructed dataset MixChain. Chain-of-Thought (CoT) reasoning ( Wei et al., 2022 ) has emerged as a powerful technique for enhancing the reasoning capabilities of large language models ( Jaech et al., 2024 ; Dubey et al., 2024 ; Abdin et al., 2024 ) , particularly in complex tasks such as mathematics and coding ( Sprague et al., 2024 ) that require multi-step inference. By simulating the process of human-like thought progression, CoT enables models to break down complex problems into sub-questions, improving accuracy and interpretability ( Joshi et al., 2023 ) . Those reasoning abilities have also been tested in different domains, such as image generation ( Ma et al., 2025 ) and visual understanding ( Shao et al., 2024 ) . Training reasoning models often involves generating extensive reasoning paths through methods such as sampling ( Wang et al., 2023 ) , tree search ( Yao et al., 2023 ; Guan et al., 2025a ; Zhang et al., 2024 ) or reinforcement learning ( DeepSeek-AI, 2025 ) to ultimately reach the correct answer. However, these long chains often incorporate redundant intermediate steps that can be unnecessary or too complex ( Lightman et al., 2024 ) , and the redundancy in the reasoning paths for training leads to inefficiencies in token usage and increased inference costs. However, crafting an optimal reasoning chain that omits extraneous details is challenging due to the limited availability of intermediate rewards to guide the process and human annotations ( Zhang et al., 2025 ) . Removing some or all of the intermediate steps and then training or distilling the model ( Liu et al., 2024b ; Yu et al., 2024 ) will degrade the performance. Alternative approaches employ information-theoretic measures ( Ton et al., 2024 ) or identify an "overthinking" solution in QwQ ( Team, 2024b ) to evaluate the contribution of each sentence to the final answer. We observe that current reasoning models, such as QwQ ( Team, 2024b ) and DeepSeek-R1 ( DeepSeek-AI, 2025 ) allocate an excessive number of tokens to simple tasks, while potentially providing insufficient tokens for complex tasks. Thus, a long reasoning path is still essential, while maintaining the ability to compress reasoning paths for simpler questions is equally important. To solve this, our goal is to fine-tune a model capable of generating both long and short reasoning paths, rather than being restricted to a compressed form. We offer a new way to control the length of CoT, which we refer to as Length-Compressible Chain-of-Thought Tuning. A central component of the proposed method is to identify an update direction in the parameter space, which, by manipulating it, acts as increasing or decreasing the length of CoT. Taking a large step in this direction leads the model to generate a short sequence, while a small step still produces a long and complex reasoning trajectory. We choose to incorporate this update direction by LoRA Hu et al. (2022) , enabling it to function as an additional branch that facilitates easy modulation of intensity while imposing minimal extra parameters on the model. We explore methods to identify this direction and demonstrate that it offers superior controllability compared to prompt-based approaches, which enables the generation of short CoT that prompt-based methods are unable to achieve. Besides, we observe that the direction can be extrapolated, allowing the reasoning chains to be extended beyond or shortened to lengths unseen in the training set. Leveraging this compressibility, we construct a dataset that pairs long and short reasoning chains for each question. This dataset is then utilized in two ways: (1) to refine the direction for more precise tuning, and (2) to progressively compress the reasoning path. We evaluate our method across different types of models, ranging from a pre-trained LLM with little reasoning ability, LLaMA-3.1-8B and LLaMA-3.2-1.5B-Instruct ( Dubey et al., 2024 ) , to post-trained reasoning models, QwQ-32B-Preview ( Team, 2024b ) , and distilled reasoning models, DeepSeek-R1 ( DeepSeek-AI, 2025 ) . Our results demonstrate that, with training for one time, our approach enables a model to generate reasoning paths of varying lengths, and we can achieve better results than previous chain compression baselines. Besides, our study highlights several interesting findings: (1) Short reasoning paths can sometimes outperform longer ones, underscoring the significance of CoT-Valve in enhancing model efficiency. (2) Not every reasoning chain, despite all leading to the correct final answer, is conducive to model optimization. Excessively long or short chains complicate the distillation of CoT, posing challenges to the model training. In summary, our contributions are: (1) CoT-Valve : Enables elastic control of length for CoT within the parameter space, allowing a single model to generate CoT from short to long. (2) MixChain Dataset : A dataset with reasoning paths of varying lengths for each question. (3) Improved Tuning & Progressive Compression : Refines the direction-tuning process based on MixChain and introduces progressive compression for inference efficiency. (4) Performance & Controllability : Achieves controllable reasoning generation and state-of-the-art results for compressed CoT. 2 Related Work Chain-of-Thought. Chain-of-thought Wei et al. (2022) reasoning has shown promising progress in recent years, especially the success of OpenAi-O1 ( Jaech et al., 2024 ) and Deepseek-R1 models ( DeepSeek-AI, 2025 ) . This introduces the test-time scaling law, apart from the traditional scaling law for training Hoffmann et al. (2022) . Several approaches have been proposed to boost the language model to have better problem-solving abilities, including the model has its self-reasoning abilities ( Team, 2024b ) or use Best-of-N ( Nakano et al., 2021 ) , beam search and Monte Carlo Tree Search ( Kocsis and Szepesvari,