TL;DR : Too Long, Do Re-weighting for Efficient LLM Reasoning Compression
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu · 2025
Read TL;DR : Too Long, Do Re-weighting for Efficient LLM Reasoning Compression by Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-02678.txt.
Introduction
Zhong-Zhi Li and a large team of co-authors present a 2025 training plan that trims long chains of thought from language models. Long thought chains lift scores but cost much time and compute at use. Heavy data labels or model blending can trim them, yet both add work. The team instead shifts the training mix as training runs.
Their plan steadily reweights fast, gut-style data against slow, step-by-step data. Redundant thought steps fade while core skill stays. Tests on two distilled models across tests of mixed hardness cut output length by near forty percent. Accuracy held steady at the same time. The authors plan to share code and data soon.
Catalog record
| CREATOR | Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-02678
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TL;DR : Too Long, Do Re-weighting for Efficient LLM Reasoning Compression Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu Abstract Large Language Models (LLMs) have recently achieved remarkable progress by leveraging Reinforcement Learning and extended Chain-of-Thought (CoT) techniques. However, the challenge of performing efficient language reasoning—especially during inference with extremely long outputs—has drawn increasing attention from the research community. In this work, we propose a dynamic ratio-based training pipeline that does not rely on sophisticated data annotations or interpolation between multiple models. We continuously balance the weights between the model’s System-1 and System-2 data to eliminate redundant reasoning processes while preserving the model’s reasoning capability. We validate our approach across models on DeepSeek-R1-Distill-7B and DeepSeek-R1-Distill-14B and on a diverse set of benchmarks with varying difficulty levels. Our method significantly reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. Our code and data will be available soon. Comparison of TLDR and baseline models in terms of average accuracy and token compression ratio. Higher values on both axes indicate better performance. 1 Introduction “He that can have patience can have what he will.” — Benjamin Franklin Recent efforts have developed reasoning-oriented Large Language Models (LLMs) capable of solving complex tasks. These models progressed from System 1 to System 2 paradigms ( yu2024distilling21 ; li_system_2_reason ) . System 1 implementations, such as GPT-4o ( GPT4o ) , LLaMA-3 ( llama3 ) , leverage rapid intuitive processing for immediate responses but struggle with complex reasoning tasks. In contrast, System 2 architectures such as DeepSeek-R1 ( Deepseek-R1 ) are fine-tuned with extended thinking chains to promote deliberate analysis through iterative self-assessment, error mitigation, and verification. However, reasoning-oriented LLMs, which employ system-2 reasoning, tend to engage in excessive deliberation even for simple problems. This results in unnecessary exploration and planning, ultimately impairing their efficiency and practicality. Direct approaches aim to addressing the issues of cognitive redundancy and excessive deliberation within reasoning LLMs. Training-free methods CoD ; yao2025activation ; TALE-EP include some that control the internal states of the model during reasoning through prompts or confidence-based techniques to compress the model. Alternatively, the pathway, exemplified by model merging, involves intervening in the parameters of the reasoning LLM to produce relatively concise solutions. Training-based methods primarily focus on sampling and synthesizing relatively concise reasoning paths on specified problem sets through various strategies TokenSkip ; TOPS ; CoT-Valve . These methods involve performing reinforcement learning SimPO ; thinkpruner ; O1-Pruner ; L1 or supervised fine-tuning (SFT) Overthink on reasoning LLMs, enabling the model to learn to generate more concise yet still correct reasoning paths. Such methods typically require careful collection of problems and precise control of the data ratio for different lengths to achieve good results, leading to a complex process of parameter tuning and data construction. For example, TOPS TOPS requires pre-processing steps to manually label SFT data to construct length-sensitive models, while CoT-Valve CoT-Valve generates data by creating intermediate models through model interpolation for sampling. This construction process is often tedious TOPS , computationally expensive L1 , or difficult to control for quality CoT-Valve . Demystifying Short/Long CoT Mixture in LLM Thinking Compression • System-1 data (Short CoT on GSM8K-like easy problems) reduces reasoning redundancy on all problem levels. (Section 2 ) Short CoT reduces reasoning redundancy on simple questions and demonstrates generalization across varying levels of problem difficulty. • System-2 data (Long CoT only on s1-like difficult problems) helps maintain performance. (Section 2 ) Incorporating a small proportion of Long CoT, particularly on challenging problems, can mitigate the accuracy degradation introduced by short CoT, while long CoT on simple questions doesn’t help much. • Dynamic re-weighting of System-1/2 data builds effcient LLM Reasoning Compression. (Section 1 ) Driven by a simple intuition, we design a dynamic reweighting algorithm for system-1/2 data, achieving strong performance in LLM reasoning compression. We investigate the effects of mixing short CoT and long CoT data on compressing reasoning. Our findings suggest that long CoT and short CoT induce divergent optimization directions in the model’s reasoning behavior. Increasing the proportion of short CoT encourages more concise reasoning patterns, but may lead to a decline in reasoning accuracy. In contrast, raising the proportion of long CoT helps preserve reasoning performance on complex tasks, though at the expense of reduced compression efficiency. This naturally raises the question: Can we identify an optimal Long-to-Short data mixture that strikes the best trade-off— maximizing reasoning efficiency while maintaining accuracy? We base our approach on an intuitive motivation: when a model is thinking too long, it should re-weight more intuitive reasoning paths to simplify the thinking process. Conversely, when the thinking is too direct, it should incorporate more slow-thinking reasoning chains to encourage deeper contemplation. We propose a dynamic T hinking L ength D ata R e-Weighting method ( TlDr ), which dynamically balances the model’s complex reasoning using long CoT and efficient reasoning using short CoT data, enabling the model to eliminate redundant cognitive processes. First, we construct short CoT data for simple problems and long CoT data for complex problems. The model begins with an initial ratio and performs reasoning compression using mixed data. After completing a compression cycle, the model re-evaluates the expected benefits of System-1 CoT data and System-2 CoT data to achieve improved performance. Specifically, and in line with intuition, System-1 CoT data can enhance efficiency, so we use an efficiency metric to measure the expected benefit of System-1 data. System-2 CoT, on the other hand, improves reasoning accuracy, and we use an accuracy metric to measure the benefit of System-2 data in terms of reasoning capability. Compared to various methods requiring fine-tuning data with different reasoning lengths, our approach enables dynamic ratio learning by utilizing the self-sampled long CoT data and the short CoT data constructed by the original instruct/base model. Through experiments on DeepSeek-Distill-7B/14B, our model achieves excellent compression results on the 7B/14B models, with only a slight decrease in reasoning capability. 2 Rethinking Short-Long CoT in Thinking Compression We first constructed the short CoT data using simple problems and recorded how, as training steps increased, this subset contributed to token compression and accuracy retention across datasets of varying difficulty in math benchmarks. Figure 1 : Impact of Combining Short CoT and Long CoT in Fixed Ratios on Thinking Compression Performance and Token Cost. We assessed the variation decay rate in output token length and accuracy on datasets of various question difficulty, spanning from GSM8K to AIME. The Normalized Token/Acc metric detail please refer to Equ. 11 and Equ. 12 . We find that short CoT thinking data for simple problems (System-1 data) can help compress the token usage across questions of various difficulty levels . We leverage the short-cut solutions obtained from simple questions in GSM8K to fine-tune the model and then observe the token compression rates and accuracy drop rates across four datasets, ranging from simple to difficult: GSM8K, MATH500, AMC, and AIME. As shown in Figure 1 , directly fine-tuning the long CoT model with short CoT data achieves good length compression for both simple and complex problems. We were pleasantly