Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models
Bin Yu, Hang Yuan, Haotian Li, Xueyin Xu, Yuliang Wei, Bailing Wang, Weizhen Qi, Kai Chen · 2025
Read Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models by Bin Yu, Hang Yuan, Haotian Li, Xueyin Xu, Yuliang Wei, Bailing Wang, Weizhen Qi, Kai Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03469.txt.
Description
Bin Yu and seven co-authors teach language models to reason in fewer words. Their paper is from two thousand twenty-five. Models trained on long, step-by-step solutions from a big teacher model tend to overthink. They write out long, rambling chains even for easy problems, which wastes time and money.
Their fix mixes long solutions with short rewrites of the same solutions. The rewrites keep the structure of the reasoning but trim the fat. Models trained on the mix scored about two percent higher on tests, on average. At the same time, their answers shrank by nearly half. The method gives smaller models real reasoning skill without the bloat. It is a simple change to a standard training recipe.
Catalog record
| CREATOR | Bin Yu, Hang Yuan, Haotian Li, Xueyin Xu, Yuliang Wei, Bailing Wang, Weizhen Qi, Kai Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-03469