Towards Reasoning Ability of Small Language Models
Gaurav Srivastava, Shuxiang Cao, Xuan Wang · 2025
Read Towards Reasoning Ability of Small Language Models by Gaurav Srivastava, Shuxiang Cao, Xuan Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11569.txt.
Description
Gaurav Srivastava, Shuxiang Cao, and Xuan Wang benchmark whether small language models can reason, challenging the belief that reasoning only emerges at large scale. They built a leaderboard comparing seventy-two small models from six families across seventeen reasoning tests, each run three times. They first checked that automatic judging matched human judgment.
Training method and data quality mattered more than raw size. Compressing big models by quantization kept reasoning intact, while pruning damaged it badly. Larger models were steadier against attacks and mid-reasoning slips, yet some small models matched or beat them. The authors foresee strong small reasoners built through careful training rather than scale alone, useful where computing is limited.
Catalog record
| CREATOR | Gaurav Srivastava, Shuxiang Cao, Xuan Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 146 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11569