Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability
Chiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao · 2025
Read Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability by Chiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24147.txt.
Description
Chiwei Zhu and six co-authors test whether taught reasoning trails truly help a language model, or can harm it. Many past works say such trails lift scores, and the claim is now stock. Their 2025 study grades trails on two axes at once: end score, and reliability, how sound the acts of the model stay. The pair view gives a fresh split from past readings.
At times trails cut the score they were meant to lift. At times they raised reliability past even the raw model with no trails. Score shift and reliability shift moved in line, and both were driven by how hard the task is. Easy tasks and hard tasks pulled the pair in split ways. The team draws rules for when to use trails, and flags care in how human thought is baked into models.
Catalog record
| CREATOR | Chiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-24147