Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability

Chiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao · 2025

Read Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability by Chiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24147.txt.

Description

Chiwei Zhu and six co-authors test whether taught reasoning trails truly help a language model, or can harm it. Many past works say such trails lift scores, and the claim is now stock. Their 2025 study grades trails on two axes at once: end score, and reliability, how sound the acts of the model stay. The pair view gives a fresh split from past readings.

At times trails cut the score they were meant to lift. At times they raised reliability past even the raw model with no trails. Score shift and reliability shift moved in line, and both were driven by how hard the task is. Easy tasks and hard tasks pulled the pair in split ways. The team draws rules for when to use trails, and flags care in how human thought is baked into models.

Catalog record

CREATORChiwei Zhu, Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Zhendong Mao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24147

Browse the full library · About the Tower of Babel