LExT: Towards Evaluating Trustworthiness of Natural Language Explanations
Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran · 2025
Read LExT: Towards Evaluating Trustworthiness of Natural Language Explanations by Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06227.txt.
Description
Krithi Shailya and co-authors propose, in 2025, a way to score how trustworthy a language model's plain-text explanations really are. Standard scoring tools check wording overlap, but an explanation can read well while being wrong or unfaithful to the model's actual reasoning. Their framework combines two qualities: how plausible the explanation sounds and how faithful it is.
The combined score is applied to six models on public medical data, mixing specialist and general-purpose systems. Results differed widely. General-purpose models sometimes sounded better yet showed weaker faithfulness than tuned medical models. The authors argue explanation scores must test truth, not just style, before use in sensitive fields.
Catalog record
| CREATOR | Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-06227