LExT: Towards Evaluating Trustworthiness of Natural Language Explanations

Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran · 2025

Read LExT: Towards Evaluating Trustworthiness of Natural Language Explanations by Krithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06227.txt.

Description

Krithi Shailya and co-authors propose, in 2025, a way to score how trustworthy a language model's plain-text explanations really are. Standard scoring tools check wording overlap, but an explanation can read well while being wrong or unfaithful to the model's actual reasoning. Their framework combines two qualities: how plausible the explanation sounds and how faithful it is.

The combined score is applied to six models on public medical data, mixing specialist and general-purpose systems. Results differed widely. General-purpose models sometimes sounded better yet showed weaker faithfulness than tuned medical models. The authors argue explanation scores must test truth, not just style, before use in sensitive fields.

Catalog record

CREATORKrithi Shailya, Shreya Rajpal, Gokul S Krishnan, Balaraman Ravindran
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-06227

Browse the full library · About the Tower of Babel