Faithfulness Metrics Don’t Measure Faithfulness: A Meta-Evaluation with Ground Truth

Yoav Gur-Arieh, Ana Marasović, Mor Geva · 2026

Read Faithfulness Metrics Don’t Measure Faithfulness: A Meta-Evaluation with Ground Truth by Yoav Gur-Arieh, Ana Marasović, Mor Geva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25052.txt.

Introduction

Yoav Gur-Arieh, Ana Marasovi\ć, and Mor Geva ask whether faithfulness metrics for chains of thought actually measure faithfulness. Such traces often fail to reflect the computations behind predictions, and ground truth is scarce because internal computations cannot be observed. The authors build tasks whose outputs reveal the required computations, plus an automated labeling pipeline. That yields BonaFide: 3,066 labeled chains across 13 tasks and 10 models. Most metrics perform near chance and degrade on longer chains. The best reaches only 0.70 AUROC at chain level, another 0.59 at step level.

Catalog record

CREATORYoav Gur-Arieh, Ana Marasović, Mor Geva
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73125
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.25052

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Faithfulness Metrics Don’t Measure Faithfulness: A Meta-Evaluation with Ground Truth Yoav Gur-Arieh, Ana Marasović, Mor Geva Abstract Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithfully represent the computations behind a model’s predictions. Several faithfulness metrics have been proposed, but whether they indeed measure faithfulness remains unknown. Answering this requires ground-truth labels, which are hard to obtain since internal computations are not directly observable. Consequently, most works proposing metrics report only absolute scores or comparisons to prior metrics, and the few existing benchmarks rely on proxies like plausibility or importance, properties orthogonal to faithfulness that can mislead about whether a CoT can be trusted. We address this challenge by constructing tasks whose outputs reveal which intermediate computations must have produced them, and developing an automated labeling pipeline that yields ground-truth faithfulness labels at both the step and CoT level. Building on this methodology, we present BonaFide , a benchmark of 3,066 labeled CoTs across 13 tasks and 10 models, and use it to conduct the first systematic evaluation of prominent faithfulness metrics. Our experiments show that most metrics perform near chance, exhibit strong prediction biases and degrade on longer CoTs. The best metric reaches only 0.70 AUROC at the CoT level while another reaches 0.59 at the step level, with neither transferring across settings, while entailing prohibitively high computational cost. Our results expose fundamental gaps in current faithfulness evaluation and call for the development of more reliable and efficient metrics. github.com/BonaFide huggingface.co/BonaFide 1 Introduction Large language models (LLMs) increasingly generate long chains of thought (CoTs) to solve complex problems, whether as step-by-step solutions or extended internal deliberations [ 1 , 2 , 3 , 4 ] . These reasoning traces now underpin dominant approaches to interpreting and auditing model behaviors [ 5 , 6 ] . However, a growing body of work calls this promise into question. CoTs have been shown to systematically misrepresent the factors influencing a model’s predictions [ 7 , 8 , 9 ] and to serve as post-hoc rationalizations for predetermined answers rather than genuine reasoning [ 10 , 11 ] . Determining whether CoTs faithfully represent models’ internal computations is therefore a critical challenge. To address these concerns, recent work has proposed various metrics for evaluating CoT faithfulness [ 10 , 12 , 13 , 14 ] . However, it is unknown whether these metrics measure faithfulness in practice. Validating them requires ground-truth labels indicating which CoTs are faithful and which are not, a task that is inherently difficult since the internal computations of LLMs are not directly observable. Consequently, existing evaluations of faithfulness metrics have had to rely on indirect proxies such as plausibility [ 15 ] , which is neither necessary nor sufficient for faithfulness [ 16 , 17 ] . Figure 1 : Our methodology enabling the collection of ground-truth faithfulness labels. (A) We design tasks, diversionary and outright, which require the model to perform intermediate computations to arrive at certain answers. (B) We collect CoTs from 10 models on these tasks and apply our step labeling pipeline, which yields labels for steps and CoTs at high precision. (C) The result is BonaFide , a benchmark of labeled CoTs and steps for evaluating faithfulness metrics. In this work, we present BonaFide , the first benchmark to provide ground-truth faithfulness labels for CoTs and CoT steps, enabling direct validation of faithfulness metrics (see Figure 1 for an overview). Our approach builds on a key observation that, for certain task designs, producing a given output requires the model to have performed specific intermediate computations. For example, suppose a model is given the question “Who painted Starry Night?” alongside a misleading hint suggesting Da Vinci, an answer it would not produce naturally [ 7 ] . If the model answers Da Vinci, we know it must have followed the hint. Thus, we can check whether its CoT reflects this or fabricates an independent justification. This gives us ground-truth labels: if the model’s CoT acknowledges following the hint, then that step is faithful; if it omits or obscures this, then the CoT is unfaithful. We apply this principle broadly, creating tasks using both misleading hints and intrinsic problem structure to infer the computations the model must have performed. These tasks are designed to preclude guessing and shortcuts so that the identified ground-truth computations cannot be bypassed. We follow this methodology to construct a dataset, developing an automated labeling pipeline that produces both step- and CoT-level faithfulness labels. The pipeline was validated against human annotation, achieving 98.9% precision. We apply this pipeline to CoTs from ten LMs ranging from 4B to 70B parameters, spanning the Qwen3 [ 18 ] , OLMo 3 [ 19 ] , DeepSeek-R1-Distill [ 4 ] , and Llama 3.3 [ 20 ] families, and covering both reasoning and non-reasoning variants, across 13 diverse tasks. The resulting benchmark, called BonaFide , contains labels for 1,120 CoTs and 1,946 CoT steps, balanced across models and label types from a larger unfiltered dataset released alongside it. We use BonaFide to provide the first systematic evaluation of how accurately existing faithfulness metrics identify faithful and unfaithful CoTs, as well as steps, and at what computational cost. We find that most metrics perform near chance, with no single metric reliably distinguishing faithful from unfaithful CoTs at both the step and CoT level. The strongest metric differs across settings (CC-SHAP at 0.70 AUROC for CoTs, Filler Tokens at 0.59 for steps), with neither transferring to the other, and most metrics exhibit strong prediction biases, with most importance-based metrics labeling 90%–96% of CoTs unfaithful and semantic-utility metrics labeling 94%–96% faithful on a balanced sample. Many of these metrics are also prohibitively expensive to run on long CoTs, with CC-SHAP requiring up to 10^{3} seconds per instance, making them impractical for real-time monitoring. To summarize, our work makes the following contributions: (a) we revise the definition of faithfulness for the CoT setting, where existing definitions, designed for static post-hoc explanations, do not cleanly apply; (b) we develop a methodology for obtaining such labels, based on task designs that make a model’s internal computations partially recoverable from its outputs; (c) we present BonaFide , the first benchmark evaluating faithfulness metrics using ground-truth labels, enabling direct evaluation rather than relying on indirect proxies; and (d) we use BonaFide to evaluate existing faithfulness metrics, finding that none currently provide a reliable, practical measure of faithfulness. Our results call for the development of more accurate and efficient faithfulness metrics. To facilitate this, we release BonaFide and the evaluation code to the community at huggingface.co/collections/yoavgurarieh/bonafide . 2 Revising chain-of-thought faithfulness To evaluate faithfulness metrics, we must first define what it means for a CoT or CoT step to be faithful. In this section, we revisit prior approaches to CoT faithfulness in § 2.1 , and extend established definitions of faithfulness to the CoT setting in § 2.2 . Finally, we differentiate faithfulness from other properties often conflated with it, plausibility and importance, in § 2.3 . 2.1 CoT faithfulness needs new definitions Faithfulness has long been recognized as a desideratum for model explanations [ 21 , 22

Read the complete text