Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
Jingyi Sun Qianli Wang Pepa Atanasova Nils Feldhus Isabelle Augenstein · 2026
Read Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization by Jingyi Sun Qianli Wang Pepa Atanasova Nils Feldhus Isabelle Augenstein (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24960.txt.
Introduction
Jingyi Sun leads a study of whether chain-of-thought traces truly mirror a language model's inner behavior. It is usually judged two ways: by perturbing inputs or traces, and by intervening on parametric knowledge. Past work compared those paradigms only descriptively. FaithMATE unifies them as one tuning setup for either paradigm. Across three models, two data sets, and six metrics, the paradigms prove positively coupled yet asymmetric. Tuning toward stored knowledge lifts both sides, while tuning toward context gains vary, so the goal needs many-sided tuning and judging.
Catalog record
| CREATOR | Jingyi Sun Qianli Wang Pepa Atanasova Nils Feldhus Isabelle Augenstein |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76286 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24960
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization Jingyi Sun Qianli Wang Pepa Atanasova Nils Feldhus Isabelle Augenstein Abstract Chain-of-Thought (CoT) faithfulness, i.e., whether CoTs genuinely reflect large language models’ (LLM) underlying behavior, is typically evaluated under two disjoint paradigms: contextual faithfulness , measured by perturbing the input or CoT trace, and parametric faithfulness , assessed by intervening on a model’s parametric knowledge. Yet prior work compares them only descriptively. We fill this gap by proposing FaithMATE , a unified preference-alignment interface for optimizing models towards either faithfulness paradigm. 1 1 1 The code will be released upon publication. It enables us to investigate the interplay between the two paradigms, examining whether and to what extent faithfulness gains generalize within and across paradigms. Across three models, two datasets, and six faithfulness metrics, we find that the two paradigms are positively coupled, yet asymmetric: optimizing towards parametric faithfulness yields consistent gains across both paradigms, whereas the contextual counterpart delivers more variable gains. Within the contextual paradigm, faithfulness gains on one metric do not consistently transfer to others, implying that existing contextual metrics capture disjoint facets of faithfulness and exposing inherent trade-offs. These findings imply that CoT faithfulness is not a monolithic objective and therefore requires multifaceted optimization and evaluation. 1 1 footnotetext: Equal contribution and share first-authorship. 1 Introduction CoT has proven effective for improving LLMs’ performance on various reasoning tasks 11 ; 27 . Alongside this success, however, questions have been raised about CoT faithfulness, i.e., whether CoTs accurately reflect the model’s actual reasoning process. In practice, CoTs can appear plausible to users while being weakly coupled with a model’s internal computations 2 ; 3 ; 18 ; 31 . Figure 1: Overview of our optimization-based meta-evaluation framework. We sample CoTs from a base model, score them using contextual or parametric faithfulness metrics, construct preference pairs, and optimize the model towards selected faithfulness objectives. We then evaluate cross-metric transfer across both faithfulness dimensions. Existing work evaluates CoT faithfulness primarily through two paradigms. Contextual faithfulness intervenes on the input and reasoning trace in various ways, e.g., by inserting an error 13 , or injecting answer hints into the context 25 ; 5 , and deems a CoT faithful if the intervention is truly reflected. However, such measures only consider contextual consistency instead of model internals. Parametric faithfulness , in contrast, intervenes on a model’s parameters rather than its context: it deems a CoT step faithful if unlearning the knowledge underlying that step changes the answer ( 26 ) . While this perspective targets model internals directly, it is limited to the knowledge explicitly verbalized by the CoT, neglecting information that shapes CoT generation but remains unexpressed. Although the two paradigms have largely been studied in isolation, they can yield divergent faithfulness conclusions for the same CoTs 31 , raising the question of whether they capture a shared notion of faithfulness or reward different properties of CoTs. As methods for improving CoT faithfulness proliferate, observational comparisons on fixed CoTs cannot reveal what each faithfulness paradigm rewards under optimization or whether and to what extent optimizing one transfers to the other. To this end, we introduce FaithMATE ( Faith fulness M etric A lignment and T ransfer E valuation), a unified framework for investigating CoT faithfulness under optimization. It constructs faithfulness-targeted preference pairs, allowing models to be optimized towards either contextual or parametric faithfulness (Figure 1 ). This unified optimization interface enables an optimization-based investigation of the interplay between the two paradigms, while also allowing us to compare what each rewards under optimization. Experiments on three models, two datasets, and six faithfulness metrics reveal the following findings. First, across faithfulness paradigms, contextual and parametric faithfulness exhibit a positively coupled, yet asymmetric relationship (§ 5.1 ): optimizing for parametric faithfulness delivers more consistent improvements across both paradigms, while contextual optimization, although occasionally larger in magnitude, is less consistent. This positions parametric faithfulness as the more reliable target for broad generalization. Second, within the contextual faithfulness paradigm, faithfulness gains from optimizing one contextual metric do not consistently transfer to others, e.g., between Adding Mistake and Paraphrasing (§ 5.2 ). This suggests that contextual faithfulness metrics are better understood as a family of related but distinct properties, rather than interchangeable measurements of a single latent quantity. Thus, we call attention to potential tensions among contextual faithfulness metrics. Additional experiments show that task-arithmetic merging of models, each optimized for a specific faithfulness target, can partially mitigate this inconsistency and amplify the faithfulness improvements. Third, we qualitatively characterize what different metrics actually reward under optimization (§ 5.4 ). We find that contextual metric-shared gains are typically driven by improved alignment between reasoning and final answer, whereas contextual metric-specific gains are mostly prompted by the mitigation of post-hoc rationalization 2 2 2 A model decides on the answer and then generates a plausible justification that explains the answer rather than reflects how it actually arrived at it 25 . . By contrast, parametric gains arise from more explicit grounding in task-relevant facts and knowledge. This suggests that evaluating and reporting gains on a single faithfulness metric can be misleading, as improvements may reflect distinct underlying behaviors that are not interchangeable. 2 Related Work CoT Faithfulness Metrics. Prior work has mainly studied contextual faithfulness , which intervenes on the CoT or input and checks whether the intervention is reflected in the answer. Corruption-based methods perturb the CoT, and measure whether the model’s answer changes ( 13 ; 30 ) . Attribution-based methods measure the alignment between input contributions to CoT generation and those to answer prediction ( 17 ; 1 ) . Another line of work uses Biasing Features, which inject answer hints and check whether the CoT verbalizes them ( 25 ; 5 ; 6 ; 29 ) . Recent work further proposes parametric faithfulness , which intervenes on a model’s parametric knowledge rather than on the CoT. 26 unlearn the knowledge involved in individual CoT steps from the model parameters and measure whether the answer changes after unlearning. Given that contextual and parametric metrics target different aspects of CoT faithfulness, we evaluate both families within our unified FaithMATE framework under optimization. Meta-Evaluation of CoT Faithfulness. 17 compare multiple contextual faithfulness metrics and find that they often assign divergent faithfulness verdicts to the same CoTs, suggesting that these metrics are better understood as measures of output-level self-consistency. 30 use model editing to construct faithful–unfaithful CoT pairs and evaluate whether the target faithfulness metrics rank the faithful CoTs higher. More recent work extends metric comparison to parametric faithfulness. 31 show that contextual and parametric faithfulness metrics can yield different conclusions on the same CoTs. Such observational comparisons, however, leave open how the two paradigms interact when models are explicitly optimized for faithfulness. FaithMATE fills this gap with a unified interface for optimizing either paradigm, enabling controlled study of faithfulness improvement transfers across metrics and paradigms. CoT Faithfulness Improvement. Several methods have been proposed to improve CoT faithfulness. Nevertheless, they are designed to optimize only one specific type of faithfulness . FRODO trains the model to prefer correct reasoning chains over counterfactual ones 18 . 14 use integrated-gradient attribution to recall question information missing from the CoT, and