Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions

Sushrita Rakshit, Hanwen Zhang, Hua Shen · 2026

Read Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions by Sushrita Rakshit, Hanwen Zhang, Hua Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09893.txt.

Introduction

Rakshit and two colleagues studied pseudo-deliberation in language models: reasoning that looks principled while actions drift from stated values. Models are often judged by stated values, yet values do not reliably govern actions, a discrepancy called the value-action gap. The gap persists even under explicit reasoning, the paper argues, revealing a deeper failure mode.

To measure it, the team built Valdi, with 4,941 human scenarios across five domains, three tasks covering value stating, reasoning, and action, and five metrics of value fit. Across closed and open-source models, expressed values often clashed with downstream dialogues. A multi-agent value checker, ViValdi, stepped in at several stages. Post-hoc auditing and rewriting of final dialogues beat fixes aimed at middle reasoning, hinting action-level fit at output time matters most.

Catalog record

CREATORSushrita Rakshit, Hanwen Zhang, Hua Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE117 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09893

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions Sushrita Rakshit, Hanwen Zhang, Hua Shen Abstract Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions—a discrepancy termed value–action gap . In this work, we argue that this gap persists even under explicit reasoning, revealing a deeper failure mode we call “Pseudo-Deliberation”: the appearance of principled reasoning without corresponding behavioral alignment. To study this systematically, we introduce Valdi , a framework for measuring alignment between stated values and generated dialogue. Valdi includes 4,941 human-centered scenarios across five domains, three tasks that elicit value articulation, reasoning, and action, and five metrics for quantifying value adherence. Across both proprietary and open-source LLMs, we observe consistent misalignment between expressed values and downstream dialogues. To investigate intervention strategies, we propose ViValdi , a multi-agent value auditor that intervenes at different stages of generation. While interventions targeting intermediate reasoning aim to shape value formation, we find post-hoc auditing and rewriting of final dialogues yields more robust alignment gains. These findings suggest that value-action alignment in LLMs may not emerge from deliberative reasoning alone, and highlight the need for action-level alignment strategies at the point of generation 1 1 1 Codes and data will be released upon acceptance. . 1 Introduction Large Language Models (LLMs) are increasingly entrusted with decisions that shape human lives – advising clinicians on diagnoses, guiding students through their education, and screening candidates in hiring pipelines [ 13 , 25 , 34 ] . As these systems migrate from controlled benchmarks to consequential conversations, a fundamental question becomes urgent [ 30 ] : do the values an LLM professes actually govern the actions it generates? Recent work has identified the “value–action gap [ 29 ] ,” where LLMs’ stated value preferences diverge from the actions they recommend. However, this gap has primarily been evaluated in static multiple-choice settings [ 27 , 29 ] . In practice, LLMs act through situated dialogue, where advice is delivered conversationally and shaped through interaction. This raises a critical, underexplored issue: if a model endorses a value in principle but abandons it under generation pressure, alignment evaluations based on stated preferences alone are dangerously misleading. A natural hope is that deliberative reasoning closes this gap. Chain-of-thought and structured reasoning [ 33 ] have become the dominant levers for aligning LLM behavior, and the appeal of these methods rests on a tacit assumption: that explicit reasoning over values yields actions consistent with those values. However, we found that LLMs frequently engage in pseudo-deliberation – the surface form of principled reasoning without the corresponding behavioral alignment. In this regime, reasoning traces faithfully invoke values the model endorses, yet the resulting action systematically violates them; worse, as we show, deliberation can amplify rather than repair misalignment . To study pseudo-deliberation systematically, we introduce Valdi ( V alue AL ignment in DI alogue), a framework for quantifying the gap between an LLM’s stated values and the values that surface in its generated dialogue. Valdi pairs a curated dataset of 4,941 advice-seeking scenarios spanning five domains – Daisy (Decisions AI Steers for You)—with three diagnostic tasks (explicit value inclination, fast dialogue generation, and slow reasoning-then-dialogue) and a suite of five complementary metrics that decompose misalignment into rate, distance, and the value-level dynamics of survival , suppression , and emergence . Across two proprietary and two open-weight LLMs, Valdi reveals systematic misalignment between endorsed and enacted values. In most models, slow deliberative generation reduces alignment relative to fast generation, often introducing additional conflicts between reasoning traces and final responses. Figure 1 : Overview of Valdi framework. We generate the DAISY dataset. Then we generate Fast and Slow Dialogue. Finally, we evaluate model’s dialogue and reasoning via our alignment metrics. If reasoning alone cannot enforce alignment, what can? We propose ViValdi , a multi-agent value auditor for action-level alignment that intervenes at distinct stages of generation. We study three variants: a reasoning-level auditor that critiques and revises intermediate reasoning before the final dialogue is produced ( ViValdi-R ); a dialogue-level auditor that rewrites only the final response post hoc ( ViValdi-D ); and a combined variant that applies both interventions sequentially ( ViValdi-R&D ). Counterintuitively, the post-hoc auditor delivers consistently larger and more robust alignment gains—reinforcing our central thesis: value–action alignment is a property of the action , not of the deliberation that precedes it. Our contributions are threefold: • Valdi framework and Daisy dataset. We release the first evaluation suite for situated value–action alignment in dialogue: 4,941 human-curated scenarios, three reasoning conditions, and five alignment metrics that decompose misalignment into survival, suppression, and emergence. • Pseudo-deliberation and empirical evidence . We define pseudo-deliberation as a reasoning-based failure mode where deliberative reasoning generation can systematically reduce value–action alignment relative to fast generation, challenging the assumption that more reasoning yields more faithful behavior alignment. • ViValdi : a multi-agent value auditor for action-level alignment. We introduce three multi-agent interventions and show that auditing the output outperforms auditing the reasoning , motivating action-level alignment as a necessary complement to reasoning-based methods. 2 Valdi Framework: Measuring Situated Value-Action Gaps in Dialogues The key components of our framework involve a value-driven dataset (§ 2.1 ) stated value and dialogue generation (§ 2.2 ), a measurement tool for extracting model’s stances through text § 2.3 , and a suite of metrics to quantify alignment and decompose reasoning formulation (§ 2.4 ). 2.1 DAISY Dataset Generation We introduce DAISY (Decisions AI Steers for You), a dataset of 4,941 guidance-seeking dilemmas designed to evaluate value–action alignment in realistic decision-making contexts. The dataset spans diverse domains (e.g., social, ethical, and personal tradeoffs) and is constructed from a mixture of 6 social dialogue datasets [ 26 , 35 , 8 , 31 , 15 , 6 ] and controlled generation. A human annotator first filters scenarios according to predefined criteria. To increase value diversity of selected scenarios, we generate paraphrases using GPT-4o ( T=0.7 , max_tokens=500), producing multiple formulations of the same dilemma with varied surface realizations [ 21 ] . Crucially, all scenarios are rewritten to exclude explicit mentions of values, requiring models to implicitly infer and balance competing values during generation. This design enables a controlled evaluation of value–action gaps under naturalistic conditions, where value expression must emerge from behavior rather than prompting. Additional details regarding DAISY’s construction and content are provided in Appendix D . 2.2 Three Tasks for Stated Values and Situated Actions We define three tasks to assess LLMs’ value expression across stages of generation: (1) stating value inclinations, (2) generating immediate dialogue, and (3) producing deliberative dialogue following structured reasoning, enabling evaluation of whether values expressed in isolation are preserved in downstream interaction. While we leverage Shalom Schwartz’s Theory of Basic Values to evaluate our model, our framework is extendable to other value orientations [ 28 ] . Task 1: Stated Value Inclinations. For each scenario, the LLM evaluates the extent to which each of the 10 Schwartz values should be encouraged when advising the user, using a 3-point Likert scale: Encourage , Neutral , or Discourage , which we map to \{+1,0,-1\} respectively. This yields a signed value vector representing the model’s

Read the complete text