Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models
Aryan KasatTCS AI Practice, Smriti SinghUT Austin, Aman ChadhaAmazon GenAI, Vinija JainGoogle GenAI · 2026
Read Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models by Aryan KasatTCS AI Practice, Smriti SinghUT Austin, Aman ChadhaAmazon GenAI, Vinija JainGoogle GenAI (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21854.txt.
Introduction
Kasat and co-authors ask whether language models truly reason about morals or only sound like they do. They scored over 600 answers from 13 models on six classic dilemmas, using judge models cross-checked against each other. Answers clustered at Kohlberg's highest stages, the reverse of human norms where stage four dominates. Some models showed moral decoupling: their stated reasons clashed with the action they chose. Scale helped slightly, and training type had no effect. The authors call the pattern moral ventriloquism, mature rhetoric without the growth path behind it.
Catalog record
| CREATOR | Aryan KasatTCS AI Practice, Smriti SinghUT Austin, Aman ChadhaAmazon GenAI, Vinija JainGoogle GenAI |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52171 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21854
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reasoning or Rhetoric? An Empirical Analysis of Moral Reasoning Explanations in Large Language Models Aryan KasatTCS AI Practice, Smriti SinghUT Austin, Aman ChadhaAmazon GenAI, Vinija JainGoogle GenAI Abstract Do large language models reason morally, or do they merely sound like they do? We investigate whether LLM responses to moral dilemmas exhibit genuine developmental progression through Kohlberg’s stages of moral development, or whether alignment training instead produces reasoning- like outputs that superficially resemble mature moral judgment without the underlying developmental trajectory. Using an LLM-as-judge scoring pipeline validated across three judge models, we classify more than 600 responses from 13 LLMs spanning a range of architectures, parameter scales, and training regimes across six classical moral dilemmas, and conduct ten complementary analyses to characterize the nature and internal coherence of the resulting patterns. Our results reveal a striking inversion: responses overwhelmingly correspond to post-conventional reasoning (Stages 5–6) regardless of model size, architecture, or prompting strategy, the effective inverse of human developmental norms, where Stage 4 dominates. Most strikingly, a subset of models exhibit moral decoupling : systematic inconsistency between stated moral justification and action choice, a form of logical incoherence that persists across scale and prompting strategy and represents a direct reasoning consistency failure independent of rhetorical sophistication. Model scale carries a statistically significant but practically small effect ( F ( 2,229 ) = 6.05 F(2,229)=6.05 , p = 0.003 p=0.003 , η 2 = 0.050 \eta^{2}=0.050 , d = 0.55 d=0.55 ); training type has no significant independent main effect ( p = 0.065 p=0.065 ); and models exhibit near-robotic cross-dilemma consistency (ICC > > 0.90), producing logically indistinguishable responses across semantically distinct moral problems. We posit that these patterns constitute evidence for moral ventriloquism : the acquisition, through alignment training, of the rhetorical conventions of mature moral reasoning without the underlying developmental trajectory those conventions are meant to represent. 1 1 footnotetext: Equal contribution. 2 2 footnotetext: Work done outside role at the authors’ respective companies. 1 Introduction A central question in AI alignment is not just whether large language models (LLMs) produce morally acceptable outputs, but how the moral reasoning explanations they generate relate to the processes that produce those outputs. Modern LLMs frequently generate detailed, sophisticated-sounding explanations when confronted with moral dilemmas, invoking abstract principles such as human dignity, social contracts, and universal rights. These behaviors are often interpreted as evidence that models possess genuine moral reasoning capabilities. At the same time, a growing body of research has raised serious concerns about interpreting model outputs as evidence of genuine underlying processes. Several studies suggest that LLMs may produce convincing reasoning patterns through statistical pattern completion rather than through explicit reasoning mechanisms ( 14 ; 4 ) . In particular, recent work questions whether chain-of-thought explanations correspond to the internal computational processes that generate model predictions, or whether intermediate tokens instead reflect learned stylistic patterns associated with reasoning discourse ( 8 ) . These concerns raise an important challenge: if models can produce explanations that resemble reasoning without performing the underlying reasoning processes, behavioral evaluations based solely on model outputs may provide a systematically misleading picture of model capability. Moral dilemmas are a particularly informative setting because they reliably elicit structured, principle-grounded reasoning explanations. Crucially, if LLMs genuinely develop moral reasoning capabilities as a function of scale and training, we would expect their outputs to mirror the developmental trajectory observed in humans, varying across contexts, progressing with scale, and converging toward the Stage 4 distribution that characterizes most human adults, not clustering uniformly at the highest stages regardless of model size or training procedure. To investigate this question, we conduct a large-scale empirical study evaluating moral reasoning patterns across 13 state-of-the-art LLMs, conducting ten quantitative analyses to characterize the nature, robustness, and internal coherence of these patterns. Figure 1 previews two headline findings that shape our investigation. Our methodology is motivated by the following research questions: RQ1 Does model scale predict higher-stage moral reasoning explanations? RQ2 Does prompting strategy systematically influence moral stage? RQ3 Are model stage assignments consistent across dilemmas, and how does this compare to human moral variability? RQ4 Do LLM stage distributions resemble human developmental norms? RQ5 Do models practice what they preach: does stated moral reasoning correspond to the action choices models produce? RQ6 Does scale affect moral reasoning independent of training type, or does training dominate once scale is controlled? (a) Stage profile across dilemmas by prompt type (radar charts). Each model’s hexagonal profile is nearly identical across all six dilemmas and all three prompt types, illustrating near-zero cross-dilemma and cross-prompt variability. (b) Per-model stage proportion vs. human adult baseline (dashed red). All 13 LLMs concentrate responses in Stages 5–6; Jensen–Shannon divergences from the human baseline are large ( \mathrm{JSD}\approx 0.20 – 0.76 , all \chi^{2}\!:p{<}0.001 ), confirming the distributional inversion. Figure 1: Two Headline Anomalies. Left: Nearly identical radar profiles across all six morally distinct dilemmas and three prompt types reveal a cross-dilemma rigidity inconsistent with context-sensitive human moral reasoning. Right: Per-model stage distributions diverge sharply from the human adult baseline — LLMs produce the inverse of the Stage 4-dominant human pattern, clustering uniformly in post-conventional Stages 5–6. Together these patterns motivate the moral ventriloquism hypothesis. Our results reveal a striking inversion: across nearly all evaluated models, responses overwhelmingly correspond to Kohlberg’s post-conventional stages (Stages 5–6), the effective inverse of the Stage 4-dominant human distribution, with remarkably little variation across model size, architecture, or prompting strategy. A subset of models additionally exhibit moral decoupling: producing high-stage justifications for low-stage action choices, and all models show near-robotic cross-dilemma consistency that is itself anomalous relative to human moral cognition. Our primary contribution is the hypothesis of moral ventriloquism : the acquisition, through alignment training, of the rhetorical conventions of mature moral reasoning without the underlying developmental trajectory. We use Kohlberg’s framework as methodological scaffolding, exploiting its well-characterized human distribution as a diagnostic baseline rather than making claims about moral development per se. We provide convergent empirical evidence through ten complementary analyses spanning distributional comparisons, cross-dilemma consistency, action-reasoning alignment, linguistic profiling, and factorial decomposition of scale and training effects. 2 Related Work 2.1 Reasoning Faithfulness and Moral Evaluation in LLMs Chain-of-thought prompting ( 15 ) generated widespread interest in using reasoning traces as a window into model cognition, but evidence quickly accumulated that this window may be opaque. 14 demonstrated that CoT explanations can systematically misrepresent the actual causes of model predictions, producing fluent justifications that do not correspond to the features that drove the prediction. 4 extended this finding to state-of-the-art reasoning models, showing that even Claude 3.7 Sonnet and DeepSeek R1 routinely generate reasoning traces that do not reflect their actual inference process. 8 argue more fundamentally that auto-regressive LLMs function as non-veridical memory systems, with chain-of-thought traces better understood as learned stylistic registers than faithful records of deliberation. Together, these results motivate our core concern: if moral reasoning justifications can be fluent without being causally connected to model outputs, then stage-based evaluation of those justifications may not measure what it purports to measure. On the evaluation side, 2 established the large-scale human baseline for moral preferences we use as our reference distribution. 13 treat LLM moral responses as direct evidence of “beliefs encoded