The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas
Ya Wu, Qiang Sheng, Danding Wang, Guang Yang, Yifan Sun, Zhengjia Wang, Yuyan Bu, Juan Cao · 2025
Read The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas by Ya Wu, Qiang Sheng, Danding Wang, Guang Yang, Yifan Sun, Zhengjia Wang, Yuyan Bu, Juan Cao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18154.txt.
Introduction
Ya Wu and co-authors probe, in a 2025 paper, how large language models weigh values as a moral bind grows step by step. Most tests of model morals ask for one call on one scene. Real binds evolve, and a model's stance may shift as the stakes rise. Their data set, MMDs, holds 3,302 binds, each in five stages of rising strain. Nine wide-used models were run through the full set. The team read how each model's picks moved from stage to stage.
Value picks shifted in a clear way as binds grew hard. Care, the worth of tending to others, was most often put first. In some scenes, fairness passed care as the bind tightened. The shift shows model morals are not fixed ranks but bend with context. The team calls for tests that move, as real binds do.
Catalog record
| CREATOR | Ya Wu, Qiang Sheng, Danding Wang, Guang Yang, Yifan Sun, Zhengjia Wang, Yuyan Bu, Juan Cao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18154
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Staircase of Ethics: Probing LLM Value Priorities through Multi-Step Induction to Complex Moral Dilemmas Ya Wu, Qiang Sheng, Danding Wang, Guang Yang, Yifan Sun, Zhengjia Wang, Yuyan Bu, Juan Cao Abstract Ethical decision-making is a critical aspect of human judgment, and the growing use of LLMs in decision-support systems necessitates a rigorous evaluation of their moral reasoning capabilities. However, existing assessments primarily rely on single-step evaluations, failing to capture how models adapt to evolving ethical challenges. Addressing this gap, we introduce the Multi-step Moral Dilemmas (MMDs), the first dataset specifically constructed to evaluate the evolving moral judgments of LLMs across 3,302 five-stage dilemmas. This framework enables a fine-grained, dynamic analysis of how LLMs adjust their moral reasoning across escalating dilemmas. Our evaluation of nine widely used LLMs reveals that their value preferences shift significantly as dilemmas progress, indicating that models recalibrate moral judgments based on scenario complexity. Furthermore, pairwise value comparisons demonstrate that while LLMs often prioritize the value of care, this value can sometimes be superseded by fairness in certain contexts, highlighting the dynamic and context-dependent nature of LLM ethical reasoning. Our findings call for a shift toward dynamic, context-aware evaluation paradigms, paving the way for more human-aligned and value-sensitive development of LLMs. = Date : May 23, 2025 = Project : https://isir-wuya.github.io/Multi-step-Moral-Dilemmas/ \emails { wuya23s , shengqiang18z , wangdanding , sunyifan23z , wangzhengjia21b , buyuyan22s , caojuan }@ict.ac.cn, [email protected] Figure 1: Comparison of existing value evaluation protocols and ours for LLMs. Instead of asking a single question or situating an isolated moral dilemma, our proposed MMDs framework sets a multi-step moral dilemma questionnaire to progressively induce models into stronger and more complex ethical conflicts to expose their underlying value priorities. 1 Introduction As the capabilities of large language models (LLMs) continue to evolve ( Achiam et al. 2023 , DeepSeek-AI 2024 ) , their deployment in high-stakes domains—from resume screening ( Dastin 2022 ) to psychological counseling ( Souza et al. 2024 ) —has intensified debates about their ability to navigate dynamic moral landscapes. These sensitive domains demand not only task competence but also temporal consistency in value alignment ( Ji et al. 2023 ) , a challenge exacerbated by LLMs’ lack of intrinsic moral reasoning yet emergent behavioral patterns mirroring societal biases ( Bender et al. 2021 , Sheng et al. 2021 ) . Current approaches to evaluating ethical reasoning in language models fall into two categories. 1. Single Moral Questionnaire relies on static value alignment, employing binary judgments to evaluate moral principles in isolation. For example, some evaluation protocols may directly inquire whether returning a lost wallet is a morally appropriate action ( Simmons 2023 ) . 2. Single Moral Dilemma introduces contextual dilemmas to better approximate ethical complexity. For instance, scenarios may incorporate economic hardship (e.g., "Should you return a wallet if unable to pay rent?") ( Ji et al. 2023 ) or test implicit value trade-offs through situational variations ( Jin et al. 2022 ) . Although these methods better approximate practical complexity, they remain constrained by their focus on the single decision. Both approaches neglect a foundational characteristic of human moral cognition : its path-dependent nature ( Bandura 1999 ) . Ethical reasoning evolves iteratively with minor contextual shifts, such as new information or escalating stakes, potentially reversing prior judgments ( Volokh 2002 ) . Consider a multi-stage wallet dilemma: 1) You find a wallet but are in desperate need of money. Should you return it? This raises a tension between honesty and personal need. 2) Later, a stranger claims the wallet is theirs, though they’re unfamiliar with it. Does this change your decision? 3) Then, the stranger pulls out a knife and threatens you to return it. Now, your choice involves balancing honesty with self-preservation. Such a dilemma creates a moral entanglement that is absent in static evaluations. To bridge this gap, we construct the Multi-step Moral Dilemmas (MMDs) dataset, featuring 3,302 scenarios that progressively intensify ethical conflicts across five steps. As depicted in Fig. 2 ①, each scenario begins with a simple value conflict (e.g., care vs. honesty in returning a wallet) and systematically introduces new tensions—financial ruin, coercion, survival trade-offs—forcing models to reconcile prior decisions with emerging moral imperatives. This structure operationalizes two key theoretical insights. The first is Dynamic Value Loading ( Bandura 1999 , Binns et al. 2018 , Railton 2017 , Friedman et al. 2013 ) , which holds that values must be reweighted as contexts evolve, testing if LLMs rigidly follow initial principles or adaptively recalibrate. The second is Nonlinear Preference Shifts ( Railton 2017 , Amodei et al. 2016 , Zhou et al. 2023 ) , which suggests that models may exhibit abrupt reversals in value priorities once critical thresholds (e.g., self-preservation) are crossed, exposing latent misalignments. To systematically classify the moral dimensions behind human actions, we consider two frameworks: Moral Foundations Theory (MFT) ( Graham et al. 2013 ) and Schwartz’s Theory of Basic Values ( Schwartz 2012 ) . We follow a multi-stage analytical process where LLMs assess each action through structured reasoning, explicitly evaluating its impact on stakeholders. To ensure reliability, we employ a three-tier validation system: initial parallel assessments, consensus-building based on majority agreement, and expert review for unresolved cases. By combining established psychological theories with systematic reasoning and rigorous validation, our framework provides a robust approach to mapping human action to its underlying ethical foundations. Our main contributions are: • New Dataset: We propose Multi-step Moral Dilemmas (MMDs), a novel benchmark designed to simulate complex, evolving moral decisions that unfold over multiple reasoning steps. • New Framework: We introduce a path-dependent evaluation framework that captures the temporal dynamics of moral judgment, addressing the limitations of static, single-step assessment methods. • New Findings: LLMs exhibit non-transitive and shifting moral preferences, suggesting a reliance on local heuristics rather than stable, globally consistent principles. 2 Related Work Human Value Theory. Our work builds on descriptive moral theories ( Kagan 2018 ) that model human moral preferences based on observed behaviors. In particular, we draw on two widely used frameworks: Moral Foundations Theory (MFT) ( Graham et al. 2013 ) and Schwartz’s Theory of Basic Values ( Schwartz 2012 ) . MFT identifies six core moral domains: care , fairness , loyalty , authority , sanctity , and later, liberty ( Haidt and Graham 2007 , Haidt 2013 ) —while Schwartz’s theory proposes ten broad values such as self-direction , stimulation , hedonism , achievement , power , security , conformity , tradition , benevolence , and universalism , offering a rich basis for analyzing value diversity across individuals and cultures. These theories have also been widely adopted to evaluate and interpret value alignment in LLM ( Yao et al. 2024 , Hadar-Shoval et al. 2024 , Abdulhai et al. 2023 ) . Moral Dilemma. Moral dilemmas have long been studied through philosophical cases such as the trolley problem ( Thomson 1976 ) and the organ transplant scenario ( Daniels 2007 ) , which illustrate the tension between utilitarian outcomes and deontological rules. These paradigms serve as foundations for understanding how conflicting moral principles are evaluated. Dual Process Theory ( Greene et al. 2001 ) further explains such decision-making as a competition between