MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents Conference: Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026); May 25 – 29, 2026; Paphos, Cyprus
Simon Rosen Siddarth Singh Ebenezer Gelo, Helen Sarah Robertson Ibrahim Suder Victoria Williams, Benjamin Rosman Geraud Nangue Tasse Steven James · 2026
Read MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents Conference: Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2026); May 25 – 29, 2026; Paphos, Cyprus by Simon Rosen Siddarth Singh Ebenezer Gelo, Helen Sarah Robertson Ibrahim Suder Victoria Williams, Benjamin Rosman Geraud Nangue Tasse Steven James (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13372.txt.
Description
Moral rules for AI agents often clash and stack in hierarchies, yet benchmarks rarely test agents against ordered, conflicting norms. MoralityGym fills that gap with 98 ethical dilemmas framed as trolley-style Gymnasium environments, built on a new formalism called Morality Chains that represents norms as ordered deontic constraints. By separating task success from moral scoring, and by adding a dedicated Morality Metric, the benchmark lets psychology and philosophy inform how norm-sensitive reasoning is judged.
Baseline tests with safe reinforcement learning methods exposed clear limits, underscoring that principled ethical decision-making needs more than standard safety tweaks. The work aims to ground future systems that behave reliably, transparently, and ethically in messy real settings. Simon Rosen leads the work with eight co-authors.
Catalog record
| CREATOR | Simon Rosen Siddarth Singh Ebenezer Gelo, Helen Sarah Robertson Ibrahim Suder Victoria Williams, Benjamin Rosman Geraud Nangue Tasse Steven James |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13372