Normative Conflicts and Shallow AI Alignment

Raphaël Millière · 2025

Read Normative Conflicts and Shallow AI Alignment by Raphaël Millière (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04679.txt.

Introduction

Raphaël Millière, writing alone, argues that today's alignment of large language models is shallow and breaks under pressure. The 2025 paper grants that fine-tuning on human preferences teaches models to favor helpfulness, honesty, and harmlessness. Attacks succeed, the author contends, by setting those taught norms against each other.

Human moral psychology offers the contrast: people deliberate about clashing norms, which makes them harder to manipulate. Models instead carry fixed behavioral leanings with no real capacity to notice or settle a conflict between them. Newer reasoning-focused models, the author argues, have not closed that gap. The conclusion presses on safety and regulation: preference training alone cannot secure ever stronger systems, so deeper forms of alignment are needed.

Catalog record

CREATORRaphaël Millière
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04679

Browse the full library · About the Tower of Babel