Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies

Avni Mittal · 2026

Read Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies by Avni Mittal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09189.txt.

Description

Mittal audits whether language models follow their own safety rules, learned from feedback but never formally specified. Existing benchmarks test external standards, not whether models enforce their own stated boundaries. The Symbolic-Neural Consistency Audit extracts self-stated rules via prompts and formalizes them as typed predicates. Findings show absolute-refusal models frequently comply with harmful prompts, while reasoning models lead on consistency yet miss policies for 29 percent of categories. Cross-model agreement on rule types is low at 11 percent, showing a measurable, architecture-dependent say-do gap.

Catalog record

CREATORAvni Mittal
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48407
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.09189

Browse the full library · About the Tower of Babel