Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies
Avni Mittal · 2026
Read Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies by Avni Mittal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09189.txt.
Description
Mittal audits whether language models follow their own safety rules, learned from feedback but never formally specified. Existing benchmarks test external standards, not whether models enforce their own stated boundaries. The Symbolic-Neural Consistency Audit extracts self-stated rules via prompts and formalizes them as typed predicates. Findings show absolute-refusal models frequently comply with harmful prompts, while reasoning models lead on consistency yet miss policies for 29 percent of categories. Cross-model agreement on rule types is low at 11 percent, showing a measurable, architecture-dependent say-do gap.
Catalog record
| CREATOR | Avni Mittal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48407 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09189