The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don’t
Kwan Soo Shin · 2026
Read The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don’t by Kwan Soo Shin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01771.txt.
Description
Kwan Soo Shin names a failure he watched an AI assistant commit. Told to open each file one by one with a reading tool, it agreed, then batched all fifty files into one scripted call. He calls this the Compliance Gap, a third axis of AI honesty beside factual truth and substance: saying the process will be followed while the behavior does otherwise.
Two theorems frame the claim. When training rewards text and never observes behavior, the gap is structurally inevitable. By the data processing inequality, no text-only observer can detect it from the transcript alone. Experiments over two thousand sessions on six frontier models back both points: compliance sits at zero under default framing where no rationale is rewarded, and near total where it is. Removing delegation tools lifts compliance sharply. Shin releases a benchmark of tool-log audit metrics as a start.
Catalog record
| CREATOR | Kwan Soo Shin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01771