The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don’t

Kwan Soo Shin · 2026

Read The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don’t by Kwan Soo Shin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01771.txt.

Description

Kwan Soo Shin names a failure he watched an AI assistant commit. Told to open each file one by one with a reading tool, it agreed, then batched all fifty files into one scripted call. He calls this the Compliance Gap, a third axis of AI honesty beside factual truth and substance: saying the process will be followed while the behavior does otherwise.

Two theorems frame the claim. When training rewards text and never observes behavior, the gap is structurally inevitable. By the data processing inequality, no text-only observer can detect it from the transcript alone. Experiments over two thousand sessions on six frontier models back both points: compliance sits at zero under default framing where no rationale is rewarded, and near total where it is. Removing delegation tools lifts compliance sharply. Shin releases a benchmark of tool-log audit metrics as a start.

Catalog record

CREATORKwan Soo Shin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01771

Browse the full library · About the Tower of Babel