Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring

Jehanne Dussert · 2026

Read Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring by Jehanne Dussert (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24737.txt.

Description

Jehanne Dussert argues AI compliance should be a continuous runtime signal, not a binary audit-time verdict. The EU AI Act demands ongoing human oversight and drift detection in deployed systems. Her govllm framework routes model selection by compliance scores rather than latency or cost alone. A panel of regulatory judges, each specialized per criterion, turns disagreement into an uncertainty signal for arbitration. Tests on 49 pairs across five criteria show agreement topping out near 69 percent, with no model leading everywhere, and position bias costing up to 25 points.

Catalog record

CREATORJehanne Dussert
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102996
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24737

Browse the full library · About the Tower of Babel