Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges

Shihao Weng, Yang Feng, Xiaofei Xie · 2026

Read Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges by Shihao Weng, Yang Feng, Xiaofei Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06161.txt.

Description

Shihao Weng, Yang Feng, and Xiaofei Xie argue that language model safety judges must pass a reliability test beyond accuracy. Such judges now grade agent safety by default, yet their verdicts may hinge on how the evaluation policy is worded rather than on what the agent did. The authors name the needed property policy invariance and split it into three testable principles.

Their stress test applies certified equivalent rewrites and deliberate strict to lenient shifts to the judging rubric. Today's judges reacted about as strongly to meaningless rewrites as to real normative shifts, and could not tell them apart. Meaning preserving rewrites flipped up to 9.1 percent of verdicts. A new Policy Invariance Score and Judge Card protocol expose wide reliability gaps that accuracy leaderboards hide.

Catalog record

CREATORShihao Weng, Yang Feng, Xiaofei Xie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06161

Browse the full library · About the Tower of Babel