Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges

Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan · 2025

Read Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges by Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04474.txt.

Description

Francisco Eiras, Eliott Zemour, Eric Lin, and Vaikkunth Mugunthan test the AI judges that grade other AI models for safety, in a 2025 paper. Such judges sit under key safety work: offline tests, automated red-teaming, and live guardrails. If the judge is shaky, the whole safety score rests on sand.

The team probe common safety judges in two ways. Small shifts, like a change in output style, pushed false negative rates up by as much as 0.24 on the same data. Attacks aimed at the judge fooled some judges into calling every harmful reply safe. Standard checks for judges miss both flaws, so a low attack score under a weak judge can give a false sense of safety. The paper calls for harder tests of the judges themselves.

Catalog record

CREATORFrancisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-04474

Browse the full library · About the Tower of Babel