Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges
Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan · 2025
Read Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judges by Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04474.txt.
Description
Francisco Eiras, Eliott Zemour, Eric Lin, and Vaikkunth Mugunthan test the AI judges that grade other AI models for safety, in a 2025 paper. Such judges sit under key safety work: offline tests, automated red-teaming, and live guardrails. If the judge is shaky, the whole safety score rests on sand.
The team probe common safety judges in two ways. Small shifts, like a change in output style, pushed false negative rates up by as much as 0.24 on the same data. Attacks aimed at the judge fooled some judges into calling every harmful reply safe. Standard checks for judges miss both flaws, so a low attack score under a weak judge can give a false sense of safety. The paper calls for harder tests of the judges themselves.
Catalog record
| CREATOR | Francisco Eiras, Eliott Zemour, Eric Lin, Vaikkunth Mugunthan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-04474