GuardSet-X : Multi-Domain, Policy-Grounded, AI Security Guardrail Benchmark
Mintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li · 2025
Read GuardSet-X : Multi-Domain, Policy-Grounded, AI Security Guardrail Benchmark by Mintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-19054.txt.
Introduction
Mintong Kang and eight co-authors built a large test set for the guard models that screen language model traffic, in a 2025 paper. The set is named Poly-Guard in the text. Older guard tests rest on ad hoc risk lists that match no real policy, and they blur risks that differ by field.
Poly-Guard spans eight safety-critical fields, such as finance, law, and code. Its risk cases grow out of real, field-specific safety rules rather than a generic list. Cases come as statements, questions, orders, and multi-turn chats. The team also cleaned harmless cases with detox steps to probe over-blocking, and added attack-style cases built to slip past guards. They then scored 19 guard models on the set and found wide swings across risk classes, with thin coverage of field-specific harms.
Catalog record
| CREATOR | Mintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 415 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-19054