GuardSet-X : Multi-Domain, Policy-Grounded, AI Security Guardrail Benchmark

Mintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li · 2025

Read GuardSet-X : Multi-Domain, Policy-Grounded, AI Security Guardrail Benchmark by Mintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-19054.txt.

Introduction

Mintong Kang and eight co-authors built a large test set for the guard models that screen language model traffic, in a 2025 paper. The set is named Poly-Guard in the text. Older guard tests rest on ad hoc risk lists that match no real policy, and they blur risks that differ by field.

Poly-Guard spans eight safety-critical fields, such as finance, law, and code. Its risk cases grow out of real, field-specific safety rules rather than a generic list. Cases come as statements, questions, orders, and multi-turn chats. The team also cleaned harmless cases with detox steps to probe over-blocking, and added attack-style cases built to slip past guards. They then scored 19 guard models on the set and found wide swings across risk classes, with thin coverage of field-specific harms.

Catalog record

CREATORMintong Kang, Zhaorun Chen, Chejian Xu, Jiawei Zhang, Chengquan Guo, Minzhou Pan, Ivan Revilla, Yu Sun, Bo Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE415 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-19054

Browse the full library · About the Tower of Babel