Automating Safety Enhancement for LLM-based Agents with Synthetic Risk Scenarios
Xueyang Zhou, Weidong Wang, Lin Lu, Jiawen Shi, Guiyao Tie, Yongtian Xu, Lixing Chen, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun · 2025
Read Automating Safety Enhancement for LLM-based Agents with Synthetic Risk Scenarios by Xueyang Zhou, Weidong Wang, Lin Lu, Jiawen Shi, Guiyao Tie, Yongtian Xu, Lixing Chen, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17735.txt.
Description
AI agents built on large language models now handle real tasks, yet they can be steered into unsafe acts through tricky prompts and tools. Xueyang Zhou led a team that built AutoSafe, a system that makes agents safer without collecting dangerous real-world examples. It starts from a threat model that spells out how unsafe behavior grows out of user wording, context, and the agent's own steps.
From that model, the system invents risky scenes on its own, thinks through safe replies, and builds a large training set from them. Agents trained this way scored about forty-five percent higher on safety tests and did far better on real-world tasks too. The team released the project in public for others to build on.
Catalog record
| CREATOR | Xueyang Zhou, Weidong Wang, Lin Lu, Jiawen Shi, Guiyao Tie, Yongtian Xu, Lixing Chen, Pan Zhou, Neil Zhenqiang Gong, Lichao Sun |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17735