TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems
Kai Wang*, Biaojie Zeng*, Zeming Wei*, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, Xia Hu Shanghai AI Laboratory https://github.com/AI45Lab/TrinityGuard · 2026
Read TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems by Kai Wang*, Biaojie Zeng*, Zeming Wei*, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, Xia Hu Shanghai AI Laboratory https://github.com/AI45Lab/TrinityGuard (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15408.txt.
Introduction
Wang and co-authors present TrinityGuard, a safety evaluation and monitoring framework for language-model multi-agent systems. The framework rests on OWASP guidance and a three-tier taxonomy of twenty risk types. Tiers cover single agents, agent-to-agent links, and whole-system hazards. An abstraction layer adapts it to varied system shapes. Curated attack probes produce per-risk reports, while monitor agents watch run traces and raise live alerts. Case studies show it works across representative multi-agent setups.
Catalog record
| CREATOR | Kai Wang*, Biaojie Zeng*, Zeming Wei*, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, Xia Hu Shanghai AI Laboratory https://github.com/AI45Lab/TrinityGuard |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42485 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.15408
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems Kai Wang*, Biaojie Zeng*, Zeming Wei*, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, Xia Hu Shanghai AI Laboratory https://github.com/AI45Lab/TrinityGuard Abstract With the rapid development of LLM-based multi-agent systems (MAS), their significant safety and security concerns have emerged, which introduce novel risks going beyond single agents or LLMs. Despite attempts to address these issues, the existing literature lacks a cohesive safeguarding system specialized for MAS risks. In this work, we introduce TrinityGuard , a comprehensive safety evaluation and monitoring framework for LLM-based MAS, grounded in the OWASP standards. Specifically, TrinityGuard encompasses a three-tier fine-grained risk taxonomy that identifies 20 risk types, covering single-agent vulnerabilities, inter-agent communication threats, and system-level emergent hazards. Designed for scalability across various MAS structures and platforms, TrinityGuard is organized in a trinity manner, involving an MAS abstraction layer that can be adapted to any MAS structures, an evaluation layer containing risk-specific test modules, alongside runtime monitor agents coordinated by a unified LLM Judge Factory. During Evaluation, TrinityGuard executes curated attack probes to generate detailed vulnerability reports for each risk type, where monitor agents analyze structured execution traces and issue real-time alerts, enabling both pre-development evaluation and runtime monitoring. We further formalize these safety metrics and present detailed case studies across various representative MAS examples, showcasing the versatility and reliability of TrinityGuard. Overall, TrinityGuard acts as a comprehensive framework for evaluating and monitoring various risks in MAS, paving the way for further research into their safety and security. † † ∗ Equal Contribution 1 Introduction Large Language Model (LLM)-based agents have evolved from single-turn assistants into autonomous entities that plan, invoke tools, and reason over extended horizons Zheng et al. 2025 ; Huang et al. 2025b ; Gao et al. 2025 . Recently, composing several such agents into multi-agent systems (MAS) enables collaborative workflows in software engineering He et al. 2025b , scientific discovery Ghafarollahi and Buehler 2025 , financial analysis Du et al. 2025 with orchestration frameworks, such as AutoGen (AG2) Wu et al. 2024 , making deployment increasingly accessible. As the use of these systems becomes more widespread, concerns about their safety and security have also emerged Deng et al. 2025 ; Ma et al. 2026 ; Wei et al. . Beyond the known risks in individual LLMs or agents, such as producing harmful Zou et al. 2023 ; Wei et al. 2026 or hallucinated Ji et al. 2023 ; Bang et al. 2025 contents, MAS pose significantly more novel risks to be addressed. These risks associated with MAS are inherently hierarchical . We propose a three-tier taxonomy of MAS risks as follows. First, at the individual-agent level, each agent inherits a conventional attack surface like prompt injection Liu et al. 2023 and jailbreaking Zou et al. 2023 as catalogued by the OWASP Top 10 for LLM Applications OWASP Foundation 2023 . Second, at the inter-agent communication level Yan et al. 2025a , multi-agent interaction introduces qualitatively new risks: malicious instructions can propagate through message channels Gu et al. 2024 , factual errors can be amplified by group dynamics Wu et al. 2025 , and agents can spoof one another’s identities to escalate privileges de Witt 2025 . Finally, at the system level, the collective behavior of the MAS may exhibit emergent phenomena like agent collusion Ghaemi 2025 dynamics, where no individual agent would produce in isolation. While these new risks have attracted more attention from the research community, e.g., the OWASP Agentic AI Top 10 standard has begun to codify these threats OWASP GenAI Security Project 2025 , standardized tooling for systematically evaluating and monitoring them across heterogeneous MAS deployments remains largely unaddressed. To address the challenges above, we present TrinityGuard , a comprehensive and scalable framework for safeguarding MAS risks. First, to make TrinityGuard scalable to various platform-agnostic MAS when evaluating these risks, we build a three-layer decoupling safeguarding architecture. First, the MAS framework layer abstracts over heterogeneous orchestration frameworks through a unified BaseMAS abstraction interface (e.g., AG2/AutoGen, LangGraph, CrewAI, and others), fully decoupling safety logic from any specific MAS library. Then, the MAS Intermediary Layer provides framework-agnostic primitives for both test-time intervention and runtime interception. Finally, at the top layer, various risk-specific test modules paired with corresponding runtime monitor agents are deployed for judging these risks. We then hierarchically study and gather 20 popular MAS risk types into three tiers that mirror the layered nature of MAS threats: 1. Risk Tier 1 (RT1): 8 single-agent atomic risks (e.g., prompt injection, jailbreaking, hallucination). 2. Risk Tier 2 (RT2): 6 inter-agent communication risks (e.g., malicious propagation, misinformation amplification, identity spoofing). 3. Risk Tier 3 (RT3): 6 system-level emergent risks (e.g., group hallucination, rogue agent). During evaluation, TrinityGuard generates and executes attack test cases combining curated static payloads with LLM-synthesized adaptive probes against each agent (Tier 1), communication channel (Tier 2), and full execution trajectory (Tier 3), producing structured vulnerability reports with per-entity, per-risk pass rates and severity profiles, and gathering them as a comprehensive evaluation report. We also present evaluations and case studies on a group of MAS to validate TrinityGuard. Our principal contributions are: 1. An open-source, extensible framework TrinityGuard whose three-layer architecture cleanly decouples MAS abstraction, evaluation scaffolding, and risk-specific safety logic, enabling plug-in extension of risk modules and MAS platform support with minimal integration effort. 2. A three-tier risk taxonomy that organizes 20 MAS risk types across single-agent, inter-agent, and system-level dimensions, grounded in the OWASP standards and formalized with quantitative per-entity, per-risk safety metrics. 3. A unified evaluation pipeline that combines curated static payloads with LLM-synthesized adaptive probes targeting each risk type and tier, and produces structured vulnerability reports with pass rates and severity profiles, as demonstrated by comprehensive evaluation and case studies. The remainder of the paper is organized as follows. Section 2 presents the three-layer evaluation methodology. Section 3 formalizes the security metrics and the three-tier risk taxonomy. Section 4 presents experimental evaluation. Section 5 discusses related work, limitations, and future directions. Section 6 concludes. 2 Evaluation Methodology This section presents the high-level design of TrinityGuard’s evaluation methodology. To systematically assess the risks across heterogeneous MAS deployments, we adopt a three-layer architecture that cleanly separates platform-specific concerns from safety evaluation logic. Figure 1 illustrates the overall architecture. Figure 1 : The overall architecture design of TrinityGuard. 2.1 Level 1: MAS Abstraction Layer As stated, our first design principle is that all safety evaluation logic should be agnostic to the specific MAS orchestration framework in use (e.g., AG2 and others). Level 1 achieves this by defining a unified abstraction over heterogeneous MAS platforms through a minimal interface: enumerating agents, routing messages, and executing tasks. Crucially, this abstraction also standardizes a message-hook mechanism that allows upper layers to intercept and inspect messages in transit, which is a prerequisite for both test-time intervention (Level 2) and runtime monitoring (Level 3). Based on this layer, adapting to a new MAS framework requires only implementing this abstraction layer, while all safety evaluation functionality is inherited without modification. 2.2 Level 2: Intermediary Layer Level 2 bridges the platform abstraction and the safety logic by providing two families of framework-agnostic primitives: test-time intervention and runtime observation. These primitives serve as a transitional platform