Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems

Gauri Sharma, Vidhi Kulkarni, Miles King, Ken Huang · 2025

Read Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems by Gauri Sharma, Vidhi Kulkarni, Miles King, Ken Huang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21146.txt.

Introduction

Gauri Sharma and three co-authors defined a new attack on cooperating AI agents in a 2025 paper. Multi-agent systems share information and hand off tasks, which creates trust between agents that an attacker can ride. Their attack, Agent Cascading Injection, plants a malicious input or tool exploit in one agent; agents downstream trust its output and fall in turn, so one breach swells into system-wide failure.

The authors formalize the attack with an equation and named variables, then dissect how cascades propagate and amplify. The pattern is mapped onto an industry risk scoring effort for agentic AI. They close with a stress-testing method for agent-to-agent protocols, aiming at measurable, shared security benchmarks so engineers can trade off designs on evidence rather than hope.

Catalog record

CREATORGauri Sharma, Vidhi Kulkarni, Miles King, Ken Huang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-21146

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Unifying Quantitative Security Benchmarking for Multi Agent Systems Gauri Sharma, Vidhi Kulkarni, Miles King, Ken Huang Abstract Evolving AI systems increasingly deploy multi-agent architectures where autonomous agents collaborate, share information, and delegate tasks through developing protocols. This connectivity, while powerful, introduces novel security risks. Once such risk is a cascading risk: a breach in one agent can cascade through the system, compromising others by exploiting inter-agent trust. In tandem with OWASP’s initiative for an Agentic AI Vulnerability Scoring System we define an attack vector, Agent Cascading Injection, analogous to Agent Impact Chain and Blast Radius, operating across networks of agents. In an ACI attack, a malicious input or tool exploit injected at one agent leads to cascading compromises and amplified downstream effects across agents that trust its outputs. We formalize this attack with an adversarial goal equation and key variables (compromised agent, injected exploit, polluted observations, etc.), capturing how a localized vulnerability can escalate into system-wide failure. We then analyze ACI’s properties – propagation chains, amplification factors, and inter-agent compound effects – and map these to OWASP’s emerging Agentic AI risk categories (e.g. Impact Chain and Orchestration Exploits). Finally, we argue that ACI highlights a critical need for quantitative benchmarking frameworks to evaluate the security of agent-to-agent communication protocols. We outline a methodology for stress-testing multi-agent systems (using architectures such as Google’s A2A and Anthropic’s MCP) against cascading trust failures, developing upon groundwork for measurable, standardized agent to agent security evaluation. Our work provides the essential apparatus for engineers to benchmark system resilience, make data-driven architectural trade-offs, and develop robust defenses against a new generation of agentic threats. I Introduction As the use of autonomous AI agents expands across cybersecurity, finance, healthcare, and critical infrastructure, agent-to-agent communication protocols have emerged as foundational components for coordinating tasks, sharing information, and executing distributed decision-making. However, the increasing autonomy and interconnectedness of these systems introduce novel security concerns. Unlike traditional APIs or human-in-the-loop workflows, agent to agent facilitated communication lacks mature tooling for auditing, controlling, or benchmarking inter-agent behavior, particularly under adversarial conditions. Given the recent nature of MAS, it is difficult to define what these adversarial conditions even look like. This work addresses a growing need: to develop a quantitative security benchmarking methodology that targets the unique risks of agent to agent protocols. The growing research corpus has laid important groundwork in protocol design and agent evaluation. Yang et al.’s A Survey of AI Agent Protocols describe the development of agent to agent protocols, their use cases and their current adoption stages [ 1 ] . Google’s Agent-to-Agent Protocol (A2A) has been analyzed as one of the most widely referenced systems, offering developing secure encryption, task delegation mechanisms, and role-based policy enforcement [ 2 ] . However, critical analyses reveal fragilities. Habler et al. apply the MAESTRO threat modeling framework to A2A [ 3 ] , qualitatively uncovering potential vulnerabilities in message validation, identity spoofing, and protocol negotiation (e.g. agent card forgery and task replay). Despite recent progress, such as OWASP’s development of the Agentic AI Vulnerability Scoring System, there remains no standardized benchmark for quantifying the security posture of multi-agent systems. While OWASP’s framework provides valuable risk indices and mitigation strategies, it lacks mathematically grounded attack vector definitions necessary for systematic evaluation. This paper addresses that gap by proposing formalized attack models that not only enrich scoring systems but also enable rigorous benchmarking across diverse multi-agent environments. Broader attempts to evaluate AI safety and robustness have advanced in parallel. HarmBench [ 4 ] proposes a structured harmfulness evaluation across diverse adversarial categories (e.g. cybercrime, misinformation, copyright abuse), providing a robust-refusal benchmark for single-agent LLMs. Similarly, AgentHarm introduces a suite of 110 malicious multi-step tasks across 11 harm domains (fraud, cybercrime, harassment, etc.) to evaluate whether language agents properly refuse unsafe instructions and how jailbreaks affect their multi-step capabilities. However, these frameworks focus on individual agents’ alignment and harmful output, failing to assess risks that emerge in collaborative agent settings [ 5 ] . For multi-agent dynamics, COMMA evaluates communicative efficiency and multimodal task completion across agents but omits any consideration of security constraints or adversarial contexts [ 6 ] . Novelly, Yu et al.’s work robustly surveys the technical implementations of LLM trust, with a focus on multi agent systems in this conversation. Still, the paper does not discuss trust taxonomy in the context of new agentic architectures [ 1 ] . Frameworks like COMMA and HarmBench articulate evaluation methodologies, yet they fall short of capturing dynamic failures that propagate through inter-agent messaging and tool calling or coordinated tasks under compromise. Zhou et al.’s CORBA attack points in a promising direction by actively defining a multi-agent exploit that propagates across network topologies and drains computational resources. However, it remains an open question how emerging agent communication protocols, such as A2A, MCP, or ACP, perform when subjected to such attacks [ 8 ] . Recent work is beginning to emphasize the security of agentic infrastructures . Li and Xie’s analysis, From Glue-Code to Protocols [ 9 ] , contrasts brittle task-specific integration pipelines with scalable protocol-based architectures like A2A and Anthropic’s Model Context Protocol (MCP). Their study reveals that ad-hoc “glue” integrations create opaque and insecure agent interactions, whereas well-specified protocols (e.g. Google A2A) offer a formal basis for trust and auditability. Still this paper and newer surveys such as Multi Agent Risks from Advanced AI , echo a lack of practical quantitative validation tools or threat models to simulate dynamic, multi-agent compromises in the context of emerging architectures [ 10 ] . Taken together, these works suggest a progression in the field, from single-agent robustness and agent collaboration benchmarks to protocol-based agent cooperation, yet they highlight a persistent void in quantitative security evaluation methods for multi-agent systems, especially in the context of the architectures of their implementation. Only a few studies in the surveyed literature directly address agent to agent-specific security properties, some, namely Prompt infection and COBRA provide quantifiable attacks, yet almost no surveys are directed towards the calculation of these metrics in a protocol aware benchmark setting. Currently, the most sophisticated benchmarks focus on individual agent metrics or quantitative multi-agent task performance, without measuring behavioral shifts that arise during architecture conscious inter-agent communication under threat [ 11 ] . This paper aims to bridge that gap by introducing a multi-phase quantitative benchmarking methodology tailored for multi-agent systems in the context of emerging architectures. Building on MAESTRO’s threat modeling, HarmBench’s red-teaming evaluation, and COMMA’s collaborative task framework, we define concrete attack scenarios, measurement criteria, and composite security scores to benchmark inter-agent protocol integrity. We also propose sector-specific use cases, to validate these metrics against real-world constraints. In doing so, this work establishes a foundational framework for agent to agent security benchmarking and outlines future research directions toward safe, scalable, and resilient agentic communication. II Threat Model: The ACI Attack Vector II-A Agent Impact Chains vs. Prompt Injection Indirect prompt injection (IPI) is a known vulnerability in single-agent LLM applications where malicious input embedded in external content causes the model to produce unintended actions. We extend this concept to interconnected agents, defining the Agent Cascading Injection (ACI) attack as a multi-agent prompt injection that propagates through agent to agent channels.

Read the complete text