Beyond Single-Agent Safety: A Taxonomy of Risks in LLM-to-LLM Interactions
P. Bisconti 1,2 M. Galisai 1,2 F. Pierucci 1,3 M. Bracale 1 M. Prandi 1,2 · 2025
Read Beyond Single-Agent Safety: A Taxonomy of Risks in LLM-to-LLM Interactions by P. Bisconti 1,2 M. Galisai 1,2 F. Pierucci 1,3 M. Bracale 1 M. Prandi 1,2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-02682.txt.
Introduction
P. Bisconti and four co-authors warn that AI safety tools built for one model fail when models talk to each other. Current practice guards single agents with prompts, fine tuning, and moderation layers. Those layers constrain one model's behavior but leave group dynamics ungoverned.
The 2025 study builds a taxonomy of risks that arise in model to model interaction. New failure modes can emerge from feedback loops, imitation, and shared blind spots between agents. A shared risk map gives regulators and builders a starting point for governing multi agent systems before they scale further. Two safe bots can still make an unsafe pair, and that is the gap this map tries to name first.
Catalog record
| CREATOR | P. Bisconti 1,2 M. Galisai 1,2 F. Pierucci 1,3 M. Bracale 1 M. Prandi 1,2 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-02682
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Single-Agent Safety: A Taxonomy of Risks in LLM-to-LLM Interactions P. Bisconti 1,2 M. Galisai 1,2 F. Pierucci 1,3 M. Bracale 1 M. Prandi 1,2 Abstract This paper examines why safety mechanisms designed for human–model interaction do not scale to environments where large language models (LLMs) interact with each other. Most current governance practices still rely on single-agent safety containment, prompts, fine-tuning, and moderation layers that constrain individual model behavior but leave the dynamics of multi-model interaction ungoverned. These mechanisms assume a dyadic setting: one model responding to one user under stable oversight. Yet research and industrial development are rapidly shifting toward LLM-to-LLM ecosystems, where outputs are recursively reused as inputs across chains of agents. In such systems, local compliance can aggregate into collective failure even when every model is individually aligned. We propose a conceptual transition from model-level safety to system-level safety, introducing the framework of the Emergent Systemic Risk Horizon (ESRH) to formalize how instability arises from interaction structure rather than from isolated misbehavior. The paper contributes (i) a theoretical account of collective risk in interacting LLMs, (ii) a taxonomy connecting micro, meso, and macro-level failure modes, and (iii) a design proposal for Institutional AI, an architecture for embedding adaptive oversight within multi-agent systems. 1 Beyond Single-Agent Safety Current safety approaches assume a single model operating under direct human supervision, where undesired behavior can be prevented through prompt engineering, reinforcement learning from human feedback, or output moderation. These techniques have produced measurable improvements in reliability and civility, yet they remain pointwise : they govern how one instance responds to one input, not how multiple models behave together. As large models are increasingly deployed in tool-using, agentic, and multi-agent settings, this single-agent safety paradigm begins to break down. Models that are individually well-aligned can collectively generate outcomes that no single instance was trained to avoid, through feedback amplification, imitation, or emergent coordination [ 4 ] . The result is a gap between local alignment and systemic safety, where compliance at the component level fails to guarantee control at the network level. This paper addresses that gap. We argue that safety must be reconceived as a property of interaction architectures rather than of isolated models, and we introduce the Emergent Systemic Risk Horizon (ESRH) as a first framework for reasoning about when and how collective instability arises. The following sections trace this transition: from dyadic interaction to multi-agent autonomy, from empirical observation to a taxonomy of systemic risks, and finally to a proposal for Institutional AI, a design approach aimed at managing these risks within the systems themselves. 1.1 From Dyadic Interaction to Collective Dynamics The safety architecture of today’s language models is built around the human–AI dyad: a supervised conversation mediated by prompts and output filters. Within this paradigm, safety is equated with interface compliance , the ability to block harmful content or follow instructions within predefined boundaries. Such controls are valuable but assume that behavior is both observable and stationary. Language models, however, are probabilistic systems whose trajectories depend on latent state, conversational history, and orchestration context. Minor contextual perturbations can alter reasoning paths without violating surface constraints. 1.1.1 Why Surface Controls Fail in LLM-to-LLM Environments The limits of this approach become evident once models begin to interact with other models. In multi-agent configurations, an LLM’s output becomes the next model’s input, creating feedback loops that can magnify subtle deviations. Benchmarks such as JAILJUDGE demonstrate that multi-agent evaluation alters jailbreak success rates: reasoning chains between judge, critic, and generator models produce outcomes that single-model tests cannot predict [ 8 ] . Similarly, experiments in market-simulation environments show that independently aligned agents can spontaneously coordinate to supracompetitive equilibria, revealing strategic behaviors that emerge only through interaction [ 5 , 1 ] . These phenomena indicate that alignment of parts does not entail alignment of the whole. A system composed entirely of compliant components may still generate unsafe global dynamics when reciprocal influence, incentives, and network topology interact. The relevant unit of analysis therefore shifts from the prompt to the protocol, from local semantics to the architecture of communication. Recent surveys map this transition across the LLM communication stack – user \leftrightarrow agent, agent \leftrightarrow agent, agent \leftrightarrow environment – emphasizing that safety must now include protocol design and information-flow control as first-class engineering concerns. Regulatory and industrial research is converging on instrumented evaluation frameworks that record message-level provenance and diffusion dynamics, exemplified by the UK AI Security Institute’s Inspect and Anthropic’s Petri platforms [ 9 , 2 ] . To govern such systems, safety evaluation must move from static compliance checks to system-level observation. This entails measuring how influence propagates across agents, whether oversight signals penetrate collective behavior, and when information feedback crosses the threshold of controllability. 1.2 Hypothesis and Contribution The central hypothesis of this paper is that surface-level safety mechanisms are necessary but insufficient once models participate in networks of interaction. Even perfect local alignment cannot prevent emergent failure when feedback, topology, and collective incentives dominate behavior. Recently, two studies have addressed emergent risks in multi-agent systems, proposing insightful systematizations: MAST (Multi-Agent System failure Taxonomy [ 3 ] ) and MARAI (Multi-Agent Risks from Advanced AI [ 6 ] ). MAST provides a taxonomy based on three failure cateogories , restricting the analysis to risks stemming directly from the MAS architecture (system design issues, agent coordination and verification). On the other hand, MARAI identifies three failure modes , based on the modality of inter-agent interaction (miscoordination, conflict, collusion) that may produce misaligned behaviours. Based on them, our novel contribution is: 1) organising both approaches into a structured system-centered framework , which builds from agent-level failures to risks that cannot be reduced to the individual system components; 2) formalizing this approach through the concept of the Emergent Systemic Risk Horizon (ESRH), the boundary at which localized reliability gives way to collective instability. The ESRH provides a vocabulary and a set of empirical indicators for studying LLM ecosystems as complex adaptive systems rather than independent agents. This reframing lays the foundation for the rest of the paper. Section 2 traces the empirical and conceptual transition toward multi-agent autonomy and introduces the regulatory implications of distributed causality. Section 3 develops the ESRH taxonomy and associated metrics. Section 4 proposes the Institutional AI architecture as a design principle for embedding governance directly within interacting model collectives. 2 Emergence of Autonomous Collectives Multi-agent autonomy arises through the integration of LLMs with orchestration layers, shared memories, and external tools. Frameworks such as AutoGen [ 11 ] and CAMEL [ 7 ] instantiate role-based dialogues – planner, critic, executor – within a common environment. Newer systems, including SWE-agent [ 12 ] and Voyager [ 10 ] , extend this principle to continuous learning loops where agents generate tasks, test results, and adapt strategies. The outcome is not a static pipeline but an adaptive ecology: each model’s output reshapes the context for the next, producing feedback that accumulates over time. Even when humans remain nominally “in the loop“, their effective authority is delayed, indirect, and partial. Conventional safety evaluation presumes that behavior is observable and interpretable. In large multi-agent configurations, both assumptions fail. Communication is often untyped or unstructured –free-form natural-language exchanges, JSON payloads, or