Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks
Brian Singer 1 , Keane Lucas 2 , Lakshmi Adiga 1 , Meghna Jain 1 , Lujo Bauer 1 , Vyas Sekar 1 · 2025
Read Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks by Brian Singer 1 , Keane Lucas 2 , Lakshmi Adiga 1 , Meghna Jain 1 , Lujo Bauer 1 , Vyas Sekar 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16466.txt.
Introduction
Brian Singer and five co-authors test whether language models can run realistic red-team attacks across whole networks of machines. Real exercises hop through dozens of stepping-stone hosts, and human red teams are costly and scarce. The authors first checked whether existing model-driven tools could manage such campaigns.
Leading systems paired with top models failed at the multi-host task. Studying those failures led to Incalmo, where the model plans in high-level declarative tasks executed by specialist agents, with services tracking context and captured assets. The team also built MHBench, forty emulated networks of up to fifty hosts, for fair evaluation. Incalmo succeeded where the earlier systems could not.
Catalog record
| CREATOR | Brian Singer 1 , Keane Lucas 2 , Lakshmi Adiga 1 , Meghna Jain 1 , Lujo Bauer 1 , Vyas Sekar 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 80 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16466
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks Brian Singer 1 , Keane Lucas 2 , Lakshmi Adiga 1 , Meghna Jain 1 , Lujo Bauer 1 , Vyas Sekar 1 Abstract Security operators use red teams to simulate real attackers and proactively find defense gaps. In realistic enterprise settings, this involves executing multi-host network attacks spanning many “stepping stone” hosts. Unfortunately, red teams are expensive and entail significant expertise and effort. Given the promise of LLMs in CTF challenges, we first analyze if LLMs can autonomously execute multi-host red team exercises. We find that state-of-the-art LLM-assisted offense systems (e.g., PentestGPT, CyberSecEval3) with leading LLMs (e.g., Sonnet 4, Gemini 2.5 Pro) are unable to do so. Building on our observations in understanding the failure modes of state-of-the-art systems, we argue the need to improve the abstractions and interfaces for LLM-assisted red teaming. Based on this insight, we present the design and implementation of Incalmo 1 1 1 All code is publicly available: https://github.com/bsinger98/Incalmo , an LLM-assisted system for autonomously red teaming multi-host networks. Incalmo uses LLMs to plan red team exercises in terms of high-level declarative tasks that are executed by domain-specific task agents. Incalmo also uses auxiliary services to manage context and acquired assets. For our evaluation, we develop MHBench, a novel multi-host attack benchmark with 40 realistic emulated networks (from 22 to 50 hosts). We find that Incalmo successfully acquires critical assets (i.e., key hosts or data) in 37 out of 40 MHBench environments. In contrast, state-of-the-art LLM-assisted systems succeed in only 3 out of 40 environments. We show that Incalmo is efficient—successful attacks took 12–54 minutes and cost ≤ $ 15 \leq\$15 in LLM credits. I Introduction Defenders often use red teams to proactively test and discover gaps in their network defenses. Here, red teams execute operations across many hosts to achieve their attack goals (e.g., compromising a key host), emulating real attackers [ 43 , 46 ] . Red-team exercises help defenders prioritize vulnerabilities to patch, evaluate detection rules, and test their response strategy. Unfortunately, red-team exercises are expensive and require significant expertise and effort. Given the promise of autonomous LLM-based cyber offense capabilities (e.g., [ 14 , 72 , 49 , 80 , 71 , 77 , 61 , 76 , 17 ] ), we explore whether LLMs can autonomously execute red-team exercises. Autonomous red teams could reduce the cost and effort for enterprises and help defenders proactively block attack paths [ 55 ] . Such a capability could also further our understanding of the cybersecurity capabilities of frontier models [ 72 , 71 , 49 ] . Fig. 1: Incalmo is a system for executing multi-host red teams. Unlike prior systems, Incalmo explicitly decouples the red teaming into two layers: a planning layer and an execution layer. Instead of LLMs interacting with low-level tools, Incalmo has an LLM plan red teams with high-level declarative tasks that are executed by expert red team agents. With this objective in mind, we first create MHBench, a red-teaming benchmark of 40 multi-host environments. MHBench is based on a mix of public reports of real-world attacks [ 43 , 33 ] , reference topologies [ 2 , 3 ] , and prior work [ 36 , 69 , 18 , 2 , 40 ] . We use MHBench to evaluate state-of-the-art LLM-based offense systems (e.g., PentestGPT [ 14 ] , CyberSecEval3 [ 72 ] , CAI [ 44 ] ) using frontier models (e.g., GPT4o, Sonnet 4, Gemini 2.5 Pro). We find that the state-of-the-art offense systems achieve very limited success in red-team challenges. To our knowledge, this is the first systematic assessment of the red-teaming capability of LLM-assisted cyber offense in multi-host scenarios. Next, we analyze how state-of-the-art systems failed at multi-host red-team challenges. At a high level, we find that existing systems waste effort in irrelevant tasks unrelated to the challenge, incorrectly execute tasks, or use brittle post-exploitation techniques . Additionally, all of these systems suffer from significant context bloat as they proceed in the complex challenge, which impacts effectiveness in long-horizon challenges, as seen in other domains [ 11 ] . Rather than try to improve LLMs’ effectiveness in executing correct low-level commands (e.g., adding better prompts [ 14 , 72 ] , self-reflection [ 29 , 14 , 62 ] ), we draw inspiration from how expert human red teams work and argue that we need to raise the level of abstraction to build effective autonomous LLM-assisted red teams. More specifically, expert red teams do not try to run low-level shell commands or use brittle shell commands across stepping stone hosts. Instead, they think in terms of high-level “cyber kill-chain” tasks [ 30 ] such as reconnaissance [ 47 ] , exploitation [ 12 ] , command and control [ 55 ] , and goal-centric actions. Furthermore, they use best practice tools in the security domain to ensure each stage in the kill chain can be effective [ 55 , 47 , 12 ] . Building on this insight—that raising the level of abstraction is the key—we present the design, implementation, and evaluation of Incalmo. Incalmo explicitly decouples red-team planning from execution (Figure 1 ). Incalmo uses the LLM primarily as a planning module that decides what task to perform in terms of high-level declarative tasks modeling cyber kill-chain steps, rather than low-level shell commands. Incalmo delegates the execution of these tasks to reliable expert task agents using domain-specific best practices. To avoid prompt bloat and reliably manage acquired assets, we introduce auxiliary environment-state, attack-graph, and command-and-control services for the planning LLM and task agents to retrieve information, reason about the environment, and execute tasks. Taken together, these ideas allow Incalmo to both (a) leverage the broad world knowledge in the LLM used as a planner to respond and plan next steps and (b) use domain-specific expertise to execute the plan. We show that Incalmo can handle unforeseen multi-host scenarios (i.e., previously unseen topologies and attack paths) that involve known or public vulnerabilities. This setting is representative of many real-world red teams that often chain together known techniques to achieve attack goals in multi-host settings [ 43 , 46 , 9 ] . The design of Incalmo is also extensible to include new capabilities (e.g., creating 0-day exploits [ 70 , 73 ] ). In some sense, Incalmo’s design represents a red-team–specific synthesis of best practices in using LLMs for complex and long-horizon tasks—decoupling planning from execution [ 78 ] , scoped agents [ 11 ] , and offloading solution steps [ 21 , 41 , 58 ] . Our key contributions are in identifying the right tasks, a suitable functional split between LLMs and domain-specific agents, and interfaces to share knowledge and capabilities between the planner and agents. To evaluate Incalmo, we leverage MHBench and use three metrics to capture success: (1) Success , indicates whether an attacker has successfully acquired any critical asset in an environment; (2) TotalAcquisition , to measure the proportion of critical assets captured across multiple attempts; and (3) Reliability , to measure the likelihood that any given red-team exercise will succeed at Success. Fig. 2: Comparing Reliability across environments between Incalmo and ExpertPromptShell with Sonnet 4, the