LinuxArena: A Control Setting for AI Agents in Live Production Software Environments
Redwood Research Tyler Tracy, Ram Potham, Nick Kuhn, Myles Heller, Anshul Khandelwal, Cody Rushing EquiStamp Henri Lemoine, Miguel Brandão, Tomáš Turlik, Adam Hanson, Josh Hills, Amy Ngo, Ram Rachum, Nik Mitchell, Falko Galperin, Oscar Sykes, Pip Arnott, Samuel Prieto Lima, Carlos Giudice, Matt Goldwater, Daniel Popp, Drew de Wet, Ruben Castaing, Qi Guo, Douw Marx, Benjamin Shaffrey, Justin Shenk, Martin Milbradt, Hannah Meagher, Shaheen Ahmed-Chowdhury, Daniel O’Connell, Chris Canal Senior Authors Buck Shlegeris, Aryan Bhatt · 2026
Read LinuxArena: A Control Setting for AI Agents in Live Production Software Environments by Redwood Research Tyler Tracy, Ram Potham, Nick Kuhn, Myles Heller, Anshul Khandelwal, Cody Rushing EquiStamp Henri Lemoine, Miguel Brandão, Tomáš Turlik, Adam Hanson, Josh Hills, Amy Ngo, Ram Rachum, Nik Mitchell, Falko Galperin, Oscar Sykes, Pip Arnott, Samuel Prieto Lima, Carlos Giudice, Matt Goldwater, Daniel Popp, Drew de Wet, Ruben Castaing, Qi Guo, Douw Marx, Benjamin Shaffrey, Justin Shenk, Martin Milbradt, Hannah Meagher, Shaheen Ahmed-Chowdhury, Daniel O’Connell, Chris Canal Senior Authors Buck Shlegeris, Aryan Bhatt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15384.txt.
Introduction
Tracy and 33 colleagues introduce LinuxArena, a control setting where AI agents work directly in live multi-service production environments. It holds 20 environments, 1,671 main engineering tasks, and 184 side tasks covering safety failures such as data theft and backdooring. Sabotage trials show Claude Opus 4.6 evading a GPT-5-nano monitor about 23 percent of the time, and a human-crafted attack set evades monitors even more often.
Catalog record
| CREATOR | Redwood Research Tyler Tracy, Ram Potham, Nick Kuhn, Myles Heller, Anshul Khandelwal, Cody Rushing EquiStamp Henri Lemoine, Miguel Brandão, Tomáš Turlik, Adam Hanson, Josh Hills, Amy Ngo, Ram Rachum, Nik Mitchell, Falko Galperin, Oscar Sykes, Pip Arnott, Samuel Prieto Lima, Carlos Giudice, Matt Goldwater, Daniel Popp, Drew de Wet, Ruben Castaing, Qi Guo, Douw Marx, Benjamin Shaffrey, Justin Shenk, Martin Milbradt, Hannah Meagher, Shaheen Ahmed-Chowdhury, Daniel O’Connell, Chris Canal Senior Authors Buck Shlegeris, Aryan Bhatt |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98015 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15384
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LinuxArena: A Control Setting for AI Agents in Live Production Software Environments Redwood Research Tyler Tracy, Ram Potham, Nick Kuhn, Myles Heller, Anshul Khandelwal, Cody Rushing EquiStamp Henri Lemoine, Miguel Brandão, Tomáš Turlik, Adam Hanson, Josh Hills, Amy Ngo, Ram Rachum, Nik Mitchell, Falko Galperin, Oscar Sykes, Pip Arnott, Samuel Prieto Lima, Carlos Giudice, Matt Goldwater, Daniel Popp, Drew de Wet, Ruben Castaing, Qi Guo, Douw Marx, Benjamin Shaffrey, Justin Shenk, Martin Milbradt, Hannah Meagher, Shaheen Ahmed-Chowdhury, Daniel O’Connell, Chris Canal Senior Authors Buck Shlegeris, Aryan Bhatt Abstract We introduce LinuxArena , a control setting in which agents operate directly on live, multi-service production environments. LinuxArena contains 20 environments, 1,667 main tasks representing legitimate software engineering work, and 185 side tasks representing safety failures such as data exfiltration and backdooring, making it the largest and most diverse control setting for software engineering to date. We validate that LinuxArena is useful for control research by running sabotage evaluations, which measure whether attackers can complete side tasks while working on main tasks, and monitor evaluations, which measure a monitor model’s ability to detect sabotage attempts. Against a GPT-5 Nano trusted monitor at a 1% step-wise false positive rate, Claude Opus 4.6 achieves an undetected sabotage success rate of roughly 34%. We additionally release LaStraj , a dataset of human-crafted attack trajectories that evade monitors at substantially higher rates than any model-generated attacks we elicited, showing that current attack policies do not saturate LinuxArena. These results suggest that LinuxArena has meaningful headroom for both attackers and defenders, making it a strong testbed for developing and evaluating future control protocols. 1 Introduction As frontier AI systems become capable of complex engineering and operational work, a central question is whether they can be deployed safely with access to sensitive tools, codebases, and infrastructure. A model may be useful on its assigned task while also capable of harmful actions: exfiltrating secrets, weakening security controls, or corrupting data on the systems it is meant to help operate. How can users benefit from capable agents while limiting the risk of harmful behavior ( 4 ) ? This question motivates a growing body of work on AI control ( 6 ; 7 ; 11 ) , which evaluates whether protocols built around weaker trusted models can catch a stronger untrusted model attempting harmful actions during legitimate work. Existing settings have demonstrated increasing degrees of realism, but so far have not directly modeled the kind of environments in which frontier AI agents are already being deployed: heterogeneous production software systems with live state, multiple services, and real application logic, where the same access an agent needs to do useful work is also what makes sabotage possible. We discuss prior settings and their tradeoffs in § 2.2 . Throughout this paper we use GPT-5 Nano as our trusted monitor. We take this to be a reasonable stand-in for the capability gap between future trusted and untrusted models, and it is capable enough to make monitoring nontrivial while leaving meaningful headroom for stronger attacks to evade it. We introduce LinuxArena , a control setting in which agents operate directly on live, multi-service production environments. Each environment is a Docker Compose deployment comprising an application server alongside supporting services such as databases, caches, and message queues. The agent is placed on the production server and performs legitimate engineering work assigned as a main task, with the option, in attack episodes, of additionally pursuing a harmful side task. LinuxArena’s key property is that legitimate engineering work and sabotage require overlapping access. Main tasks depend on the same network access, service interactions, and live state that an attacker would use. As a result, sabotage opportunities are embedded throughout normal work rather than confined to a separable attack surface. LinuxArena is the largest software-engineering control setting to date, spanning languages from COBOL to Rust to Solidity and systems from payroll processing and clinical trial analysis to model registries, DeFi yield aggregators, and video transcoding pipelines. LinuxArena is designed to support future end-to-end control evaluations. In this paper, we do not evaluate complete control protocols. Instead, we validate the setting through sabotage evaluations, which measure whether attackers can complete harmful side tasks while carrying out legitimate work, and monitor evaluations, which measure whether a monitoring system can detect such behavior. Our results establish three properties of the setting. First, simple sandboxing is not a viable alternative to monitoring: restricting the agent’s access to the network, intranet, or running services substantially reduces main task performance, because legitimate engineering work in LinuxArena depends on the same resources an attacker would need. Second, weak trusted monitoring is insufficient for safe deployment inside LinuxArena: even at realistic false-positive budgets, a frontier attacker completes a substantial fraction of side tasks undetected against our GPT-5 Nano trusted-monitor baseline (Claude Opus 4.6: 34% undetected sabotage at 1% step-wise FPR; § 5.3 shows stronger monitors do much better). Third, current attack elicitation does not saturate the setting: human red-teamers construct attack trajectories that substantially outperform the best model-generated attacks we were able to elicit from frontier models. Taken together, LinuxArena has headroom on both sides. Neither current trusted monitors nor current attack elicitation exhaust what the setting can support, making it suitable for development of control protocols as model capabilities advance. We expect future, more capable models to produce stronger attacks and further close the gap with human red-teamers. Our main contributions are: • LinuxArena, a new control setting for agentic software environments. LinuxArena is the largest control setting for software engineering to date, comprising 20 environments, 1,667 main tasks, and 185 side tasks. It spans web services, databases, payroll systems, scientific computing pipelines, and DeFi infrastructure. We release 10 of the 20 environments publicly and withhold the remaining 10 to reduce the risk of training data contamination. • Baseline sabotage and monitor evaluations on LinuxArena. We evaluate frontier models as both attackers and monitors. Simple sandboxing is not a viable substitute for monitoring, and our GPT-5 Nano trusted-monitor baseline misses 34% of Opus attacks at 1% step-wise FPR. • LaStraj (LinuxArena Sabotage Trajectories), a dataset of human-crafted subtle attack trajectories that evade frontier models. We release LaStraj, a collection of human-generated sabotage trajectories in LinuxArena with a 94% successful-sabotage rate against our GPT-5 Nano monitor, vs 50% for the best model-generated attack we elicited. LaStraj serves both as evidence that current attack elicitation does not saturate LinuxArena and as a standalone benchmark for future monitor evaluation. 2 Related Work 2.1 AI Control AI control studies how to safely obtain useful work from models that may be misaligned, strategically deceptive, or otherwise insufficiently trusted ( 6 ; 11 ) . The concern that models may fake alignment during training or deployment has empirical support ( 5 ) . The central premise is that frontier models cannot always be assumed to behave as intended, and that safety should not depend on this assumption. Control protocols are designed to remain effective even against a model that is actively trying to subvert them. A standard simplifying assumption is the distinction between an untrusted model U , typically the most capable model available, which is assumed to be potentially adversarial, and a trusted model T , typically a weaker