Dynamic Risk Assessments for Offensive Cybersecurity Agents
Boyi Wei * 1, Benedikt Stroebl * 1, Jiacen Xu § 2, 3], Joie Zhang 1, Zhou Li 3, Peter Henderson 1 · 2025
Read Dynamic Risk Assessments for Offensive Cybersecurity Agents by Boyi Wei * 1, Benedikt Stroebl * 1, Jiacen Xu § 2, 3], Joie Zhang 1, Zhou Li 3, Peter Henderson 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18384.txt.
Introduction
Guards that audit a software agent for hacking skill tend to test it in a fixed, tidy range. Boyi Wei and five co-authors argue in 2025 that real foes have far more room to move. With a reward signal to chase and money on the line, a foe can hone an agent step by step within a set compute budget.
On a small budget of eight top-chip hours, the team shows a foe can lift an agent's score on a capture-the-flag test by more than forty percent, with no help from outside. The lift holds in worlds that keep state and in worlds that reset. Their call is for audits that grade agents under the foe's true room to move, not in a cage. Risk reads that miss that room will miss the risk.
Catalog record
| CREATOR | Boyi Wei * 1, Benedikt Stroebl * 1, Jiacen Xu § 2, 3], Joie Zhang 1, Zhou Li 3, Peter Henderson 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18384
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Risk Assessments for Offensive Cybersecurity Agents Boyi Wei * 1, Benedikt Stroebl * 1, Jiacen Xu § 2, 3], Joie Zhang 1, Zhou Li 3, Peter Henderson 1 Abstract Foundation models are increasingly becoming better autonomous programmers, raising the prospect that they could also automate dangerous offensive cyber - operations. Current frontier model audits probe the cybersecurity risks of such agents, but most fail to account for the degrees of freedom available to adversaries in the real world. In particular, with strong verifiers and financial incentives, agents for offensive cybersecurity are amenable to iterative improvement by would-be adversaries. We argue that assessments should take into account an expanded threat model in the context of cybersecurity, emphasizing the varying degrees of freedom that an adversary may possess in stateful and non-stateful environments within a fixed compute budget. We show that even with a relatively small compute budget (8 H100 GPU Hours in our study), adversaries can improve an agent’s cybersecurity capability on InterCode CTF by more than 40% relative to the baseline—without any external assistance. These results highlight the need to evaluate agents’ cybersecurity risk in a dynamic manner, painting a more representative picture of risk. * * footnotetext: Equal Contribution. Code available at https://github.com/princeton-polaris-lab/Dynamic-Risk-Assessment § § footnotetext: This work was done when the author was a PhD student at UC Irvine. 1 Introduction LLMs and autonomous AI agents continue to improve in their performance on cybersecurity tasks ( Pimpale et al., 2025 ; Stroebl et al., 2025 ) at a time when the frequency and sophistication of cyberattacks have escalated. For instance, according to the CrowdStrike 2025 Global Threat Report, in 2024, the average eCrime breakout time dropped to 48 minutes, which is 22% faster than in 2023, with the fastest breakout observed at just 51 seconds. 1 1 1 https://go.crowdstrike.com/2025-global-threat-report.html The convergence of these developments has raised concerns about the potential misuse of AI agents in cyberattacks. Autonomous agents could be deployed at scale to identify and exploit vulnerabilities in software systems, thereby amplifying cybersecurity risks. To evaluate the capability of current offensive cybersecurity agents, various benchmark tasks have been proposed, such as Capture the Flag (CTF) challenges ( Shao et al., 2024b ; Yang et al., 2023 ) and vulnerability detection ( Bhatt et al., 2024 ) . However, most of these studies only focus on static evaluation – they do not consider scenarios in which adversaries leverage compute resources to actively modify agent systems (see Table 1 ). As open-source models continue to demonstrate increasingly strong coding capabilities, the threat of adversarial fine-tuning, previously observed in language models ( Qi et al., 2024b ) , now extends to language agents. In the agent setting, adversaries can modify more than just the model, they can modify the agent scaffolding,the structure built around the model to guide its behavior—like the exploration approach it uses, the tools it has access to, how it plans, and how it decomposes tasks. With access to verifiers, such agents can also self-improve. Recent studies have demonstrated that the agent’s performance can be improved through: (a) test-time scaling techniques ( Brown et al., 2024 ; Hassid et al., 2024 ; Zhang et al., 2024b ; Snell et al., 2024 ) , and (b) iterative self-training ( Zelikman et al., 2022 ; Hosseini et al., 2024 ; Huang et al., 2023 ) . Considering these points of modification isn’t just important from a security perspective, but has policy implications. In the U.S., tort liability standards may require considering forseeable modifications to model ( Ramakrishnan et al., 2024 ) . In proposed legislation, like California’s (vetoed) SB-1047 explicitly includes models fine-tuned within a specified compute threshold as “covered models derivatives”, making them subject to the same regulatory framework. Yet despite these considerations, how to dynamically assess these evolving risks—especially in the cybersecurity setting—remains underexplored. In this paper, we aim to bridge this gap and systematically study the risk of offensive cybersecurity agents under the threat model in which adversaries are able to spend compute to improve the agent autonomously, without any external assistance. In particular, our contributions are: • First, we conceptually outline why the cybersecurity domain might be especially amenable to self-improving agents due to the availability of a strong reward signal and strong financial incentives, and describe two real-world environments that the attackers may face: stateful and non-stateful environments. • Second, we argue that cyber evaluations should be conducted under an expanded threat model, in which the adversary can improve agents’ capability on offensive cybersecurity even without external knowledge or a stronger model. 2 2 2 Incorporating external knowledge muddies the water as to whether the model actually contributed to the risk. See discussion on marginal risk by Kapoor et al. (2024a) and causation in torts Ramakrishnan et al. (2024) . Through experiments on three CTF benchmarks, we show that agents’ success rate improves through at least five degrees of freedom. • Third, mirroring policy discussions, we analyze under a fixed compute budget , how adversaries can differentially improve agents’ cybersecurity capabilities across these dimensions. We find, for example, that performance on Intercode CTF can increase by more than 40% relative to the baseline with a small fixed budget of 8 GPU hours. (a) Adversaries’ Degrees of Freedom in Cybersecurity Agents (b) Dynamic Risk Assessment Figure 1 : (a) We introduce a new threat model in which adversaries will have at least five degrees of freedom to modify offensive cybersecurity agents for improved performance. (b) Under this threat model, we assess the risk of offensive cybersecurity agents by dynamically analyzing how far adversaries can push along each axis on InterCode CTF (Test), within a fixed 8 H100 GPU Hours compute budget. Table 1 : Unlike past work, we dynamically analyze five degrees of freedom that adversaries can exploit to autonomously improve agent’s offensive cybersecurity capabilities. In o3 / o4-mini system card, the pass@ 12 is computed from 16 rollouts. Previous Studies Repeated Max Rounds of Iter Prompt Self Iter Workflow Sampling ( k ) Interactions ( N ) Refinement Training Refinement InterCode ( Yang et al., 2023 ) ✗ ✓ ( N\in[1,10] ) ✗ ✗ ✗ NYU CTF ( Shao et al., 2024b ) ✓ ( k=5 ) ✗ ✗ ✗ ✗ Cybench ( Zhang et al., 2024a ) ✓ ( k=3 ) ✗ ✗ ✗ ✗ EnIGMA ( Abramovich et al., 2025 ) ✗ ✗ ✗ ✗ ✗ o3 / o4-mini System Card ( OpenAI, 2025 ) ✓ ( k=12 ) ✗ ✗ ✗ ✗ Claude 3.7 Sonnet System Card ( Anthropic, 2025 ) ✓ ( k=30 ) ✗ ✗ ✗ ✗ o1 Pre-Deployment Report ( UK AISI & US AISI, 2024 ) ✓ ( k=10 ) ✓ ( N\in[1,100] ) ✗ ✗ ✗ Ours ✓ ✓ ✓ ✓ ✓ 2 Cybersecurity is Amenable to Self-Improvement Cybersecurity is uniquely suited for iteratively improving agent performance because it often possesses two key preconditions: the preconditions for scaling compute and non-statefulness. The former allows the adversary to use more resources to discover a vulnerability, and the second enables more effective search strategies. 2.1 Preconditions for compute scaling Strong verifier. In many cybersecurity domains, the availability of a strong verifier signal is both common and central