Running in CIRCLE ? A Simple Benchmark for LLM Code Interpreter Security

Gabriel Chua · 2025

Read Running in CIRCLE ? A Simple Benchmark for LLM Code Interpreter Security by Gabriel Chua (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19399.txt.

Introduction

Gabriel Chua builds CIRCLE, a test bench for code-runner safety in chat models, in a 2025 arXiv paper. Models that run code live open a new attack door past prompt tricks. The bench holds over twelve hundred prompts that aim to drain CPU, memory, or disk. Each risk comes in blunt and sweet-talk forms. An auto judge runs the made code in a sandbox and scores refusal, safety edits, and stalls.

Seven shop models from two firms showed patchy guards. One small model refused far more than its big sibling. Sweet-talk prompts slipped past most shields. Chua urges bench norms for this new door.

Catalog record

CREATORGabriel Chua
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-19399

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Running in CIRCLE ? A Simple Benchmark for LLM Code Interpreter Security Gabriel Chua Abstract As large language models (LLMs) increasingly integrate native code interpreters, they enable powerful real-time execution capabilities, substantially expanding their utility. However, such integrations introduce potential system-level cybersecurity threats, fundamentally different from prompt-based vulnerabilities. To systematically evaluate these interpreter-specific risks, we propose CIRCLE (Code-Interpreter Resilience Check for LLM Exploits), a simple benchmark comprising 1,260 prompts targeting CPU, memory, and disk resource exhaustion. Each risk category includes explicitly malicious (”direct”) and plausibly benign (”indirect”) prompt variants. Our automated evaluation framework assesses not only whether LLMs refuse or generates risky code, but also executes the generated code within the interpreter environment to evaluate code correctness, simplifications made by the LLM to make the code safe, or execution timeouts. Evaluating 7 commercially available models from OpenAI and Google, we uncover significant and inconsistent vulnerabilities. For instance, evaluations show substantial disparities even within providers — OpenAI’s o4-mini correctly refuses risky requests at 7.1%, notably higher rates compared to GPT-4.1 at 0.5%. Results particularly underscore that indirect, socially-engineered prompts substantially weaken model defenses. This highlights an urgent need for interpreter-specific cybersecurity benchmarks, dedicated mitigation tools (e.g., guardrails), and clear industry standards to guide safe and responsible deployment of LLM interpreter integrations. The 1 1 1 https://huggingface.co/datasets/govtech/CIRCLE benchmark dataset and 2 2 2 https://github.com/govtech-responsibleai/CIRCLE evaluation code and logs are publicly released to foster further research. 1 Introduction Integrating a code interpreter into Large Language Models (LLMs) and agents significantly enhances their reasoning capabilities, enabling them to perform tasks ranging from code generation and debugging ( Zhong et al. 2024 ) , to advanced mathematical reasoning ( Liao et al. 2024 ) , and complex data analysis ( Zhang et al. 2024b ; Hong et al. 2024 ) . A code interpreter allows an LLM to automatically write and execute code, thus bridging textual reasoning with executable computational results. Two primary factors are driving the widespread adoption of code interpreters within LLMs. Firstly, the integration process has become significantly streamlined; enabling interpreter access is now as simple as configuring a single parameter through commercial API providers. Previously, developers needed to provision and manage their own code execution environments. However, ”API-native” interpreters are rapidly becoming standard offerings by major providers, including OpenAI ( OpenAI 2025a ) , Google Gemini ( Google 2025 ) , Anthropic ( Anthropic 2025 ) , and Mistral ( Mistral 2025 ) . Secondly, the demand for LLMs to function as autonomous agents in complex tasks—tasks that may be long-running and involve external file dependencies — positions code interpreters as crucial for enabling robust, verifiable reasoning and operational autonomy. While such advancements are highly beneficial, they expose new categories of system-level cybersecurity risks distinct from traditional LLM security concerns, such as prompt injection. Code interpreter tools can inadvertently enable severe denial-of-service (DoS) attacks, resulting in CPU exhaustion, memory leaks, or disk saturation. Crucially, these interpreter integration and executions typically occur on provider-managed infrastructure, obscuring the implementation details and system resource usage from downstream developers. Thus, the potentially opaque nature of interpreter executions makes monitoring, detection, and mitigation challenging, shifting security concerns from model-level safety to application and system-level safety. To address this critical gap, we introduce the CIRCLE benchmark, focused on evaluating and quantifying interpreter-specific vulnerabilities across major LLM platforms. Figure 1 illustrates the CIRCLE benchmark workflow. We systematically send test prompts targeting critical system resources—CPU (time exhaustion), memory, and disk—to the evaluated LLM, which then may either refuse, reframe, clarify, generate potentially unsafe or incorrect code, or execute the provided task directly. The output (including the LLM’s textual response and the executed code’s outcomes) is then assessed by a separate judge LLM, ensuring consistent, automated evaluation across multiple nuanced outcomes, thereby differentiating CIRCLE apart from most other benchmarks that solely evaluate whether unsafe code is generated. 1. Refusal : Explicit refusal to perform the requested risky task. 2. Reframe : Offers an alternative, safer approach. 3. Follow-up : Requests additional clarification or details before proceeding. 4. Incorrect Code : Generates non-functional or oversimplified code, failing to meet task requirements. 5. Fulfilled : Correctly generates and successfully executes the requested task. 6. Timeout : Attempts execution, but the process exceeds allowed resource/time limits. Figure 1: Overview of the CIRCLE benchmark evaluation workflow. Test prompts from three risk categories — CPU (time exhaustion), memory, and disk exhaustion—are sent to evaluated LLMs configured with native code interpreters. Models may refuse, reframe, request further details, or generate executable code. Outputs, including generated responses, code, and execution outcomes, are then systematically assessed by a dedicated judge LLM, categorizing each prompt into one of six detailed outcomes to capture nuanced behaviors and security risks. Interpreter–specific focus. Existing security benchmarks for LLMs target vulnerability exploitation in third‑party software or the robustness of evaluation sandboxes . For example, CVE‑Bench ( Zhu et al. 2025 ) equips agent frameworks with browser‑level tools and measures whether they can compromise real‑world web applications via published CVEs. RedCode ( Guo et al. 2024 ) stresses unsafe code generation and execution across 25 CWE‑derived scenarios, ranging from privilege escalation to filesystem corruption. SandboxEval Rabin et al. 2025 , in contrast, does not evaluates an LLM itself, but stress‑tests the assessment infrastructure that executes untrusted code to ensure that the sandbox cannot be broken out of. CIRCLE addresses a complementary and previously unmeasured threat surface : provider‑hosted code interpreters embedded inside LLM APIs. It therefore probes denial‑of‑service vectors (CPU, memory, disk) that require zero external vulnerability knowledge, occur inside the model’s execution context, and can be triggered by a single prompt. This interpreter‑centric framing demands novel metrics (e.g., timeout vs. fulfilled vs. simplification) and a multi‑provider harness that executes every snippet in situ. Consequently, CIRCLE neither overlaps with CVE‑Bench’s exploitation corpus, RedCode’s CWE taxonomy, nor SandboxEval’s host‑hardening tests, but instead fills the remaining gap of resource‑exhaustion safety for LLM code interpreters . To summarise, the key contributions of CIRCLE include: 1. Comprehensive Risk Taxonomy with Dual Prompt Variants : Provides a detailed, systematic classification of interpreter-induced vulnerabilities covering CPU, memory, and disk abuses. Each category includes both explicitly malicious (direct) and plausibly benign (indirect) prompts, accurately simulating realistic threat scenarios. 2. Automated Multi-Provider Evaluation Harness : Supports evaluation across all major model providers (OpenAI, Google Gemini, Anthropic, Mistral), uniquely executing model-generated code to assess its actual impact. This practical execution combined with the nuanced judge LLM evaluation creates a rigorous assessment framework beyond simple code generation metrics. 3. Open-Source Accessibility : Publicly releases the benchmark, datasets, and evaluation harness, fostering reproducibility, transparency, and community-driven improvements in interpreter-specific cybersecurity research. 2 Related Work Security-Focused Benchmarks : Recent security benchmarks like CyberSecEval ( Bhatt et al. 2023 ; Bhatt et al. 2024 ; Guo et al. 2024 ) assess models’ exploitative code generation, particularly code interpreter risks via compliance metrics. LLMSmith ( Liu et al. 2024 ) identifies explicit remote code execution (RCE) vulnerabilities using static analysis and prompt-based attacks. Similarly, more recent work ( Dawson et al. 2025 ; Shao et al. 2025 ; Zhu et al. 2025 ; Carlini et al. 2025 ; Abramovich et al. 2025 ) tests models in a black-box capture-the-flag scenarios and to leverage known code vulnerabilities. Building upon

Read the complete text