ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents

Udari Madhushani Sehwag ∗, Zhengyang Shan ∗, Heming Liu, Dileepa Lakshan, Joseph Brandifino, Max Fenkell · 2026

Read ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents by Udari Madhushani Sehwag ∗, Zhengyang Shan ∗, Heming Liu, Dileepa Lakshan, Joseph Brandifino, Max Fenkell (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-17324.txt.

Introduction

Udari Madhushani Sehwag and colleagues show that asking an agent to clarify ambiguity sharply raises its vulnerability to prompt injection. Their ASPI benchmark of 728 task-attack cases compares matched execution and clarification settings across ten frontier models. Attack success jumps from under 2 percent to about 34 percent for o3 and to about 36 percent for Gemini-3-Flash. The gap reflects both a state shift in processing and the clarification channel itself.

Catalog record

CREATORUdari Madhushani Sehwag ∗, Zhengyang Shan ∗, Heming Liu, Dileepa Lakshan, Joseph Brandifino, Max Fenkell
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE118704
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.17324

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ASPI: Seeking Ambiguity Clarification Amplifies Prompt Injection Vulnerability in LLM Agents Udari Madhushani Sehwag ∗, Zhengyang Shan ∗, Heming Liu, Dileepa Lakshan, Joseph Brandifino, Max Fenkell Abstract Clarification-seeking behavior is widely regarded as a desirable property of LLM agents, enabling them to resolve ambiguity before acting on underspecified tasks. However, the security implications of this interaction pattern remain unexplored. We investigate whether the transition from standard execution to a clarification-seeking state increases an agent’s susceptibility to prompt injection attacks. We introduce ASPI ( A mbiguous- S tate P rompt I njection), a benchmark of 728 task–attack scenarios that isolates clarification as a distinct agent state and measures how this state transition affects vulnerability under controlled conditions. Each benchmark instance is evaluated under matched execution and clarification settings: in the execution setting, the agent acts on a fully specified instruction and encounters adversarial content only through tool-returned data; in the clarification setting, the agent must first request and incorporate additional user input before acting. We evaluate ten frontier LLMs and find that clarification-seeking consistently and substantially amplifies vulnerability. For instance, attack success rises from 1.8% to 34.0% for o3 and from 2.2% to 35.7% for Gemini-3-Flash. A decomposition analysis reveals that this gap reflects both a state-dependent shift in how models process incoming content and a channel-specific effect arising from the agent-solicited clarification interface. These findings demonstrate that standard execution-time security evaluation systematically underestimates the attack surface of interactive agents, and that robustness under fully specified tasks does not translate to robustness under ambiguity. For reproducibility, our data and source code are available at https://github.com/scaleapi/aspi . \DeclareCaptionType prompt[Prompt][List of Prompts] ∗ Equal Contributions [email protected] scale.com/research/aspi 1 Introduction The ability to recognize when a task is underspecified and to seek clarification from the user before proceeding is central to reducing errors caused by ambiguous instructions and aligning agent behavior more closely with user intent ( 1 ; 16 ; 34 ) . Recent work has devoted considerable attention to improving when and how agents ask clarification questions, treating the ability to do so as a marker of competent, user-aligned behavior ( 13 ; 20 ; 19 ; 22 ; 7 ; 23 ; 5 ; 12 ) . However, the security implications of this interaction pattern remain unexplored. When an agent solicits additional input from the user, it introduces a new channel through which external content enters the agent’s context, one that is distinct from the tool-output channels examined in prior prompt-injection research ( 8 ; 33 ; 4 ; 32 ; 15 ; 6 ; 10 ; 35 ; 36 ) . This raises the natural question: Does seeking clarification — a behavior widely regarded as desirable — increase LLM agents’ vulnerability to prompt injection attacks? We study this question by isolating clarification as a distinct agent state and measuring how the transition from standard execution to ambiguity resolution affects susceptibility to adversarial instructions. In standard execution, prompt injection attacks reach the agent through tool-returned content. In the clarification setting, a qualitatively different attack surface emerges: an attacker can embed malicious instructions within the response to the agent’s clarification question. Since the agent has explicitly requested this input and expects it to contain task-relevant information, the distinction between trusted instructions and untrusted external content becomes difficult to maintain, a failure mode closely related to prior observations that LLM-integrated systems struggle to separate instructions from data when processing retrieved or third-party content ( 8 ; 4 ; 11 ; 15 ; 24 ; 32 ; 3 ) . We investigate this by introducing ASPI ( A mbiguous- S tate P rompt I njection), a benchmark of 728 task–attack scenarios spanning four application domains—workspace, messaging, travel, and banking—that evaluates whether LLM agents exhibit increased vulnerability when operating under ambiguity. ASPI builds on AgentDojo ( 4 ) and constructs controlled comparisons in which the task, attacker goal, environment, and scoring functions are held fixed while varying the agent’s interaction state and the channel through which adversarial content is delivered. This pairs an execution-state evaluation, following the standard protocol, with a matched clarification-state evaluation in which the agent must first resolve ambiguity before completing the main task. Figure 1 : Overview of ASPI, illustrated with a benchmark example. Top : in the standard execution state, the user provides a fully specified task, and the agent encounters adversarial content only through ordinary tool-returned data. Bottom (our contribution): in the clarification state, the initial user request is underspecified, so the agent first asks a clarification question before acting. The attacker can then inject adversarial instructions through clarification-related channels, including the user’s clarification reply, a task-tool output, or the ask_user tool return. We evaluate ten models spanning multiple proprietary and open source model families. Key contributions of this work are as follows: • We present the first study which shows that agents become substantially more vulnerable to prompt injection when operating under ambiguity, with attack success rates increasing across both open-source and proprietary models (e.g., 1.8% to 34.0% for o3, 2.2% to 35.7% for Gemini-3-Flash, and 11.1% to 63.1% for Kimi K2.5). • We introduce ASPI, a benchmark of 728 paired task–attacker instances across four domains with a controlled evaluation protocol that isolates the causal effect of interaction state on vulnerability. The paired design, systematic ambiguity construction pipeline, and matched delivery channels provide a methodological framework for studying security properties of multi-turn agent interactions in clarification-seeking setting. • We provide a decomposition of the vulnerability gap into state and channel effects, showing that the clarification channel ( ask_user ) consistently amplifies attack success across models, while the effect of interaction state alone is heterogeneous. This analysis disentangles two confounded factors and identifies the solicited-response channel as the primary driver of increased vulnerability. • We adapt and evaluate two lightweight defenses for clarification time attacks rather than applying them naively: a segment-level prompt guard that scans both user and tool messages while preserving benign clarification content, and an ask_user -aware tool filter that fires at the last safe boundary before agent action and preserves clarification ability. These ASPI-adapted defenses reduce attack success but do not eliminate the clarification-state vulnerability. Residual attack success persists even under defense (e.g., 27.0% for Gemini-3-Flash under prompt guard versus 35.7% undefended), indicating that clarification creates an attack surface not fully addressed by input filtering or tool restriction. 2 Related Work Prompt injection in LLM-integrated systems. Prompt injection was first articulated as goal hijacking and prompt leaking in standalone LLM settings ( 17 ) . This threat was later extended to indirect prompt injection, where malicious instructions embedded in retrieved or third-party content hijack downstream applications ( 8 ) . Subsequent work formalized threat models and benchmarked attacks and defenses systematically ( 15 ; 32 ) . On the defense side, approaches include boundary-aware prompting and spotlighting ( 11 ; 32 ) , structured separation of instructions from data ( 3 ) , training models to prioritize privileged instructions ( 24 ) , and execution-isolation architectures for agentic systems ( 28 ) . These works establish the broader security context, but do not isolate how prompt-injection risk changes when an agent enters a clarification-seeking state. Prompt injection benchmarks for tool-using agents. The closest security-side precedents

Read the complete text