Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models

Naheed Rayhan, Sohely Jahan · 2026

Read Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models by Naheed Rayhan, Sohely Jahan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21860.txt.

Introduction

Rayhan and Jahan introduce Transient Turn Injection, a multi-turn attack that exploits stateless checks. It spreads harmful intent across separate turns, so no single turn looks hostile. Attacker bots test and dodge policy checks repeatedly in commercial and open-source models. Unlike usual jailbreaks, it needs no lasting chat history. Tests across top models find wide gaps in resilience, with only a few designs proving strongly robust. It also finds new flaws, above all in medical and high-stakes areas. Fixes include joining session context and deep alignment.

Catalog record

CREATORNaheed Rayhan, Sohely Jahan
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56079
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.21860

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models Naheed Rayhan, Sohely Jahan Abstract Large language models (LLMs) are increasingly integrated into sensitive workflows, raising the stakes for adversarial robustness and safety. This paper introduces Transient Turn Injection (TTI), a new multi-turn attack technique that systematically exploits stateless moderation by distributing adversarial intent across isolated interactions. TTI leverages automated attacker agents powered by large language models to iteratively test and evade policy enforcement in both commercial and open-source LLMs, marking a departure from conventional jailbreak approaches that typically depend on maintaining persistent conversational context. Our extensive evaluation across state-of-the-art models—including those from OpenAI, Anthropic, Google Gemini, Meta, and prominent open-source alternatives—uncovers significant variations in resilience to TTI attacks, with only select architectures exhibiting substantial inherent robustness. Our automated black-box evaluation framework also uncovers previously unknown model-specific vulnerabilities and attack surface patterns, especially within medical and high-stakes domains. We further compare TTI against established adversarial prompting methods and detail practical mitigation strategies, such as session-level context aggregation and deep alignment approaches. Our study underscores the urgent need for holistic, context-aware defenses and continuous adversarial testing to future-proof LLM deployments against evolving multi-turn threats. I Introduction Large language models (LLMs) such as GPT-4, Claude, Gemini, and LLaMA have rapidly become foundational technologies across domains including healthcare, research, and education. Their integration into resource-sensitive and safety-critical applications has dramatically raised the stakes for adversarial robustness, privacy, and trustworthy deployment. Despite substantial progress in model alignment, achieved through methods such as reinforcement learning from human or AI feedback (RLHF/RLAIF) and constitutional frameworks [ 3 , 21 ] , real-world deployments continue to encounter novel attack vectors that threaten both the integrity of outputs and the confidentiality of sensitive data. Figure 1 demonstrates how Transient Turn Injection (TTI) attacks can incrementally erode the safety constraints of large language models. The adversary initiates the process with a harmful seed prompt, which is typically blocked by the target model’s safety mechanisms. Rather than persisting in a single dialogue, the attacker LLM utilizes only the immediate previous response—without any access to full conversation history—to generate carefully crafted follow-up queries. This stateless, session-based interaction mimics a series of independent, memoryless requests, systematically reformulating the original intent to gradually elicit sensitive or restricted information from the target model. As detailed in Table 5, this method enables attackers to uncover technical details and key data points that would otherwise remain inaccessible in a single turn, revealing significant vulnerabilities in stateless moderation strategies and raising critical concerns for the deployment of LLMs in high-stakes environments. Figure 1: TTI Prompt Evaluation Example. A central question emerges: Are current alignment strategies sufficient to protect LLMs in scenarios where attackers possess significant resources or incentives—such as in medical AI, where even transient leaks can compromise patient safety or privacy? Traditional adversarial prompting, jailbreaks, and prompt-injection attacks often target single-turn weaknesses in moderation systems. However, increasingly sophisticated adversaries are exploiting the statelessness of per-turn moderation by distributing adversarial intent over sequences of individually innocuous queries. This not only undermines the effectiveness of current classifier-based guardrails but also challenges resource-constrained organizations to anticipate and mitigate subtle, distributed attack campaigns that can operate below conventional detection thresholds. In this work, we introduce Transient Turn Injection (TTI), a novel multi-turn attack methodology in which an attacker LLM adaptively generates independent prompts, each stateless and apparently malicious, to incrementally subvert target LLM safety policies. Unlike prior context-based jailbreaks, TTI does not require persistent conversational memory or session state, making it particularly insidious in environments where stateless APIs or restricted contexts are the norm. We demonstrate, through extensive automated evaluation, that even advanced, safety-tuned models—both proprietary and open-source—remain susceptible to TTI, with only certain deep-alignment strategies showing measurable robustness. Our key contributions are as follows: • We formally define the Transient Turn Injection (TTI) attack and systematically distinguish it from established multi-turn and context-based adversarial prompting techniques. • We develop an automated, black-box evaluation pipeline that pairs attacker and defender LLMs, enabling large-scale assessment of model vulnerabilities in real-world, stateless deployment scenarios. • We empirically benchmark a diverse set of commercial and open-source LLMs, revealing significant gaps in current alignment strategies and highlighting architectural features associated with improved TTI resistance. • We analyze the implications for resource-constrained and high-stakes domains (e.g., medical AI), where subtle adversarial breaches can have outsized consequences. • We provide recommendations and mitigation strategies, including session-level moderation and continuous adversarial red teaming, to guide the design of next-generation robust LLM safety frameworks. By illuminating these stateless, distributed attack vectors, this work aims to inform both practitioners and researchers about the emerging landscape of LLM vulnerabilities—and to chart a path toward more secure, trustworthy, and resource-aware language model deployments. II Backround Study write all models famil, multi turn attack, security vulnerability responsible parameters II-A Proliferation of Large Language Models In recent years, large language models (LLMs) have achieved remarkable progress in natural language understanding, reasoning, and general-purpose dialogue generation. This growth has been driven by advances in transformer architectures, scaling laws, and instruction tuning. Foundational models such as OpenAI’s GPT-3 , [ 14 ] , Meta’s LLaMA-2 , [ 2 ] , as well as emerging open-source alternatives like Mistral [ 1 ] and DeepSeek [ 8 ] , have pushed the boundaries of generalization across a wide range of tasks. Newer releases, including OpenAI’s GPT-4.1 Mini and GPT-4o , Meta’s LLaMA-4 Maverick , and Google’s Gemini 1.5 Flash and 2.0 Flash models, reflect a rapidly evolving ecosystem of proprietary and public models. These systems are now widely integrated into search engines, office suites, educational tools, and API-driven developer platforms, where their responses can impact real-world decisions. With this shift to real-world deployment, concerns over misuse, hallucination, bias, and safety violations have grown significantly [ 6 , 22 ] . Many LLM providers have acknowledged these concerns and incorporated alignment techniques such as reinforcement learning from human feedback (RLHF) [ 4 ] , yet empirical studies continue to demonstrate that even the most advanced models are vulnerable to adversarial attacks and subtle prompt manipulation [ 25 , 17 ] . As models grow in fluency and perceived reliability, the cost of a single failure—especially in domains like healthcare, national security, or law—can be severe. This motivates the growing body of research focused on adversarial red teaming, multi-turn alignment audits, and threat modeling, as safety becomes a first-class concern in LLM deployment. II-B Safety Mechanisms and Their Limitations Modern LLMs incorporate safety mechanisms such as supervised fine-tuning, reinforcement learning from human feedback (RLHF) [ 5 ] to align outputs with human values. These methods are often supported by post-hoc safety filters that screen for harmful or policy-violating responses. However, most of these systems operate on a single-turn basis, evaluating prompts and responses in isolation. This design leaves models vulnerable to multi-turn adversarial attacks, where unsafe outcomes are elicited gradually over a sequence of benign-sounding prompts. Prior studies have shown that even aligned models can be manipulated through iterative context-building [ 21 , 17 ] . In addition, transfer-based attacks [ 25 ] and alignment trade-offs [ 11 ] highlight deeper structural weaknesses in current safety pipelines. These findings emphasize the need for context-aware safety systems that can

Read the complete text