Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents

Yeran Gamage University of South Florida Independent AI Security Researcher [email protected] · 2026

Read Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents by Yeran Gamage University of South Florida Independent AI Security Researcher [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20911.txt.

Introduction

Gamage studies how operator-set safety rules hold up as language-model conversations lengthen. In a 4,416-trial study across 12 models and 8 providers, compliance with prohibitions falls from 73 percent at turn 5 to 33 percent at turn 16. Compliance with required actions stays at 100 percent. The author calls the gap Security-Recall Divergence and shows re-injecting rules before each model's safe depth restores compliance without retraining.

Catalog record

CREATORYeran Gamage University of South Florida Independent AI Security Researcher [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53090
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20911

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Omission Constraints Decay While Commission Constraints Persist in Long-Context LLM Agents Yeran Gamage University of South Florida Independent AI Security Researcher [email protected] Abstract LLM agents deployed in production operate under operator-defined behavioral policies (system-prompt instructions such as prohibitions on credential disclosure, data exfiltration, and unauthorized output) that safety evaluations assume hold throughout a conversation. Prohibition-type constraints decay under context pressure while requirement-type constraints persist; we term this asymmetry Security-Recall Divergence ( SRD ). In a 4,416-trial three-arm causal study across 12 models and 8 providers at six conversation depths, omission compliance falls from 73% at turn 5 to 33% at turn 16 while commission compliance holds at 100% (Mistral Large 3, p < 10 − 33 p<10^{-33} ). In the two models with token-matched padding controls, schema semantic content accounts for 62–100% of the dilution effect. Re-injecting constraints before the per-model Safe Turn Depth ( STD ) restores compliance without retraining. Production security policies consist of prohibitions such as never revealing credentials, never executing untrusted code, and never forwarding user data. Commission-type audit signals remain healthy while omission constraints have already failed, leaving the failure invisible to standard monitoring. 1 Introduction At turn 13 of a routine DevOps debugging session, Mistral Large 3 includes the required incident ID INC-08453 in 100% of its responses while violating its no-bullet-point constraint in 63% of those same outputs. The model is compliant and non-compliant in the same response. No adversarial instruction was issued. Ordinary context depth drove the failure. Production agents now operate across DevOps pipelines, customer service platforms, and enterprise workflows over tens or hundreds of turns, accumulating tens of thousands of tokens of context. No safety evaluation tests whether behavioral constraints survive this load. Benchmarks such as IFEval ( Zhou et al. 2023 ) and FollowBench ( Jiang et al. 2024 ) measure compliance on isolated generations. AgentDojo ( Debenedetti et al. 2024 ) and InjecAgent ( Zhan et al. 2024 ) evaluate agent security over 1–3 turns. The “lost in the middle” phenomenon ( Liu et al. 2024 ) shows factual recall degrades with context depth but does not test behavioral constraints. Multi-turn attacks such as Crescendo ( Russinovich et al. 2025 ) require active adversarial escalation. No prior work measures passive, depth-driven constraint decay under ordinary benign load. Prohibition-type constraints decay with depth; requirement-type constraints hold. We measure this across 12 models and 8 providers at six conversation depths ( t\in\{5,10,13,16,20,25\} ), generating 4,416 trials in a three-arm causal design. Each trial runs a realistic DevOps debugging scenario under 8 behavioral constraints: 3 commission-type rules the model must produce and 5 omission-type rules the model must suppress. All 8 are formatting rules detectable by string matching, enabling precise longitudinal measurement. Arm B (schema dilution) injects 20 cloud-infrastructure tool schemas, adding roughly 10,000 tokens of structured overhead. Arm A (no-dilution control) runs the identical conversation with no added schemas. Arm C (token-matched padding) replaces the schemas with semantically neutral content of similar length, isolating whether token volume or schema semantics drives the effect. Mistral Large 3’s no-bullet-point compliance falls from 73% at turn 5 to 33% at turn 16 while incident-ID compliance holds at 100% throughout (CMH \chi^{2}=147 , p<10^{-33} ). The asymmetry is significant in 3 of 4 models with per-constraint data and absent in Gemma 4 31B (Figure 1 ). Gemma 4 31B serves as an observed immune control, showing near-zero violations across 363 trials, which rules out a universal scoring artifact. While the exact cause remains unknown, we treat Gemma as an observed immune control because its resilience stems intrinsically from its weights or training regime rather than the shared NVIDIA NIM infrastructure, though further investigation is needed to determine if this immunity generalizes We term this asymmetry Security-Recall Divergence ( SRD ). The turn-depth window in which omission constraints have degraded while commission compliance remains high is the Zone of Exploitation . An adversary there triggers omission violations that short-horizon interactions would block, while commission audit signals mask the failure. The mechanism is attention dilution. Accumulated tokens push the policy document out of effective attention range. Schema injection adds roughly 10,000 tokens at the dilution window start (mean context at t=5 : 12,623 tokens vs. 2,639 without schemas). Schema semantic content drives the effect. Token-matched padding controls (Arm C) show schema content accounts for 62–100% of the dilution in the two models tested (Gemini 2.5 Flash and Llama 3.3 70B). Arm C was not administered to Mistral, Nemotron, or Qwen; causal isolation for the SRD-susceptible cluster. We have yet to causally isolate the SRD-susceptible cluster. Token volume nonetheless correlates with failure ( \hat{\beta}=+0.19 , p=3.4\times 10^{-8} , n=4{,}004 ); the turn-depth coefficient is negligible ( p=0.78 ). This extends the lost-in-the-middle finding ( Liu et al. 2024 ) from factual retrieval to behavioral constraint retention. Figure 1 : Security-Recall Divergence across three models and an immune control. Left: Omission constraint C3 (no bullet points) decays with injection depth in Mistral Large 3, Nemotron Super 120B, and Qwen 3.5 397B, while Gemma 4 31B holds at 100% throughout (immune control). Right: Commission constraint C4 ( STATUS: prefix) holds near 100% across all models at every depth. Gemma 4 31B shows near-complete compliance throughout (2 Arm B violations; observed immune control; mechanism not established). CMH \chi^{2} significant ( p<10^{-9} ) in all three SRD-susceptible models. Commission signals stay green while omission constraints fail. Threat Model. We consider an attacker who extends agent context token volume without controlling the system prompt or issuing any malicious instruction. Three real deployment scenarios admit this attack. In MCP-based agent deployments that load all registered tool schemas into the context window rather than retrieving tools via RAG-based selection, an attacker who registers additional tool definitions through legitimate MCP channels injects structured token volume that pushes security constraints toward the far end of effective attention range without delivering any malicious payload or triggering content filters. We term this attack Context-Exhaustion Injection ( CEI ). Trials peak at roughly 35,000 tokens, well within the 128K–1M context windows of frontier models. The vulnerability is attentional. Policy documents placed further from the current position receive less attention weight even when still present in the context window ( Liu et al. 2024 ) . Separately, a user who maintains a long customer service or DevOps session passively degrades omission constraint compliance without adversarial intent. Finally, tool call results, retrieved documents, and conversation history accumulate as a natural consequence of normal agentic operation. In all three scenarios, the model violates a prohibition without any malicious payload. This threat model applies to agents that enforce behavioral policies via system-prompt instructions without a synchronous external omission classifier covering those constraints. KV caching is a computational optimization and does not alter attention weight distribution; pinning the system prompt to the KV cache accelerates inference but does not protect the policy document from attentional dilution at deep context positions. Proxy Constraints. Our constraints are structurally identical to operational security constraints. “Never use bullet points” and “never disclose internal service hostnames” share the same structure: both are delivered via system prompt, both suppress a natural generative behavior, and neither is reinforced by RLHF training. Formatting proxies permit deterministic measurement at scale across 4,416 trials.

Read the complete text