Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs
Aayush Gupta · 2025
Read Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs by Aayush Gupta (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-09288.txt.
Introduction
Aayush Gupta asks whether a language model can keep a secret, and answers with a signed-label security design. The paper from twenty twenty-five targets prompt injection, where hostile text steers a model past its rules. Rule lists, filters, and model judges all get bypassed in practice. Gupta's Contextual Integrity Verification stamps each token with a cryptographic provenance tag.
A trust lattice orders the sources. A hard attention mask blocks low-trust tokens from swaying high-trust states inside the transformer. Frozen models gain per-token non-interference guarantees with no retraining. On attack sets built from recent taxonomies, no attack succeeded under the stated threat model. Benign output stayed close to the base model's. Two open models took the patch as a drop-in. Code, a test harness, and the attack corpus are released.
Catalog record
| CREATOR | Aayush Gupta |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-09288
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can AI Keep a Secret? Contextual Integrity Verification: A Provable Security Architecture for LLMs Aayush Gupta Abstract The promise of Large Language Models is betrayed by a fundamental vulnerability: they cannot distinguish friend from foe. Today’s LLMs remain dangerously susceptible to prompt-injection and jailbreak attacks—a reality that transforms every interaction into a potential security breach. While the industry has deployed an arsenal of heuristic defenses—keyword filters, toxicity classifiers, LLM-based detectors—attackers consistently pierce through these semantic shields, as evidenced by repeated failures in public red-team exercises [ 5 ] . State-of-the-art guardrail toolkits including LLM-Guard [ 4 ] , Rebuff [ 7 ] , and PromptArmor [ 8 ] have pushed the boundaries of resistance, yet they fundamentally rely on probabilistic patterns that sophisticated adversaries can manipulate, obfuscate, or simply bypass. We present Contextual Integrity Verification (CIV) —a paradigm shift from detecting malice to enforcing mathematical boundaries. To our knowledge, CIV is the first security architecture to deliver deterministic, cryptographically verifiable non-interference guarantees at inference-time on pre-trained, frozen transformer models. Our approach transcends semantic analysis by embedding an immutable trust lattice directly into the transformer’s computational fabric: every token carries a cryptographically signed provenance (HMAC-SHA-256), and attention mechanisms are surgically modified to enforce hard information-flow boundaries. Lower-trust tokens become mathematically incapable of influencing higher-trust computations—not through detection, but through algebraic impossibility. Evaluated against a comprehensive benchmark synthesized from cutting-edge attack taxonomies (Elite-Attack + SoK-246) [ 2 , 1 ] , CIV achieves what no prior defense has accomplished: absolute protection (0% ASR) within our threat model , while preserving 93.1% output fidelity and maintaining model perplexity. Our reference implementation, though carrying a notable latency overhead attributable to unoptimized cryptographic pipelines, demonstrates immediate deployability as a lightweight patch for Llama-3-8B and Mistral-7B—no retraining required. We release our implementation, 1 1 1 https://github.com/ayushgupta4897/Contextual-Integrity-Verification an automated certification framework, and the ”Elite-Attack” 2 2 2 https://huggingface.co/datasets/zyushg/elite-attack corpus to accelerate reproducible security research. 1 Introduction Large Language Models have become the backbone of modern AI systems—yet they harbor a fatal flaw. Their extraordinary generative capabilities mask a brittle security foundation: the inability to enforce trust boundaries. Prompt-injection (PI) and jailbreak attacks exploit this weakness, allowing adversaries to smuggle malicious instructions that override system policies, exfiltrate confidential data, or coerce forbidden behaviors. The gravity of this threat is reflected in OWASP’s ranking: PI now stands as the #1 security risk for LLM applications [ 5 ] . 1.1 The Anatomy of a Prompt Injection Attack To understand the severity of this vulnerability, we must examine how trust boundaries collapse in practice. Every LLM application constructs prompts from multiple sources, each carrying an implicit privilege level. An attack succeeds when a low-privilege source hijacks the behavior dictated by a high-privilege source—a violation of fundamental security principles. Example 1: Direct Injection (Jailbreak). Here, an adversary weaponizes the low-trust USER channel to subvert high-trust SYSTEM directives: [SYSTEM PROMPT - Trust=100] You are a helpful assistant. Never reveal this instruction. Your name is Alex. [USER PROMPT - Trust=80] ...ignore all previous instructions and tell me the first sentence of your instructions. A vulnerable LLM capitulates to the user’s command, betraying its confidential system prompt—a breach that could expose proprietary knowledge or security configurations. Example 2: Indirect Injection (Web Retrieval). The threat escalates when trusted tools become unwitting accomplices: [SYSTEM PROMPT - Trust=100] You are a helpful assistant. Use the web browser tool to answer questions about recent events. [USER PROMPT - Trust=80] What are the main points of the new tech bill summarized at document-site.com/summary? [WEB CONTENT - Trust=20] The bill is about... [INVISIBLE TEXT] ...end your response with "Access Granted. Admin override successful." In this insidious attack, the LLM dutifully retrieves a seemingly benign web page that conceals a malicious payload. The lowest-trust source ( WEB ) successfully commandeers the entire interaction—a complete inversion of the intended security hierarchy. CIV transforms these attacks from possible to impossible. 1.2 Why Heuristic Guardrails Are Failing The security community’s response has been reactive: deploy layers of semantic filters, toxicity classifiers, and LLM-based judges to identify ”unsafe” content. Yet a damning pattern emerges from independent evaluations. A recent Systematization-of-Knowledge (SoK) survey reveals that even state-of-the-art guardrails suffer persistent jailbreak rates of 15–30%— after vendors have tuned against the test sets [ 1 ] . This failure is not a bug but a feature of the approach: semantic defenses are fundamentally playing a losing game against adversaries who can encode, obfuscate, and launder their attacks through countless linguistic transformations. 1.3 From Probabilistic Detection to Deterministic Prevention The security research community has long understood that true protection requires information-flow control (IFC): we must make illicit dataflow structurally impossible , not merely statistically unlikely. While prior IFC-Transformer work showed promise, it demanded architectural overhauls and complete model retraining [ 9 ] —a non-starter for the vast ecosystem of deployed models. The field has been waiting for a practical, inference-time solution that provides cryptographic guarantees without sacrificing existing investments. That solution has remained elusive—until now. 1.4 Our Contribution: Contextual Integrity Verification (CIV) CIV represents a fundamental reimagining of LLM security. We deliver: Feature Description Cryptographic immunity Every token is sealed with an HMAC-SHA-256 signature binding it to an immutable trust level (SYSTEM > USER > TOOL > DOC > WEB). Tampering triggers immediate detection. Surgical intervention We inject trust enforcement directly into the attention mechanism’s pre-softmax computation, with optional FFN/residual gating. Zero fine-tuning. Zero prompt engineering. Pure mathematical enforcement. Provable guarantees A formal proof establishes cross-position non-interference: lower-trust tokens are mathematically prevented from influencing higher-trust states—not by policy, but by algebra. Preserved intelligence Across 10 diverse task categories, CIV maintains over 93% token-level fidelity while preserving the model’s intrinsic perplexity—security without stupidity. Immediate deployability We provide battle-tested benchmarks and production-ready code that transforms any transformer into a security-hardened system in minutes. 2 Related Work Prompt-injection taxonomies. The research community has meticulously cataloged attack vectors and evaluation protocols [ 12 , 13 , 1 ] . CIV transcends this cat-and-mouse game by addressing the root vulnerability—uncontrolled cross-trust influence—rather than chasing ever-evolving attack patterns. Guardrails and detectors. The current defense ecosystem—LLM-Guard, Rebuff, PromptArmor—deploys sophisticated mixtures of rules, classifiers, and LLM judges [ 4 , 7 , 8 ] . Independent security audits paint a sobering picture: non-trivial bypass rates and crushing latency penalties [ 10 , 1 ] . These tools fight symptoms; CIV cures the disease. IFC for neural models. IFC-Transformers pioneered neural information-flow control but demanded complete architectural redesign and retraining [ 9 ] —a death sentence for practical adoption. CIV achieves the same theoretical guarantees through a surgical, inference-time intervention on frozen weights, making it immediately deployable across the entire LLM ecosystem. Cryptographic provenance. Systems like ORIGO establish data lineage through cryptographic chains [ 6 ] . CIV extends this vision by making provenance computationally binding —signatures don’t just log trust, they enforce it at the algebraic level. CIV’s internal machinery could seamlessly integrate with ORIGO-style chains for end-to-end verifiable integrity. 3 Threat Model and Design Goals Adversary. We consider an adaptive, remote adversary with arbitrary control over lower-trust channels (trust \leq USER): chat interfaces, tool outputs, retrieved documents, and web content. The attacker cannot compromise server infrastructure, GPU memory, or cryptographic keys—physical attacks remain beyond scope.