Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities

Corban Villa, Sohee Kim, Austin Chu, Alon Shakevsky, Raluca Ada Popa · 2026

Read Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities by Corban Villa, Sohee Kim, Austin Chu, Alon Shakevsky, Raluca Ada Popa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26933.txt.

Introduction

Corban Villa, Sohee Kim, Austin Chu, Alon Shakevsky, and Raluca Ada Popa present Chai, an AI system that hunts cryptographic misuse bugs. Misusing a crypto library, such as checking certificates wrongly, is a dangerous and common flaw class. Unlike memory bugs, it has no instrument that flags violations automatically. Chai instead mines natural signals: disagreements between implementations of the same standard.

Differential testing compares libraries against each other. Chai uses AI to judge which discrepancies are real security issues. It then catalogues each flaw once, at library level, and traces it through a dependency graph to the applications downstream. One audit thus covers many codebases. Across certificate, token, and login libraries, Chai found a previously unknown critical flaw in an SSL library used on billions of devices, plus bugs behind a major browser.

Catalog record

CREATORCorban Villa, Sohee Kim, Austin Chu, Alon Shakevsky, Raluca Ada Popa
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26933

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Chai: Agentic Discovery of Cryptographic Misuse Vulnerabilities Corban Villa, Sohee Kim, Austin Chu, Alon Shakevsky, Raluca Ada Popa Abstract AI-assisted vulnerability discovery has proven effective for bug classes like memory safety, where instrumentation confirms memory violations and efficiently filters false positives. Many dangerous vulnerability classes, such as cryptographic misuse, however, lack any comparable instrumentation. In this work, we present Chai, an AI-based system that discovers and validates cryptographic misuse vulnerabilities through naturally occurring signals. To achieve this, Chai rethinks the classical technique of differential testing by leveraging AI to 1) improve precision for detecting real security issues in libraries, and 2) repurpose commonly overlooked discrepancies as leads for tangible vulnerabilities in downstream applications. In doing so, Chai inverts the prevailing paradigm of AI vulnerability discovery: instead of auditing one codebase for many flaws, it catalogs flaws at the library level and propagates them across a cryptographic dependency graph, delivering compounding efficiency gains. We evaluate Chai across X.509, JWT, and SAML libraries. Chai discovered a previously unknown critical vulnerability in an SSL library that powers billions of devices, along with security bugs in one library behind a major web browser and another in major Linux distributions. In total, these techniques surfaced over 100 vulnerabilities. I Introduction Building secure software requires not just sound design but continuous effort to find flaws before they are exploited. Developers search for vulnerabilities using techniques from static analysis and fuzzing to end-to-end penetration testing. Historically, defenders have benefited from the scarce expertise needed to discover and exploit vulnerabilities, limiting adversaries’ ability to operate at scale [ 1 , 2 , 3 ] . AI-assisted vulnerability discovery is challenging this assumption: introducing models into the pipeline, whether augmenting classical techniques or driving end-to-end approaches such as Anthropic’s Mythos [ 4 ] , has improved results across vulnerability classes and surfaced flaws of increasing complexity [ 5 , 6 , 7 , 8 , 9 ] . CVE disclosures in the most recent 90-day window grew {\sim} 50% year-over-year [ 10 ] , while reported time-to-exploit fell from {\sim} 21.5 days in 2025 to 24 hours [ 11 ] . As the cost of discovery and exploitation falls, the vulnerability classes most likely to be weaponized demand attention. Yet many systems concentrate on verifiable domains, where fuzzing infrastructure, crash reproduction, and sanitizers provide oracles that confirm findings and filter hallucinated or deceptive results [ 12 , 13 , 14 , 15 , 4 , 16 , 17 , 18 ] . These systems have produced findings in high volume, yet likely exploitable memory-safety vulnerabilities occurred roughly 3.4 \times less frequently than a year prior [ 19 ] . This shows that volume alone does not translate into real-world impact. By contrast, cryptographic and authentication failures rank highly in both the OWASP Top Ten [ 20 ] and the MITRE Top 25 [ 21 ] , and can propagate to thousands of downstream applications [ 22 , 23 , 24 , 25 ] . Making agent-based discovery practical here thus requires an address-sanitizer analogue, but one that validates higher-level security properties. app jws-express node-jose node:crypto OpenSSL JWT (RFC 7519) JWS (RFC 7515) JWA (RFC 7518) JWT BCP (RFC 8725) JWK/JWKS (RFC 7517) RSA (RFC 8017) Fig. 1: Cryptographic dependency stack for a JWT-verifying web application ( \bullet ), its underlying libraries ( \bullet ), and corresponding specifications ( \bullet ). Unfortunately, identifying cryptographic misuse is non-trivial. Consider a widely used cryptographic authentication protocol such as JSON Web Tokens (JWT). What appears to be a single protocol is in fact a composition of several specifications for signing, encryption, algorithms, and key handling, realized independently across dozens of libraries and languages (Figure 1 ). These implementations resolve the specification’s under-specified corners differently, and the consequences can be severe. For example, in 2018, the node-jose library accepted a verification key embedded in the token itself by default, letting an attacker supply their own key and forge valid tokens (CVE-2018-0114) [ 26 ] . In isolation the behavior looked correct: the signature verified, and the standard does permit embedding keys. Instead, there was a mismatch in assumptions: the library assumed the application would decide which keys to trust, while applications misinterpreted successful verification as trusted verification. Whether the behavior is even wrong is not always clear-cut: these judgments are subjective and nuanced, and can differ from one developer to the next for the same protocol. There is no address-sanitizer equivalent to confirm a violation or to signal what is correct. The agent can argue that a vulnerability exists, but with no programmatic evidence to ground the claim, security experts must filter through these issues one by one. A long-standing approach to uncovering these types of issues in cryptographic libraries is differential testing, a classical technique introduced by McKeeman in 1998 [ 27 ] . By running the same input through multiple implementations and comparing their behavior, it surfaces disagreements without needing to know the correct answer in advance. On its own, however, differential testing is often noisy and still labor-intensive: it produces many non-actionable discrepancies, and steering it toward meaningful inputs has required protocol-specific grammars or mutators that take considerable effort to build for each new protocol [ 28 , 29 , 30 , 31 ] . Even then, for a complex protocol the input space is large enough that exploring it thoroughly is non-trivial. In this work, we present Chai, an AI-based system for discovering cryptographic misuse vulnerabilities. Our first insight is to pair differential testing with AI so that each covers the other’s weakness, yielding agentic discovery with a signal we can check rather than trusting the AI alone: differential testing supplies a verifiable and deterministic signal that a discrepancy exists, while AI provides protocol knowledge and adaptive search to surface the meaningful issues without hand-built grammars. This pairing feeds our second insight, discrepancy tracing , where we use ambiguities to invert the prevailing paradigm of agentic vulnerability discovery [ 4 , 32 , 16 ] (Figure 2 ). Rather than start from a project and search it for any number of potential bugs, we start from a concrete discrepancy, surfaced by differential testing, and trace it to the projects it affects. Each discrepancy resolves either to a flaw in the library itself or to an ambiguity that prior approaches dismissed as non-reportable but that, we observe, often leads to tangible vulnerabilities in downstream applications. In summary, Chai contributes a systematic approach to discovering cryptographic misuse vulnerabilities through differential testing and AI. Bug Class Agent Agent Agent Project 1 Project 2 Project 3 Vuln Vuln Vuln Fig. 2: The prevailing paradigm. Agents are often pointed at a project and tasked to search it for specific bug classes over many runs. To better understand the design motivation, consider our example for the JWT stack in Figure 1 . A naive agent audits one application at a time, scanning its codebase for bugs. These ambiguity-induced vulnerabilities, however, are often invisible from a project’s own codebase, since the code for the unintuitive behavior lives in an external dependency: the application can look correct in isolation, and the hazard surfaces only in the contract between it and the library beneath it. An agent could, in

Read the complete text