Legal Zero-Days: A Novel Risk Vector for Advanced AI Systems
Greg Sadler, Nathan Sherburn · 2025
Read Legal Zero-Days: A Novel Risk Vector for Advanced AI Systems by Greg Sadler, Nathan Sherburn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10050.txt.
Introduction
Greg Sadler and Nathan Sherburn name a new risk for top smart systems, hidden flaws in law itself, in a 2025 paper. Such flaws can be used to cause wide harm at once with no court step first. They call these flaws legal zero days, like code flaws of the same name.
The pair gives a risk model to find and grade such flaws. A past dual citizen crunch in Australia shows how a small law slip can stall a whole state. Puzzle tests can probe if smart tools can find such flaws. Now tools find them with poor odds, but next tools may do more well.
Catalog record
| CREATOR | Greg Sadler, Nathan Sherburn |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10050
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Legal Zero-Days: A Novel Risk Vector for Advanced AI Systems Greg Sadler, Nathan Sherburn Abstract We introduce the concept of "Legal Zero-Days" as a novel risk vector for advanced AI systems. Legal Zero-Days are previously undiscovered vulnerabilities in legal frameworks that, when exploited, can cause immediate and significant societal disruption without requiring litigation or other processes before impact. We present a risk model for identifying and evaluating these vulnerabilities, demonstrating their potential to bypass safeguards or impede government responses to AI incidents. Using the 2017 Australian dual citizenship crisis as a case study, we illustrate how seemingly minor legal oversights can lead to large-scale governance disruption. We develop a methodology for creating "legal puzzles" as evaluation instruments for assessing AI systems’ capabilities to discover such vulnerabilities. Our findings suggest that while current AI models may not reliably find impactful Legal Zero-Days, future systems may develop this capability, presenting both risks and opportunities for improving legal robustness. This work contributes to the broader effort to identify and mitigate previously unrecognized risks from frontier AI systems. 1 Introduction As artificial intelligence systems become increasingly capable, the AI safety community has focused on evaluating "dangerous capabilities" that could pose catastrophic or existential risks. Current evaluation frameworks primarily assess well-established threat vectors: CBRN (chemical, biological, radiological and nuclear) capabilities, cyber offensive operations, large-scale misinformation campaigns, acceleration of AI development, and autonomous power-seeking behaviors [ 1 , 2 , 3 ] . Focus on these risks becoming "canonical" can leave significant blind spots in our understanding of how advanced AI systems could cause harm. We introduce the concept of Legal Zero-Days, a novel class of AI capabilities that could undermine institutional safeguards and governmental responses to AI-related incidents. Analogous to software vulnerabilities, a Legal Zero-Day is a previously unknown flaw in legal systems that, when exploited, can cause immediate and significant disruption to governmental operations or societal stability. Unlike traditional software exploits, these vulnerabilities exist at the intersection of complex legal frameworks, regulatory systems, and administrative processes that form the backbone of institutional governance. The 2017 Australian dual citizenship crisis exemplifies this phenomenon. Section 44(i) of the Australian Constitution, in place since 1901, prohibited dual citizens from serving in Parliament. However, the complex interplay between Australian constitutional law and various international citizenship regimes created a latent vulnerability. When discovered, this legal flaw forced the Deputy Prime Minister to resign, threatened the government’s parliamentary majority, potentially invalidated numerous administrative decisions, and paralyzed normal governmental operations for approximately 18 months. Crucially, this was not the result of new legislation or court rulings emerging from within the legal system; it was a sudden realization of what the existing law mandated. Although Legal Zero-Days may not constitute catastrophic risks in isolation, they represent a concerning capability for advanced AI systems. A sufficiently capable AI could accumulate such vulnerabilities as resources to bypass regulatory safeguards, disrupt responses to AI accidents, or systematically weaken the institutional frameworks designed to govern AI development and deployment. In misuse scenarios, such capabilities could enable malicious actors to paralyze governmental oversight or create regulatory chaos during critical periods. Current AI systems have demonstrated remarkable abilities to identify and exploit loopholes in rule-based systems, from video games [ 4 ] to trading algorithms. As these systems become more sophisticated, their ability to parse vast quantities of legal text, trace complex logical dependencies across regulatory frameworks and identify consequential vulnerabilities may pose novel risks to legal and institutional stability. This paper makes three key contributions: (1) we formalize Legal Zero-Days as a new category of AI risk and provide a concrete risk model based on real-world examples; (2) we develop the first systematic evaluation framework for measuring AI systems’ ability to discover legal vulnerabilities, using expert-crafted "legal puzzles" that simulate real legal flaws; and (3) we establish baseline performance across frontier AI models, finding that while current systems show limited capabilities in this domain (with the best model achieving 10.00% accuracy), the evaluation framework successfully differentiates model performance and provides a foundation for tracking this capability as AI systems advance. Our work extends beyond traditional AI safety evaluations by examining how AI capabilities might interact with the legal and regulatory infrastructure that governs society. As AI systems become more autonomous and influential, understanding their potential to discover and exploit legal vulnerabilities becomes increasingly critical for maintaining institutional stability and effective AI governance. 2 Legal Zero-Days risk model We define a Legal Zero-Day as a vulnerability in legal systems that meets five specific criteria, distinguishing it from ordinary legal disputes, ambiguities, or evolutionary changes in jurisprudence. Understanding these criteria is essential for recognizing when legal flaws pose systemic risks rather than routine legal challenges. 2.1 Defining characteristics A Legal Zero-Day must be: (1) a novel discovery about the functioning of a law or the interaction between multiple laws; (2) have immediate effect, with ramifications that impact real-world systems without requiring subsequent litigation, lengthy legal processes or discretionary action; (3) emerge externally rather than from within the legal system itself - disruptive legislation or executive actions do not qualify; (4) cause significant disruption that meaningfully impairs governmental or regulatory operations or other societal functions; and (5) be time-consuming to rectify, lasting weeks or months and resisting simple administrative or discretionary corrections. These criteria distinguish Legal Zero-Days from three related but distinct phenomena: "desuetude" (laws that lose their legal force through prolonged disuse), the "pacing problem" [ 5 ] (the challenge of legal and regulatory frameworks lagging behind rapid technological innovation) and routine legal evolution. While courts regularly interpret laws in new ways, Legal Zero-Days represent external discoveries of logical flaws that have immediate, disruptive consequences. The Australian dual citizenship crisis meets all five criteria: it involved a novel interpretation of how constitutional law interacted with international citizenship regimes, immediately affected parliamentary eligibility, was triggered externally rather than created through normal legal processes, significantly disrupted government operations and required 18 months of constitutional crisis and by-elections to resolve. 2.2 Prevalence and examples Legal Zero-Days appear to be more common than initially expected. Beyond the Australian case, we observe similar patterns globally. For example, New Zealand’s 2013 discovery that Oaths of Office were being administered incorrectly [ 6 ] , invalidating numerous appointments and various instances of mass prisoner releases due to errors in legal processes [ 7 ] . The frequency of these discoveries by human experts, working within domain-specific knowledge and limited time constraints, suggests a vast landscape of undiscovered vulnerabilities. Unlike software systems, legal frameworks lack automated testing mechanisms and often involve multiple overlapping jurisdictions with inconsistent maintenance and oversight. 2.3 Implications for the AI risk landscape While Legal Zero-Days may not pose a direct safety risk, they do operate to undermine safeguards and magnify other risks. Consider a scenario in which an advanced AI system seeking to expand its capabilities discovers legal flaws that invalidate key provisions of AI safety legislation, paralyze regulatory agencies through administrative chaos, or exploit constitutional vulnerabilities to challenge governmental authority over AI governance. The compounding effects of multiple simultaneous Legal Zero-Days could create regulatory paralysis during critical periods when swift governmental response is essential. In misuse scenarios, malicious actors with access to Legal Zero-Day discovery capabilities could weaponize institutional disruption, timing legal