Verifying Restrictions on Frontier AI Research

Aaron Scher · 2026

Read Verifying Restrictions on Frontier AI Research by Aaron Scher (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28694.txt.

Introduction

Aaron Scher, writing alone, asks how the world could ever check a promise to pause frontier artificial intelligence research. Proposals for an international halt to superintelligence development depend on verification, and since algorithmic progress comes from research itself, the research may need restricting too, not just computers and data. Scher stays agnostic about which research would be banned and studies only the checking.

Early chapters map the considerations that make research restrictions verifiable or not, including the experimental infrastructure serious work requires. The core is a catalogue of twenty-eight candidate verification mechanisms, from whistleblowers and search warrants to audits of training code and conventional intelligence gathering. Some are not yet implementable and some may prove undesirable. Mapping the option space, Scher argues, is the foundation for developing the promising few into tools.

Catalog record

CREATORAaron Scher
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.28694

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Verifying Restrictions on Frontier AI Research Aaron Scher Abstract The premature development of artificial superintelligence poses major risks to humanity, so researchers have proposed international agreements halting such development until it can be done safely. AI progress depends primarily on compute, algorithms, and data; a durable halt would address all three so that advances in one input do not counteract restrictions on another. Improvements to AI algorithms are driven largely through research activities, so this research may need to be restricted during a halt. Given low international trust, signatories will want to verify compliance. This paper analyzes how such restrictions on AI research could be verified, while remaining agnostic about what specific research would be prohibited. It first explores key considerations that affect the verifiability of research restrictions, such as the computational infrastructure necessary for experiments. It then catalogs 28 candidate verification mechanisms. These mechanisms include whistleblowers, search warrants, reviews of AI training code, standard intelligence gathering tools, and more. Some of these mechanisms are not yet implementation-ready, and some might be undesirable upon further inspection. By examining the space of potential options, this work provides a foundation for future research to develop the most promising mechanisms into deployable tools. 1 Introduction Many experts in the field of artificial intelligence (AI) worry that increasingly intelligent AI systems will pose catastrophic risks to humanity, including human extinction ( 11 ; 55 ) . To address this threat, some have proposed delaying the development of artificial superintelligence (ASI) until development can be done safely ( 21 ; 5 ; 42 ; 40 ; 31 ) . To this end, 42 provide a draft international agreement aimed at preventing the premature creation of ASI. For simplicity, we use the agreement from 42 as a scaffold for this work, asking how research restrictions could be verified under that agreement; however, these mechanisms would also be useful in other policy proposals that involve research restrictions. The agreement has two main pillars focused on stopping the development of better AIs: chip controls and research restrictions . Both of these are needed; neither would be sufficient on their own. The primary drivers of AI progress are compute, data, and algorithms ( 10 ) . If only compute is controlled, progress would still continue via advances in data and algorithms ( 28 ; 29 ; 27 ) . The proposed agreement includes research restrictions to fill this gap, focused primarily on research that is likely to advance the state of frontier AI capabilities (e.g., frontier AI research, general-purpose AI capabilities research, potentially AI chip design research). Certain research would also endanger the verification system established in the agreement, so it may be desirable to prohibit such research as well (e.g., decentralized training methods). The question of which specific research should be restricted during a halt is out of scope for this paper. We focus instead on the verification question: How could governments verify compliance with restrictions on AI research, especially within signatory countries? We answer this question in two ways: • First, we analyze key factors influencing the verifiability of research restrictions. These include the need for compute in testing research ideas, the present rarity of human experts who can push the frontier, and the likely automation of AI research activities. • Second, we map the space of potential verification mechanisms, cataloging 28 high-level mechanisms that could be used to verify compliance with research restrictions. Examples of these mechanisms include whistleblowers, automated code reviews, and sting operations. To simplify, we use specific terminology throughout this paper. Restricted research refers to research that is prohibited by an international agreement, for instance because it contributes to AI capabilities or threatens the verification system; this paper remains agnostic about where the boundaries of that category should be drawn. 42 use restricted research as an umbrella term covering two forms: controlled research, which is subject to monitoring, and banned research, which is disallowed. In this paper, we use restricted research to refer to the latter. High-risk organizations and researchers are organizations and individuals that are relatively likely to carry out impactful restricted research, for instance because their legitimate work is similar to restricted research (i.e., it falls in the controlled category) or because they have relevant expertise and resources. For example, current AI developers such as OpenAI and Anthropic would qualify as high-risk organizations due to their expertise and resources. After an agreement, they would not be allowed to continue conducting frontier capabilities research, but they could, for instance, serve existing models. The verification authorities are a group of people responsible for verification, for instance a technical body staffed by representatives from certain signatory countries, as in the agreement from 42 (though we aim to be agnostic to the design of such a body as this is primarily a political rather than technical question). The choice of verification authorities affects the political viability and security risk of mechanisms, but these considerations will be balanced against countries’ need to obtain justified confidence in compliance, not trusting each other to self-enforce an agreement. Inspectors are employees of the verification authority who carry out specific monitoring or inspection tasks. Finally, we use the term verification to refer to a broad set of methods and actions that could make one party justifiably more confident that another is adhering to agreed-upon rules. This is somewhat broader than the standard use as it includes preventative mechanisms and useful components as well as core information collection—we note when our use is more broad than typical. To ground this work, our investigation is motivated by three main threat actors who may attempt to violate research restrictions. First, covert nation-state actors who are highly competent at security, may have secret data centers, and whose existence may not be known to verification authorities (e.g., a secret Manhattan Project for ASI). Second, established organizations that are known to verification authorities, are highly competent at AI development, and are generally law-abiding (e.g., OpenAI, DeepSeek, Prime Intellect). Third, rogue academics and less-resourced researchers who may be ideologically motivated and are likely to make progress either slowly (and with contributions from many individuals) or through new paradigm advances, if at all. In some cases, the potential violator is known, so verification would involve countries agreeing to monitoring and inspections to verify compliance. In other cases the main verification difficulty is to detect the presence of such a project. 2 Key considerations for the verifiability of research restrictions Compute is needed to test many research ideas within the machine learning paradigm. 7 investigates how much compute was used to develop the algorithmic innovations used in Llama 3 and DeepSeek-V3 and finds that around half of the innovations were developed using less than 8 H100 equivalents of computing capacity (or around 1\times 10^{21} FLOP when measuring total computations). This quantity is larger than what average consumers own, but small enough to be effectively undetectable: we are not aware of any detection mechanism that could reliably find all such clusters in the world, though such an analysis is outside the scope of this work. Validating research ideas at scale requires substantial compute. Industry practitioners report that many ideas which perform well in small experiments nonetheless scale poorly, and to

Read the complete text