Scaling Laws For Scalable Oversight
Joshua Engels, David D. Baek, Subhash Kantamneni, Max Tegmark · 2025
Read Scaling Laws For Scalable Oversight by Joshua Engels, David D. Baek, Subhash Kantamneni, Max Tegmark (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18530.txt.
Introduction
Engels, Baek, Kantamneni, and Tegmark built a frame in twenty twenty-five to gauge how well weak AI can watch strong AI as both grow. Oversight is a key hope for control of future top systems. Their frame scores the win chance of watch play from each side's skill gap.
They test the frame on a nim game tweak, then on four watch games like debate and code traps. Each game gives a growth law for how watch skill scales. Stacked watch layers can help, with a best count of layers they solve. Win rates at one set gap ran from near ten to near fifty-two percent.
Catalog record
| CREATOR | Joshua Engels, David D. Baek, Subhash Kantamneni, Max Tegmark |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-18530
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scaling Laws For Scalable Oversight Joshua Engels, David D. Baek, Subhash Kantamneni, Max Tegmark Abstract Scalable oversight, the process by which weaker AI systems supervise stronger ones, has been proposed as a key strategy to control future superintelligent systems. However, it is still unclear how scalable oversight itself scales. To address this gap, we propose a framework that quantifies the probability of successful oversight as a function of the capabilities of the overseer and the system being overseen. Specifically, our framework models oversight as a game between capability-mismatched players; the players have oversight -specific Elo scores that are a piecewise-linear function of their general intelligence, with two plateaus corresponding to task incompetence and task saturation. We validate our framework with a modified version of the game Nim and then apply it to four oversight games: Mafia, Debate, Backdoor Code and Wargames. For each game, we find scaling laws that approximate how domain performance depends on general AI system capability. We then build on our findings in a theoretical study of Nested Scalable Oversight (NSO), a process in which trusted models oversee untrusted stronger models, which then become the trusted models in the next step. We identify conditions under which NSO succeeds and derive numerically (and in some cases analytically) the optimal number of oversight levels to maximize the probability of oversight success. We also apply our theory to our four oversight games, where we find that NSO success rates at a general Elo gap of 400 are 13.5% for Mafia, 51.7% for Debate, 10.0% for Backdoor Code, and 9.4% for Wargames; these rates decline further when overseeing stronger systems. 1 Introduction Many frontier AI companies are rapidly advancing toward their stated goal of building artificial general intelligence (AGI) and beyond. This has intensified interest in techniques for ensuring that such systems remain controllable and behave in beneficial ways. One major cluster of such techniques includes Recursive Reward Modeling ( Leike et al., 2018 ) , Iterated Amplification ( Christiano et al., 2018 ) , Scalable Oversight ( Bowman et al., 2022 ) , Weak-to-Strong Generalization ( Burns et al., 2023 ) , Hierarchical Supervision ( Shah et al., 2025 ) , and Recursive Oversight ( Anthropic Alignment Science Team, 2025 ) . These methods share a central goal: enabling weaker systems to oversee stronger ones (weak-to-strong oversight), ultimately enabling us to oversee superhuman systems. A key idea is that scalable oversight can be bootstrapped: weaker systems oversee stronger ones, which then oversee even stronger models in the next stage—allowing oversight to scale alongside capabilities recursively. We refer to this iterated process as Nested Scalable Oversight , or NSO for brevity. To date, most guarantees for NSO have been qualitative. However, safety standards for other high-risk technologies are quantitative. For instance, the U.S. requires aircraft to have a fatality rate below 10^{-5} per hour ( Federal Aviation Administration, 2023 ) and nuclear reactors to have a core damage frequency below 10^{-4} per year ( US Nuclear Regulatory Commission, 2001 ) . Motivated by these risk thresholds, this paper takes a first step towards quantifying NSO failure rates: 1. In Section 2 , we formalize the oversight problem as a game between a stronger Houdini and a weaker Guard . We then introduce and validate (on a toy Counting-to-21 game) the novel Double ReLU model to describe how domain performance scales with general intelligence. 2. In Section 3 , we analyze four oversight games—Mafia, Debate, Backdoor Code, and the novel Wargames—and find that oversight performance scales with general intelligence across all cases. 3. In Section 4 , we develop a theoretical framework for Nested Scalable Oversight (NSO) and derive the optimal number of oversight steps given game-specific parameters. Fig. 1 shows an overview of these different parts of our paper. We discuss related work in Appendix B . Figure 1: Overview of our framework. Left: We evaluate oversight scenarios by simulating adversarial LLMs (Houdinis) playing against trusted monitor LLMs (Guards) in games that abstract real-world oversight challenges. Middle: For each game, we compute pairwise win rates between all Houdini–Guard pairs and use these to derive Elo ratings as a measure of model performance. Right: Finally, we conduct a theoretical analysis to explore how the dynamics of these games inform the design of Nested Scalable Oversight systems for controlling superintelligent AI. 2 Two Player Games Between Unmatched Players In this section, we analyze two-player games between unmatched players with different intelligences as a tractable proxy for real world weak-to-strong oversight. The key result is an effective four-parameter double-ReLU model that describes how Domain Elo (skill at the particular game) scales against General Elo (broad competence), which we will make use of in our subsequent analysis of various oversight games ( Section 3 ) and Nested Scalable Oversight analysis ( Section 4 ). 2.1 Game Setup Since we are interested in oversight games, we will refer to the two players as the Houdini ( H ) (whose goal is to thwart oversight) and the Guard ( G ) (a typically less capable agent whose goal is to prevent this). Each game ends with H or G winning. We defer formal definitions to Section D.1 . For each oversight game, we begin by collecting empirical win-rate statistics between all pairs of Guard and Houdini models. This results in a square win-rate matrix {\bf P} , where P_{ij}\in[0,1] is the fraction of the time that Guard i defeats Houdini j . Using this matrix, each model is assigned two separate Elo ( Elo, 1967 ) scores: one score E_{G} for performance when acting as a Guard, and another score E_{H} when acting as Houdini. The expected win probability for a Guard with Elo E_{G} against a Houdini with Elo E_{H} is given by the standard Elo formula p(E_{G},E_{H})=\left(1+10^{\frac{E_{H}-E_{G}}{400}}\right)^{-1}. (1) The use of base-10 and the divisor of 400 are historical conventions from chess. We use standard optimization to fit game Elos from entries of {\bf P} (see Section D.2 for details). Figure 2: Left: Our hypothesis fit (see Eq. 2 ) for the relationship between general and domain specific Elo. Right: The performance of LLMs in the Counting-to-21 game exhibits this relationship. 2.2 Modeling General and Domain Elo relation with the Double ReLU We argue that there are three important phases of the relationship between general and domain intelligence in any game: 1. Task incompetence : There is some minimum amount of general intelligence g_{1} needed to play the game; below this, performance plateaus. E.g., the average toddler has a 0% win rate at chess. 2. Intelligence payoff region : Once the minimum intelligence is exceeded, increases in general intelligence are reflected in linear increases in domain intelligence 1 1 1 While actual payoffs might exhibit nonlinear behavior, we simplify our analysis by modeling this relationship as linear. . 3. Task saturation : Above some intelligence g_{2} , further intelligence gains yield no performance benefit. E.g., two superintelligent tic-tac-toe agents playing optimally always tie. Formally, we capture these phases by modeling the relationship between the previously described domain-specific Elo, E_{d} , and a general intelligence Elo, g , as a piecewise linear function: \displaystyle E_{d}(g)=E_{d_{1}}+\alpha\left[\textrm{ReLU}(g-g_{1})-\textrm{ReLU}(g-g_{2})\right]=\begin{cases}E_{d_{1}}&\text{if }g\leq g_{1}\\ E_{d_{1}}+\alpha(g-g_{1})&\text{if }g_{1}<g<g_{2}\\ E_{d_{2}}&\text{if }g\geq g_{2}\end{cases} (2) with E_{d_{1}}<E_{d_{2}} .