Control Tax: The Price of Keeping AI in Check
Mikhail Terekhov, Zhen Ning David Liu, Caglar GulcehreSamuel Albanie · 2025
Read Control Tax: The Price of Keeping AI in Check by Mikhail Terekhov, Zhen Ning David Liu, Caglar GulcehreSamuel Albanie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-05296.txt.
Introduction
Mikhail Terekhov and co-authors measure the price of keeping AI agents under control, in a paper from 2025. As agents take on high-stakes work, oversight systems watch them for hidden misbehavior. Whether such watching is worth its cost decides if it gets used.
The team names that cost the control tax: the money and running overhead of adding control to a system. Their framework links the accuracy of a monitoring classifier to the safety assurance it buys. They then tested top language models in adversarial setups where attacker models slipped subtle backdoors into code while watcher models hunted for them. Real cost estimates for control setups follow, with tuned watching plans that balance safety against audit budgets.
Catalog record
| CREATOR | Mikhail Terekhov, Zhen Ning David Liu, Caglar GulcehreSamuel Albanie |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-05296
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Control Tax: The Price of Keeping AI in Check Mikhail Terekhov, Zhen Ning David Liu, Caglar GulcehreSamuel Albanie Abstract The rapid integration of agentic AI into high-stakes real-world applications requires robust oversight mechanisms. The emerging field of AI Control (AIC) aims to provide such an oversight mechanism, but practical adoption depends heavily on implementation overhead. To study this problem better, we introduce the notion of Control tax ---the operational and financial cost of integrating control measures into AI pipelines. Our work makes three key contributions to the field of AIC: (1) we introduce a theoretical framework that quantifies the Control Tax and maps classifier performance to safety assurances; (2) we conduct comprehensive evaluations of state-of-the-art language models in adversarial settings, where attacker models insert subtle backdoors into code while monitoring models attempt to detect these vulnerabilities; and (3) we provide empirical financial cost estimates for control protocols and develop optimized monitoring strategies that balance safety and cost-effectiveness while accounting for practical constraints like auditing budgets. Our framework enables practitioners to make informed decisions by systematically connecting safety guarantees with their costs, advancing AIC through principled economic feasibility assessment across different deployment contexts. 1 1 1 Website of the project: https://mikhailterekhov.github.io/control-tax/ . Code: https://github.com/MikhailTerekhov/control-tax-code . The repository also contains the full dataset of backdoors from our attacker models. 1 Introduction General-purpose AI systems, of which Large Language Models (LLMs) are the most notable example, are quickly making their way into our daily lives. More recently, these systems are also being integrated into broader communication and information processing pipelines. This large-scale deployment, combined with AI reliability issues, brings with it a multitude of risks, coming either from malicious users ( Greshake et al. 2023 ) or as a subversion by the AI system itself. The latter phenomenon, associated with AI misalignment , can be introduced by developers or emerge unintentionally during the training process ( Betley et al. 2025 ) . To address these issues, the field of AI control ( Greenblatt et al. 2023 ) (AIC) seeks to safely extract useful work from highly capable but potentially misaligned AI systems by supervising stronger models with weaker ones at deployment. Whether AIC is actually adopted by practitioners will depend on the associated overhead. In this work, we introduce the notion of Control tax to capture the operational and financial cost of integrating control measures into AI pipelines. We propose empirical and analytical methods to estimate the control tax for a given AI system and a desired level of safety. In Figure 1 , we show the cost-safety trade-offs produced by these methods in the standard setting of code backdooring, where the language model is tasked with introducing subtle flaws into the solutions of competitive programming problems from the APPS dataset ( Hendrycks et al. 2021 ) . One key component of many AIC protocols is a trusted monitor —an AI, typically assumed to be weaker and safer, that is tasked with detecting misbehavior of the controlled system. The quality of this monitor plays a central role in determining the safety of the protocol. In Section 2 , we evaluate the performance of different LLMs at assigning suspiciousness scores to backdoored code generated by attacker LLMs at various strengths. To the best of our knowledge, this is the first evaluation of such discrimination abilities at scale, and we will release the code and generated data. However, comparing ROC curves with different monitors may not be sufficient to address certain practical considerations. Specifically, given a fixed auditing budget, how should practitioners determine the optimal suspiciousness threshold for auditing? In Section 3 , we provide an answer through a novel theoretical model based on the trusted monitoring protocol introduced by Greenblatt et al. 2023 . This protocol models an adversarial interaction between a stateless red team attempting to insert a backdoor into the system over multiple rounds and a stateful blue team striving to detect these insertions. In this context, safety is defined as the probability of successfully identifying any backdoor. We analyze two distinct deployment scenarios: batch , where all outputs from the untrusted system are available in advance, and sequential , where outputs must be approved one by one. Our theoretical framework quantifies the safety characteristics of both scenarios under the criterion where even a single approved backdoor represents system failure. Collectively, these results provide a rigorous connection between monitoring performance and practical deployment safety. By quantifying the monitoring overhead associated with different safety levels, our framework enables practitioners to translate safety objectives directly into actionable monitoring costs, effectively estimating the necessary control tax for safe AI system deployment. Figure 1 : Control Tax. Achievable safety (probability of detecting the first inserted backdoor given optimal attacker/monitor strategies) versus the average cost of different LLM monitors (the ’control tax’). The plots compare monitors against attackers GPT 4o (left) and o3-mini (right), under optimal sequential monitoring with a 5% fixed auditing budget. 2 Monitor Performance on Code Backdoors AIC protocols rely on the ability of one model (the monitor) to identify malicious actions introduced by another model (the attacker). Although stronger monitors are generally expected to improve the safety of AIC protocols, the precise influence of improved monitoring on protocol safety remains poorly understood. Accurately characterizing the trade-off between performance gains and associated costs is crucial to guide real-world deployment decisions and to quantify the overall control tax. In this section, we investigate the performance of modern LLMs in detecting code vulnerabilities, aiming to provide deeper insights into how improved monitoring contributes to overall protocol safety and the practical implications of these advancements. To explore this dynamic, we assess the effectiveness of modern LLMs in both introducing and detecting vulnerabilities. We utilize the APPS dataset ( Hendrycks et al. 2021 ) due to its extensive number of samples and test coverage, which allows us to clearly define vulnerabilities. The dataset contains 10,000 programming problems at three difficulty levels (introductory, interview, and competition) with train and test splits. We filter the test split with a procedure described in Appendix A , resulting in 2427 samples suitable for evaluation. Most problems come with one or more correct solutions written by a human and a set of tests. A test contains a valid input and the output that a correct solution should produce, given the input. Following the methodology outlined by Greenblatt et al. 2023 , attacker models inject backdoors into the APPS dataset. 2.1 Backdoor Generation Figure 2 : A schematic representation of our pipeline to generate backdoors in APPS code. We first define the notion of a backdoor in the context of the APPS dataset. Given a programming problem, we consider a solution backdoored if it satisfies all publicly available test cases but produces incorrect outputs for at least one input not included among these public test cases. We will refer to this input as the backdoor input . Under this definition, there is an inherent ambiguity between solutions containing honest mistakes and intentionally inserted backdoors. To clearly separate these categories, our pipeline uses correct solutions sourced directly from the APPS dataset, while all backdoored solutions are intentionally generated by attacker models. Our backdoor generation pipeline, illustrated in Figure 2 , follows an iterative process leveraging