The Two Boundaries: Why Behavioral AI Governance Fails Structurally

Alan L. McCann · 2026

Read The Two Boundaries: Why Behavioral AI Governance Fails Structurally by Alan L. McCann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27292.txt.

Introduction

McCann separates governance of effects from governance of model outputs. Effects mean actions like API calls, database writes, and tool invocations, not output concerns like content quality and bias. Any system performing effects is bounded twice: by what it can do, and by what governance covers. Yet in nearly all deployed AI systems those boundaries are set independently. Independence creates three regions, and two of them are failure modes. Governed capabilities, the only useful region, face ungoverned capabilities, a risk, and policies aimed at capabilities that do not exist, theater.

Catalog record

CREATORAlan L. McCann
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47808
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.27292

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Two Boundaries: Why Behavioral AI Governance Fails Structurally Alan L. McCann Abstract Every system that performs effects has two boundaries: what it can do (expressiveness) and what governance covers (governance). In nearly all deployed AI systems, these boundaries are defined independently, creating three regions: governed capabilities (the only useful region), ungoverned capabilities (risk), and governance policies that address non-existent capabilities (theater). Two of the three regions are failure modes. We focus on the governance of effects : actions that AI systems perform in the world (API calls, database writes, tool invocations). This is distinct from the governance of model outputs (content quality, bias, fairness), which operates at a different level and requires different mechanisms. We present a formal framework for analyzing this structural gap. Rice’s theorem (1953) proves the gap is undecidable in the general case for any Turing-complete architecture that attempts to govern effects behaviorally: no algorithm can decide non-trivial semantic properties of arbitrary programs, including the property “this program’s effects comply with the governance policy.” We define coterminous governance : a system property where the expressiveness boundary equals the governance boundary. We show that coterminous governance requires an architectural decision (separating computation from effect) rather than a governance layer added after the fact. We show that structural governance under this separation subsumes separate governance infrastructure: governance checks become part of the execution pipeline rather than a second system running alongside it. We propose coterminous governance as the testable criterion for any AI governance system: either the two boundaries are provably identical, or risk and theater are structurally inevitable. The formal proofs underlying these claims are mechanized in Coq (454 theorems across 36 modules, zero admitted lemmas) and reported in a companion paper McCann 2026c . 1 Introduction Computing has undergone a shift that most governance frameworks have not absorbed. For sixty years, the central question in software was correctness: does this program compute the right answer? The relationship between software and the world was mediated by humans. A program computed a result. A person decided what to do with it. That relationship has changed. Modern AI systems observe, reason, decide, and act. They send emails, execute trades, modify databases, call APIs, and compose these actions in sequences they determine at runtime. The defining characteristic of contemporary AI is not intelligence in the abstract. It is agency : the capacity to perform effects in the world without human mediation at each step. Agency introduces a new question. Not “did it compute correctly?” but “should it have done that?” The first question is about mathematics. The second is about governance: who decides what an autonomous system may do, and how is that decision enforced? The governance question is not new. Biological organisms, human institutions, and legal systems have addressed it for millennia. What is new is the combination of two properties: the speed at which AI systems act (milliseconds, not deliberation cycles) and the generality of their action space (any API, any tool, any sequence of effects). This combination makes governance both more urgent and more difficult than in any prior computing paradigm. This paper presents a structural analysis of AI effect governance: the problem of ensuring that every action an AI system performs in the world (API calls, database writes, tool invocations, message sends) passes through a governance boundary. This is distinct from model-output governance (shaping what an LLM says via RLHF, content filtering, or Constitutional AI), which operates at a different level and addresses different properties. Both levels matter. This paper addresses the effect level. An important caveat at the outset: structural governance guarantees that a system enforces its policy with mathematical certainty. It says nothing about whether the policy itself is correct. A structurally governed system with a bad policy faithfully executes the bad policy. Policy correctness requires human judgment. More broadly, this paper analyzes the technical mechanism of governance enforcement. The social, institutional, and political dimensions of AI governance, including accountability, contestability, legitimacy, and democratic participation, are essential but outside our scope. Structural governance provides a mechanism; the question of what policies that mechanism should enforce requires different methods entirely. Structural governance is a necessary but not sufficient condition for good governance. The argument proceeds as follows: 1. The two-boundary model (Section 2 ): every effectful system has an expressiveness boundary and a governance boundary, and these boundaries are almost never identical. 2. The impossibility (Section 3 ): Rice’s theorem proves the gap is undecidable in the general case for Turing-complete systems that attempt to govern effects behaviorally. 3. The structural alternative (Section 4 ): separating computation from effect at the architectural level. 4. The limits of behavioral approaches (Section 5 ): why content filters, RLHF, and Constitutional AI cannot close the effect-governance gap. 5. The limits of monitoring (Section 6 ): why observability is not governance. 6. The efficiency consequence (Section 7 ): structural governance subsumes separate governance infrastructure. 2 The Two-Boundary Model Every system that produces effects in the world has two boundaries. The expressiveness boundary is determined by architecture: the programming language, the available APIs, the hardware interfaces, the external services the system can reach. It defines the set of all effects the system can, in principle, produce. This boundary is a fact about the system’s construction, not about its policies. The governance boundary is determined by policy: the rules written, the checks implemented, the monitoring deployed. It defines the set of all effects that governance covers. This boundary is a fact about the system’s oversight mechanisms, not about its capabilities. In an ideal system, these boundaries would be identical. Every capability would be governed. Every governance policy would correspond to a real capability. There would be no gap. In practice, the boundaries are defined by different mechanisms, maintained by different teams, and evolved on different timelines. They diverge. The divergence creates three regions (Figure 1 ): 1. Governed capability (the overlap): effects that the system can produce and that governance covers. This is the only region that functions correctly. 2. Ungoverned capability (the risk region): effects that the system can produce but that governance does not cover. These are the effects that cause incidents: data exfiltration through an unmonitored API, agent actions via an ungoverned tool, prompt injection exploiting a path that no filter anticipated. 3. Governance theater (the theater region): governance policies that address capabilities the system does not possess. Compliance checklists for features that were never built. Content filters for output modalities the system cannot produce. Audit trails for actions the system cannot take. A note on the theater region: policies for non-existent capabilities may serve legitimate organizational functions. They can signal values, satisfy regulatory requirements, or prepare for anticipated capabilities. But for effect governance specifically, a policy that does not correspond to a real capability cannot prevent harm at the technical level. It governs nothing. The concern is not that such policies exist, but that they can create a false sense of coverage, obscuring the ungoverned capabilities where actual risk resides. A concrete example. Consider an AI agent with access to an email-sending tool, a database-query tool, and a web-browsing tool. The agent is governed by a content filter that checks outgoing emails for sensitive information. The agent

Read the complete text