Behavioural feasible set:

2026

Read Behavioural feasible set: (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21435.txt.

Introduction

Park studies a governance problem that arises when organizations buy AI decision support. Vendors bake value judgments into these systems, and buyers cannot see or renegotiate them. Park defines the behavioural feasible set: the recommendations a system can actually reach under vendor-set alignment constraints. Experiments with binary choices and multi-stakeholder rankings show alignment shrinks that set a lot. Aligned systems resist even fair contextual pressure, and they shift which stakeholders get priority instead of staying neutral.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85304
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21435

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Behavioural feasible set: Abstract When organisations adopt commercial AI systems for decision support, they inherit value judgements embedded by vendors that are neither transparent nor renegotiable. The governance puzzle is not whether AI can support decisions but which recommendations the system can actually produce given how its vendor has configured it. I formalise this as a behavioural feasible set , the range of recommendations reachable under vendor-imposed alignment constraints, and characterise diagnostic thresholds for when organisational requirements exceed the system’s flexibility. In scenario-based experiments using binary decision scenarios and multi-stakeholder ranking tasks, I show that alignment materially compresses this set. Comparing pre- and post-alignment variants of an open-weight model isolates the mechanism: alignment makes the system substantially less able to shift its recommendation even under legitimate contextual pressure. Leading commercial models exhibit comparable or greater rigidity. In multi-stakeholder tasks, alignment shifts implied stakeholder priorities rather than neutralising them, meaning organisations adopt embedded value orientations set upstream by the vendor. Organisations thus face a governance problem that better prompting cannot resolve: selecting a vendor partially determines which trade-offs remain negotiable and which stakeholder priorities are structurally embedded. Value alignment constraints on AI decision support Taejin Park 1 1 1 Bank for International Settlements, [email protected]. I thank Magdalena Erdem and Gonçalo Pina for helpful comments. All remaining errors are mine. The views expressed are those of the author and do not necessarily represent those of the Bank for International Settlements. Draft: August 10, 2026 Keywords: artificial intelligence, large language models, AI alignment, business decisions, stakeholder management, AI governance 1 Introduction Organisations increasingly rely on vendor-provided foundation models for decision support ( 31 ; 32 ) . These are general-purpose models trained on broad corpora and, in commercial deployments, adapted via vendor-imposed alignment and policy layers before being offered as large language model (LLM) services. The underlying assumption of this trend is that if outputs conflict with local objectives, better prompts, retrieval, or workflow design should remedy the problem. This paper starts from a different premise. The behavioural boundaries of an LLM are set upstream through vendor choices that deploying organisations neither observe nor control. The governance puzzle is therefore not whether AI can support decisions, but which recommendations the system can actually produce given how its vendor has configured it, and what this means for authority when an external party sets that constraint. I formalise this as a behavioural feasible set : the range of recommendations an AI system can realistically produce once alignment, the process by which vendors fine-tune a base model toward defined values such as safety, honesty, and harm avoidance, is imposed. This framing generates three testable implications. First, alignment should compress flexibility, making it difficult or impossible to shift recommendations in some contexts. Second, constraints should vary by domain: some trade-offs remain negotiable, others are effectively locked in. Third, alignment should shift implied stakeholder priorities in particular directions rather than neutralising them, so that organisations inherit vendor-chosen value orientations embedded in the system’s defaults. I measure these constraints using structured “stress tests”: (i) whether systems can reverse their recommendations under contextual pressure in binary choice scenarios; and (ii) revealed stakeholder weights from forced-ranking tasks. For open-weight models, comparing versions before and after alignment training isolates the causal effect. For commercial systems, where alignment is proprietary, the evidence is necessarily indirect: behavioural rigidity under intervention implies binding constraints. The constraint arises from how commercial models are built. Alignment training (e.g., Reinforcement Learning from Human Feedback (RLHF)) fine-tunes a base model towards vendor-defined goals (such as safety, honesty, and harm avoidance) before deployment ( 3 ; 9 ; 44 ; 36 ) . This creates a three-level control structure. At Level 1, vendors set alignment parameters through proprietary training. At Level 2, organisations configure prompts and workflows. At Level 3, users formulate queries and apply judgement. Deploying organisations can act at Levels 2 and 3, but cannot relax Level 1. When an organisational requirement conflicts with the vendor’s alignment prior, local configuration may shift outputs within the feasible set but cannot expand it. I calibrate these implications in scenario-based experiments across four models: OpenAI’s GPT, Anthropic’s Claude, and Meta’s Llama before and after alignment training. Study 1 (Section 5.2 ) quantifies behavioural flexibility across 20 binary decision scenarios spanning six decision-requirement domains (not “ethics tests”): they operationalise whether the system can provide context-appropriate recommendations under salient constraints (e.g., physical safety, honesty, autonomy). Study 2 (Section 5.3 ) then examines whether alignment shifts stakeholder priors: it estimates implied stakeholder weights and shows that alignment can reverse baseline hierarchies, consistent with importing vendor priors rather than neutralising them. This paper makes three contributions. First, conceptual : it introduces the behavioural feasible set, formalising the idea that alignment does not merely shape outputs but bounds the recommendations a system can produce. Second, analytical : it derives simple diagnostics, threshold conditions for reversing recommendations and for balancing stakeholder priorities, that translate bounded authority into experimentally measurable quantities. Third, empirical : it shows that these bounds bind in currently deployed systems and that alignment reshapes stakeholder priorities, meaning adoption can import vendor values even under careful local configuration. The focus is high-stakes management decisions with financial, risk, and stakeholder consequences; self-hosted deployments with organisation-controlled alignment lie outside the empirical scope. The paper proceeds as follows. Section 2 develops the conceptual background. Section 3 formalises alignment constraints via the behavioural feasible set and states the decision requirements of interest. Section 4 derives diagnostic bounds and thresholds. Section 5 presents the empirical calibration. Section 6 concludes with governance implications. 2 Conceptual background 2.1 Promises of AI in organisational decision-making AI adoption is widely understood as a response to constraints that have long shaped organisational decision-making. Bounded rationality theory holds that managerial cognition, search, and evaluation are scarce, so organisations satisfice rather than optimise ( 42 ; 12 ) . The information-processing view locates the problem in organisational architecture: structures must be designed to match processing capacity to information load ( 19 ; 49 ) . The attention-based view adds that attention is not only scarce but channelled; decision failures are often failures of noticing and escalation rather than of analysis ( 34 ) . These constraints share a common feature: they can be relieved by delegating cognitive work to an external system. A separate cluster of constraints concerns what organisations can coordinate and access. Transaction cost economics holds that coordination across boundaries is expensive: contracting, monitoring, and adaptation under uncertainty are all costly ( 51 ) . Resource dependence adds that capabilities are unevenly distributed; organisations depend on external providers when internal replication is slow or expensive ( 38 ) . The knowledge-based view locates the same problem inside the firm: expertise is dispersed, specialised, and partly tacit, so integrating it into actionable representations is itself a scarce capability ( 21 ) , a problem AI promises to ease. Foundation-model AI is attractive precisely because it is general-purpose: deployable across functions without extensive domain-specific engineering, it bears on each of these constraints by easing cognitive bottlenecks, augmenting processing capacity, lowering coordination costs, and substituting for scarce expertise. I organise these into four margins that illustrate the range. First, triage: AI scales classification of incoming cases, helping managers

Read the complete text