Getting in Contract with Large Language Models – An Agency Theory Perspective on Large Language Model Alignment

Sascha Kaltenpoth, Oliver Müller · 2025

Read Getting in Contract with Large Language Models – An Agency Theory Perspective on Large Language Model Alignment by Sascha Kaltenpoth, Oliver Müller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07642.txt.

Introduction

Sascha Kaltenpoth and 1 co-author view language model alignment through agency theory in a 2025 paper. Firms that adopt large language models can get off-topic, biased, or harmful output. The authors frame the problem as a contract issue. The adopter, the principal, cannot see inside the black-box agent. So errors made during adoption go unnoticed. Alignment research has rarely treated that imbalance of information, or the adoption process itself.

Their framework is called LLM ATLAS, an agency theory-led alignment strategy. It is aimed at the phases of adoption inside an organization. A conceptual review of the literature is organized by adoption phase and by agency concepts. It yields an extended analysis process for alignment methods, and a first map from alignment problems to solutions. The aim is practical. Firms should spot and contract around alignment risks before deployment, not after an incident.

Catalog record

CREATORSascha Kaltenpoth, Oliver Müller
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07642

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Getting in Contract with Large Language Models – An Agency Theory Perspective on Large Language Model Alignment Sascha Kaltenpoth, Oliver Müller Abstract Adopting Large language models (LLMs) in organizations potentially revolutionizes our lives and work. However, they can generate off-topic, discriminating, or harmful content. This AI alignment problem often stems from misspecifications during the LLM adoption, unnoticed by the principal due to the LLM’s black-box nature. While various research disciplines investigated AI alignment, they neither address the information asymmetries between organizational adopters and black-box LLM agents nor consider organizational AI adoption processes. Therefore, we propose LLM ATLAS (LLM A gency T heory- L ed A lignment S trategy) a conceptual framework grounded in agency (contract) theory, to mitigate alignment problems during organizational LLM adoption. We conduct a conceptual literature analysis using the organizational LLM adoption phases and the agency theory as concepts. Our approach results in (1) providing an extended literature analysis process specific to AI alignment methods during organizational LLM adoption and (2) providing a first LLM alignment problem-solution space. Keywords: Large Language Models, Organizational LLM Adoption, LLM Alignment, Agency Theory Research Paper 1 Introduction Adopting highly capable Large language models (LLMs) such as Llama 3 ( Meta 2024 ) or GPT-4 ( OpenAI 2023 ) in organizational processes, e.g., as domain-specific conversational agents, has the potential to revolutionize our lives and work ( Feuerriegel et al. 2023 ) . However, they can generate off-topic, discriminating, or harmful content under certain circumstances ( Teubner et al. 2023 ; To ​​​​​​​uvron et al. 2023 ) . In this case, the artificial intelligence (AI) agent’s actual behavior differs from that intended by its principal, e.g., the user or designer. This AI alignment problem ( Gabriel 2020 ; Hadfield-Menell & Hadfield 2019 ) often stems from misspecifications during the LLM adoption (e.g., implementation of LLM-driven systems into the organizational infrastructure) ( Shen et al. 2023 ; Wang et al. 2023 ) , unnoticed by the principal due to the LLM’s black-box nature. This information asymmetry between the adopting organization and the LLM agent possibly results in serious consequences, such as lawyers facing sanctions for citing fake cases ( Bohannon 2023 ) or failures in chatbot services ( Moench 2024 ) , for which users usually blame the company rather than the LLM-based system itself ( Pavone et al. 2022 ) . Therefore, it is crucial for organizations to mitigate information asymmetries during LLM adoption and ensure that these models are aligned with human, organizational, and social values ( Askell et al. 2021 ; Kenton et al. 2021 ) . Various research disciplines investigated AI alignment, particularly LLM alignment. Machine learning (ML) researchers focused on technical solutions, such as high-quality demonstration training data ( Ouyang et al. 2022 ; Zhou et al. 2023 ) , incorporating external knowledge ( Lewis et al. 2020 ; Zhang et al. 2024 ) , output verification ( Li et al. 2023 ; OpenAI 2023 ) , or alignment benchmarks ( Zha et al. 2023 ; Chang et al. 2024 ) . AI ethics researchers analyzed possible harms and provide governance recommendations ( Henderson et al. 2018 ; Weidinger et al. 2021 ; Mökander et al. 2023 ) . Furthermore, multiple literature reviews explained, categorized, and assessed LLM alignment approaches ( Ji et al. 2024 ; Shen et al. 2023 ; Wang et al. 2023 ) . While these studies provide some guidance for organizations that strive to adopt LLMs, they neither address the information asymmetries between the organizational adopter and the black-box LLM agent nor do they explicitly consider organizational AI adoption processes. Like most AI-based approaches, LLMs’ organizational adoption requires selection, training, evaluation, deployment, and monitoring of the model ( Kreuzberger et al. 2023 ; Symeonidis et al. 2022 ) . Therefore, we propose LLM ATLAS (LLM A gency T heory- L ed A lignment S trategy) a conceptual framework grounded in agency (contract) theory, to mitigate alignment problems during organizational LLM adoption. Highlighting the information asymmetry between the adopting organization and the black-box LLM, we adapt the idea of Hadfield-Menell & Hadfield 2019 to frame LLM alignment as an agency problem. We conduct a conceptual literature analysis ( Schryen 2015 ) using the combination of the organizational LLM adoption phases and the agency theory to analyze the literature. While we draw on existing AI and LLM alignment literature reviews instead of search and assessment, we synthesize a problem-solution mapping for LLM alignment during the organizational LLM adoption by applying agency theory to all organizational LLM adoption phases. LLM ATLAS provides a simple approach to identifying agency problems in the organizational LLM adoption process and synthesizing solutions from the literature. Our approach extends the existing AI alignment literature and supports organizational LLM adoption by (1) providing an extended literature analysis process specific to AI alignment methods during organizational LLM adoption and (2) providing a first proof-of-concept of the problem-solution space derivation. In future research, we plan to extend this initial literature analysis with a comprehensive literature review and an extendable format for the presentation of the problem-solution space in the form of a multi-contributor website. We hope to encourage other researchers to participate in the research on the intersection of LLM alignment and organizational LLM adoption. 2 Background 2.1 Large Language Models - A black-box Architecture Large language models (LLMs) such as GPT-4 or LLaMA, are neural networks, usually based on a decoder-only Transformer structure ( OpenAI 2023 ; To ​​​​​​​uvron et al. 2023 ) , that autoregressively predict the next token t_{n+1} based on the conditional probability distribution P(t_{n+1}|t_{1}...t_{n}) , where t_{1}...t_{n} refers to the context sequence of tokens ( Shanahan et al. 2023 ) . They contain up to billions of parameters and are (pre-)trained on vast amounts of textual data ( Bommasani et al. 2022 ; Shanahan et al. 2023 ) . These characteristics enable them to be pre-trained to solve many cognitive tasks, such as answering questions, generating coherent, meaningful texts, and passing various human exams at a human-like level ( Teubner et al. 2023 ; Chang et al. 2024 ) . Furthermore, pre-trained LLMs possess foundational knowledge and can be adapted to various downstream tasks and domains ( Bommasani et al. 2022 ) . Therefore, they are often called foundation models. However, foundation models are black-box architectures in terms of their neural network architecture and their vast amount of training data ( Bommasani et al. 2022 ) . Consequently, the learned representations and their alignment to human, organizational, and social values are not visible to a potential adopter ( Askell et al. 2021 ) , causing an information asymmetry between the LLM and its adopter. 2.2 The AI and LLM Alignment Problem The AI alignment problem refers to situations where an AI’s observed behavior differs from that intended by its designers or expected by its users ( Gabriel 2020 ; Hadfield-Menell & Hadfield 2019 ) . This problem typically originates from information asymmetries between the AI (agent) and its designers and users (principal) ( Hadfield-Menell & Hadfield 2019 ) . A main alignment concern in the context of LLMs is hallucinations, that is, factually incorrect or unfaithful responses ( Ji et al. 2023 ) . Another common concern is social, gender-,

Read the complete text