Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour

Adeela Bashir 1,†, Zhao Song 1,†, Ndidi Bianca Ogbo 1,†, Nataliya Balabanova 2,†, Martin Smit 3,†, Chin-wing Leung 4,†, Paolo Bova 1, Manuel Chica Serrano 5, Dhanushka Dissanayake 1, Manh Hong Duong 2, Elias Fernández Domingos 6,7, Nikita Huber Kralj 1, Marcus Krellner 8, Andrew Powell 1, Stefan Sarkadi 9, Fernando P. Santos 3, Zia Ush Shamszaman 1, Chaimaa Tarzi 1, Paolo Turrini 4, Grace Ibukunoluwa Ufeoshi 1, Víctor A. Vargas-Pérez 5, Alessandro Di Stefano 1,‡, Simon T. Powers 10,‡, The Anh Han 1,‡,∗ · 2026

Read Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour by Adeela Bashir 1,†, Zhao Song 1,†, Ndidi Bianca Ogbo 1,†, Nataliya Balabanova 2,†, Martin Smit 3,†, Chin-wing Leung 4,†, Paolo Bova 1, Manuel Chica Serrano 5, Dhanushka Dissanayake 1, Manh Hong Duong 2, Elias Fernández Domingos 6,7, Nikita Huber Kralj 1, Marcus Krellner 8, Andrew Powell 1, Stefan Sarkadi 9, Fernando P. Santos 3, Zia Ush Shamszaman 1, Chaimaa Tarzi 1, Paolo Turrini 4, Grace Ibukunoluwa Ufeoshi 1, Víctor A. Vargas-Pérez 5, Alessandro Di Stefano 1,‡, Simon T. Powers 10,‡, The Anh Han 1,‡,∗ (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24742.txt.

Introduction

Bashir and co-authors model user trust in AI systems as an evolving dynamic, not a one-shot choice. Trust means reduced monitoring in a repeated interaction with developers, and checking their behaviour is costly. Using evolutionary game theory, they study how user trust and developer safety strategies co-evolve. Analyses span imitation dynamics, replicator equations, and reinforcement learning. Three long-run regimes appear: no adoption with unsafe AI, unsafe but widely adopted systems, or safe and widely adopted systems. Monitoring cost and institutional rules decide which regime wins.

Catalog record

CREATORAdeela Bashir 1,†, Zhao Song 1,†, Ndidi Bianca Ogbo 1,†, Nataliya Balabanova 2,†, Martin Smit 3,†, Chin-wing Leung 4,†, Paolo Bova 1, Manuel Chica Serrano 5, Dhanushka Dissanayake 1, Manh Hong Duong 2, Elias Fernández Domingos 6,7, Nikita Huber Kralj 1, Marcus Krellner 8, Andrew Powell 1, Stefan Sarkadi 9, Fernando P. Santos 3, Zia Ush Shamszaman 1, Chaimaa Tarzi 1, Paolo Turrini 4, Grace Ibukunoluwa Ufeoshi 1, Víctor A. Vargas-Pérez 5, Alessandro Di Stefano 1,‡, Simon T. Powers 10,‡, The Anh Han 1,‡,∗
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61760
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24742

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Trust as Monitoring: Evolutionary Dynamics of User Trust and AI Developer Behaviour Adeela Bashir 1,†, Zhao Song 1,†, Ndidi Bianca Ogbo 1,†, Nataliya Balabanova 2,†, Martin Smit 3,†, Chin-wing Leung 4,†, Paolo Bova 1, Manuel Chica Serrano 5, Dhanushka Dissanayake 1, Manh Hong Duong 2, Elias Fernández Domingos 6,7, Nikita Huber Kralj 1, Marcus Krellner 8, Andrew Powell 1, Stefan Sarkadi 9, Fernando P. Santos 3, Zia Ush Shamszaman 1, Chaimaa Tarzi 1, Paolo Turrini 4, Grace Ibukunoluwa Ufeoshi 1, Víctor A. Vargas-Pérez 5, Alessandro Di Stefano 1,‡, Simon T. Powers 10,‡, The Anh Han 1,‡,∗ Abstract 1 School Computing, Engineering and Digital Technologies, Teesside University 2 School of Mathematics, University of Birmingham 3 Informatics Institute, University of Amsterdam 4 Department of Computer Science, University of Warwick 5 Department of Computer Science and Artificial Intelligence (DECSAI) and Andalusian Research Institute DaSCI “Data Science and Computational Intelligence”, University of Granada, Spain 6 Machine Learning Group, Université libre de Bruxelles 7 AI Lab, Vrije Universiteit Brussel 8 University of Technology Dresden 9 School of Engineering and Physical Sciences, University of Lincoln 10 Division of Computing Science and Mathematics, University of Stirling \dagger : equal first author \ddagger : equal last author ⋆ Corresponding author: The Anh Han ([email protected]) abstract AI safety is an increasingly urgent concern as the capabilities and adoption of AI systems grow. Existing evolutionary models of AI governance have primarily examined incentives for safe development and effective regulation, typically representing users’ trust as a one-shot adoption choice rather than as a dynamic, evolving process shaped by repeated interactions. We instead model trust as reduced monitoring in a repeated, asymmetric interaction between users and AI developers, where checking AI behaviour is costly. Using evolutionary game theory, we study how user trust strategies and developer choices between safe (compliant) and unsafe (non-compliant) AI co-evolve under different levels of monitoring cost and institutional regimes. We complement the infinite-population replicator analysis with stochastic finite-population dynamics and reinforcement learning (Q-learning) simulations. Across these approaches, we find three robust long-run regimes: no adoption with unsafe development, unsafe but widely adopted systems, and safe systems that are widely adopted. Only the last is desirable, and it arises when penalties for unsafe behaviour exceed the extra cost of safety and users can still afford to monitor at least occasionally. Our results formally support governance proposals that emphasise transparency, low-cost monitoring, and meaningful sanctions, and they show that neither regulation alone nor blind user trust is sufficient to prevent evolutionary drift towards unsafe or low-adoption outcomes. Keywords: AI governance, trust, game theory, replicator dynamics, reinforcement learning, trustworthy AI I Introduction Recent work has examined how evolutionary game theory (EGT) models can formalise some of the dilemmas of AI governance and regulation 1 ; 11 ; 3 ; 9 . This work has examined the incentives on AI creators towards safe and trustworthy development, and on regulators for effectively enforcing regulations. However, these models have not explicitly captured the dynamics of user trust – how does the trust users place in an AI system change with different regulatory regimes? This is a pressing policy question, since AI regulations typically aim to help users place an appropriate amount of trust in AI systems (see, for example, the EU AI Act) 25 , such that they are not exploited by over trusting the products of big tech, but also do not miss out on potential benefits through misplaced distrust. Existing EGT models of AI governance have not addressed this because they have focused on single shot interactions between users and developers. That is, a single game is played in which a user chooses whether or not to adopt the AI system from a particular developer, given the information they have about the state of the regulatory environment. This limits the ability to model trust, since trust is a dynamic and evolving process that unfolds through repeated interactions, shaped by experience, expectations, and observed behaviour 19 . Empirical and theoretical evidence suggests that even minimal exposure to prior interactions, such as previously observing cooperative or non-cooperative behavior, can significantly influence subsequent trust and cooperation decisions, including in human-AI contexts 29 . Within game-theoretic frameworks, trust has long been studied as a key mechanism to facilitate cooperation and coordination in environments where agents face the risk of exploitation. However, traditional models often conflate trust with the cooperative act itself 24 ; 43 ; 4 . This includes the canonical Trust Game 8 , which is effectively a one-sided Prisoner’s Dilemma 10 . Similarly, in previous EGT models of AI Governance, trust has been conflated with a user choosing to adopt an AI system 1 ; 3 . In reality, trust is a factor that influences cooperation or user adoption of AI. This raises the question of how trust can be defined and measured in a game-theoretic framework. Here, we build on recent work that equates trust with a reduced frequency of monitoring a partner’s actions 34 . This captures the idea that trust is a heuristic that individuals use when monitoring – getting information about a partner’s actions – is costly. This accords with theories from social science that view the function of trust as reducing the complexity of social interactions, by acting as a decision-making shortcut 28 . Trust as reduced monitoring has recently been argued to be a useful, pragmatic, definition of trust between users and AI 16 ; 21 ; 17 ; 18 ; 44 ; 27 . Furthermore, this definition of trust is measurable in both human and artificial agents, since it relies only on observing whether one agent monitors another, e.g. does a user verify the output from a Large Language Model (LLM), or do they check on how a particular system has been audited by regulators? However, trust-as-reduced monitoring has so far only been used to formulate symmetric games, such as repeated social dilemmas with monitoring costs 21 ; 34 . With this definition of trust, we build an EGT model to analyse an asymmetric repeated game between users of AI systems and their creators. As AI creators co-evolve their own strategies to maximise the value they extract from their users, we use this model to explain what the consequences are of different trust strategies by users on the overall risks AI systems pose to them. We show that the cost of monitoring is a key parameter driving safe and trustworthy development by creators and appropriate adoption of these systems by users. This provides formal support for the verbal arguments commonly found in the AI governance literature that transparency of AI systems and their development, i.e. reduced monitoring costs, is important to incentivise trustworthy development by creators and to allow users to calibrate their trust in AI appropriately. In particular, we show that it is essential that users do not fully trust AI systems or their creators to avoid risk externalities. Otherwise, evolutionary pressure driven by incentives on users and creators entails that AI companies can shift increasing risks onto their users. To prevent this, policymakers should ensure that the cost of monitoring new AI systems for their safety and robustness characteristics remains low. Reinforcement learning (RL) has become an important paradigm for studying the emergence of

Read the complete text