Controllable Complementarity: Subjective Preferences in Human-AI Collaboration

Chase McDonald, Cleotilde Gonzalez · 2025

Read Controllable Complementarity: Subjective Preferences in Human-AI Collaboration by Chase McDonald, Cleotilde Gonzalez (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05455.txt.

Introduction

Teams of people and AI systems are usually judged by score alone, yet how the work feels to the human partner matters too. Chase McDonald and Cleotilde Gonzalez study that side of the partnership in this 2025 paper. They used a learning algorithm that lets people directly shape the AI partner's behavior in a shared workspace task.

In one test, with controls hidden, the shaping method produced AI policies as strong as self-taught ones. In a second test, people who could steer the AI rated it as more effective and more fun to work with. The authors argue that design should chase both task scores and human preference, since the two together define real teamwork.

Catalog record

CREATORChase McDonald, Cleotilde Gonzalez
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05455

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Controllable Complementarity: Subjective Preferences in Human-AI Collaboration Chase McDonald, Cleotilde Gonzalez Abstract Research on human-AI collaboration often prioritizes objective performance. However, understanding human subjective preferences is essential to improving human-AI complementarity and human experiences. We investigate human preferences for controllability in a shared workspace task with AI partners using Behavior Shaping (BS), a reinforcement learning algorithm that allows humans explicit control over AI behavior. In one experiment, we validate the robustness of BS in producing effective AI policies relative to self-play policies, when controls are hidden. In another experiment, we enable human control, showing that participants perceive AI partners as more effective and enjoyable when they can directly dictate AI behavior. Our findings highlight the need to design AI that prioritizes both task performance and subjective human preferences. By aligning AI behavior with human preferences, we demonstrate how human-AI complementarity can extend beyond objective outcomes to include subjective preferences. 1 1 1 Experiment ethical approval was received by the university’s Institutional Review Board. 1 Introduction The past several decades have seen a significant increase in research focusing on human-AI teams (HATs) and human-AI collaboration. Both refer to situations where there exists at least one human and an autonomous agent, where autonomous agents play a significant role and are treated as teammates O’Neill et al. 2022 . Conceptually, HATs have existed for several decades; since computers became commonplace, researchers have been interested in how they can be used to work alongside humans Nass et al. 1996 . Indeed, as AI systems have advanced, it has become increasingly clear that AI has usage beyond just that of a tool to support human activities; it can serve the role of a teammate. The focus in the HAT literature is typically on improving team performance, often through human-AI complementarity, where the combination of humans and AI exceeds the performance or capabilities of either in isolation Bansal et al. 2021 ; Steyvers and Kumar 2023 . The emphasis in this definition is on objective performance. However, this neglects other considerations, particularly those related to subjective human preferences. The stance that we take is that the scope of objective performance in human-AI interaction is too narrow and that it can be expanded to account for subjective preferences: human-AI complementarity emerges when subjective preferences are met alongside objective performance, enhancing human experiences beyond what would be possible without AI. It may not always be the case that humans seek only to reach some final outcome, but also seek a particular path to that outcome—the experience along the way is important Gandhi et al. 2024 . Prior work has pointed out the importance of accommodating human preferences and capabilities in building HATs Zhao et al. 2023a ; van den Bosch et al. 2019 , although there is a gap in the literature on the extent to which humans are will accept degradations in objective outcomes to improve their subjective utility. We can also consider this from an alternative perspective. Given a fixed objective level of performance, what factors contribute to the human subjective preferences of their HAT experience? How important are these relative to their objective performance? As AI technologies progress, it is not difficult to imagine scenarios in which HATs are worse than teams composed only of AI agents; in many settings this is already the case, with AI teams reaching performance that exceeds human performance in a number of tasks Silver et al. 2017 ; Brown and Sandholm 2018 ; Berner et al. 2019 ; Vinyals et al. 2019 . The incorporation of humans, particularly in low-stakes settings, may then be to satisfy some subjective preferences, rather than on optimal objective outcomes. Our primary focus in this work is on subjective preferences for the control of AI partners. Control has been reported to be a desired component of AI teammates Klien et al. 2004 ; Lim and Dey 2009 ; Myers and Morley 2001 . Although highlighted as an important component, control has not played a significant role in the existing HAT literature Tsamados et al. 2024 and the need for additional methods to enable control has been noted National Academies of Sciences, Engineering, and Medicine 2021 . Our aim is to address open questions in human-AI teaming by exploring the following research questions: • RQ1 : In shared workspace tasks, do humans have preferences for the controllability of their AI teammates? • RQ2 : How do humans balance subjective preferences and task performance? Furthermore, how can we account for subjective preferences to enhance human experiences in human-AI collaboration? We contribute to the machine learning literature by developing controllable AI policies to investigate these questions. Specifically, we use reinforcement learning Sutton and Barto 2018 as our baseline and augment the training regime to allow explicit human control over the learned policy through a method we refer to as Behavior Shaping (BS). Through a user study, we demonstrate the efficacy of BS in creating robust AI partners capable of collaborating effectively with humans, offering an improvement relative to the standard self-play baseline. Then, in a second study, we use BS to address the outlined research questions. By systematically manipulating participants’ ability to use and observe controls over AI teammate behavior, we examine how these manipulations influence subjective evaluations of the AI partner. 2 Related Work We first outline existing work relevant to the human-AI teaming literature and the development of collaborative AI. Human-AI Teaming. There has been significant effort to outline frameworks and desiderata for the characteristics of HATs, which is often based on the literature on human-human teams or human preferences in surveys Klien et al. 2004 ; Hauptman et al. 2023 ; Endsley 2017 ; McNeese et al. 2021 . A noteable amount of effort has also investigated appropriate or desired levels of autonomy for AI teammates Schermerhorn and Scheutz 2009 ; Ball and Callaghan 2012 ; Alan et al. 2014 ; Endsley 2017 ; Sundar 2020 ; Salikutluk et al. 2024 . Although there is a significant amount of research in the AI literature on how to build effective and adaptive AI teammates Carroll et al. 2019 ; Strouse et al. 2021b ; Zhao et al. 2023a ; Zhang et al. 2022 ; Yu et al. 2023 , there are limited empirical evaluations and validations of the aforementioned frameworks and desiderata in shared workspace settings Salikutluk et al. 2024 , where humans and AI act independently in an environment together. Therefore, further investigation and evaluation of human behavior and preferences is needed in the context of HATs. A commonly reported attribute of AI teammates in HATs is that they must be predictable , particularly when they have high levels of autonomy Hauptman et al. 2023 . Predictability arises from the need to coordinate behavior in collaborative settings and is critical in teams with any type of composition. In fact, predictability is closely related to theory of mind, in which teammates must be able to anticipate others’ behaviors to plan their actions accordingly Klien et al. 2004 ; Byom and Mutlu 2013 ; DeChurch and Mesmer-Magnus 2010 . Being able to anticipate what a teammate will do and take actions that complement anticipated behavior is critical for team performance. Existing work has demonstrated that developing

Read the complete text