Guardrails versus Gatekeepers: Understanding Product Managers’ Ethical Decision-Making in Generative AI

Genevieve Smith, Natalia Luka, Merrick Osborne, Brian Lattimore, Jessica Newman, Brent Mittelstadt, Brandie Nonnecke · 2025

Read Guardrails versus Gatekeepers: Understanding Product Managers’ Ethical Decision-Making in Generative AI by Genevieve Smith, Natalia Luka, Merrick Osborne, Brian Lattimore, Jessica Newman, Brent Mittelstadt, Brandie Nonnecke (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16531.txt.

Introduction

Genevieve Smith and co-authors studied how product managers make ethical choices about generative AI in a 2025 paper. Product managers are often cast as gatekeepers of product teams, yet their role in responsible AI was unclear. Policies can drift apart from practice when incentives point the other way. The team drew on twenty five interviews and a global survey of more than three hundred people in product roles.

Uncertainty about what responsible AI requires, plus a sense that responsibility belongs to everyone and no one, held managers back. Leadership commitment and clear company principles freed them to act, making some good practices many times more likely. Managers fought the drift two ways: cheap solo actions they could take alone, and costly joint actions that need company backing. The authors argue that closing the policy to practice gap at team level is where responsible AI is won or lost.

Catalog record

CREATORGenevieve Smith, Natalia Luka, Merrick Osborne, Brian Lattimore, Jessica Newman, Brent Mittelstadt, Brandie Nonnecke
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16531

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Guardrails versus Gatekeepers: Understanding Product Managers’ Ethical Decision-Making in Generative AI Genevieve Smith, Natalia Luka, Merrick Osborne, Brian Lattimore, Jessica Newman, Brent Mittelstadt, Brandie Nonnecke Abstract . Abstract What is the role of product managers (PMs) in the responsible use of generative AI (genAI) in products and everyday work—and what enables or constrains their ability to take action? Past literature has examined the ways in which organizational policies can become decoupled from practices when incentives for responsible action are misaligned or impeded by profit motives. While the role of engineers and professional ethicists in the context of AI has been examined in detail, the role of product managers—who are frequently portrayed as “gatekeepers” or critical decision-makers in product teams—remains unclear, particularly regarding genAI. In this paper, we examine what organizational conditions promote responsible use of genAI by product managers by drawing on twenty-five interviews and a global survey of over three hundred respondents in product management-related roles. First, we find that uncertainty around responsible AI and a sense of diffused responsibility constrain ethical action, while leadership commitment and organizational principles enable ethical action—making some responsible practices up to fourteen times more likely. Further, our study finds two sets of actions product managers take to “recouple” ethical commitments and practices. The first set includes low-resource, individual actions product managers can implement without explicit organizational incentives (e.g., individual or team-wide reviews and safeguarding standards around data privacy). The second set includes high-resource, collective actions that require organizational incentives (e.g., conducting audits or delaying shipment of products). In contrast to existing findings in the literature, product managers view themselves as "guardrails" rather than “gatekeepers” when it comes to making ethics-related decisions for genAI use. Our research suggests that recoupling ethical policies and practices at the level of product teams requires institutional buy-in and higher level leadership commitment. Nevertheless, we show that individual actors are able to exhibit agency through some meaningful, low resource actions, even in the absence of organizational incentive structures even in the absence of organizational incentives, though this alone is insufficient to operationalize responsible AI at scale. 1 1 1 This is the author’s version of the work. This preprint is posted here for your personal use. The definitive Version of Record is published in the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada. \acmArticleType Research \acmContributions \conffull ( \confshort ), \confdate , \confloc Conference: The 2026 ACM Conference on Fairness, Accountability, and Transparency; June 25–28, 2026; Montreal, QC, Canada The 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’26), June 25–28, 2026, Montreal, QC, Canada DOI: 10.1145/3805689.3812395 ISBN: 979-8-4007-2596-8/2026/06 CCS: Computing methodologies Artificial intelligence CCS: Social and professional topics Socio-technical systems † † cc-license: by 1. Introduction As the adoption of AI has grown, so has the idea of “responsible AI” ( Jobin et al., 2019 ; Mittelstadt, 2019 ) . Responsible AI (sometimes used synonymously with “ethical AI”) refers to the development, deployment, and governance of AI technologies to align with human values and societal norms ( Deshpande and Sharp, 2022 ; Birhane et al., 2022 ) . Past studies have examined the phenomenon of “ethics washing” and the ways in which organizational policies can become decoupled from practices when incentives for responsible action are misaligned or impeded by profit motives ( Greene et al., 2019 ; Metcalf and Moss, 2019 ; Attard-Frost et al., 2023 ; Neff, 2020 ) . Practitioners, ranging from software engineers to professional ethicists, have struggled with a lack of structural power when questioning whether to uphold their ethics and values in the workplace ( Rakova et al., 2021 ; Nedzhvetskaya and Tan, 2022 ; Widder et al., 2023 ; Ali et al., 2023 ) . Product managers are the dominant form of mid-level manager within product-focused organizations, particularly within the software industry and software-focused divisions of companies ( Maglyas et al., 2013 ) . Existing studies have noted that product managers play a privileged role in the power structures of most tech companies, serving as gatekeepers, controlling most day-to-day decision-making in the product development life cycle. ( Rakova et al., 2021 ; Ali et al., 2023 ; Widder et al., 2023 ) Ali, et al. (2023) found that professional ethicists spent a significant amount of their time “negotiat[ing] with product managers for resources, theoriz[ing] ethics-oriented interventions as functionally superior or “pragmatically legitimate” by linking them to product quality, and advocat[ing] for early integration into the product development cycle”. Widder, et al. (2023) found that, among software engineers, product managers were viewed as “decision-makers” that had to first be convinced in order for certain ethical actions and interventions to be carried out ( Widder et al., 2023 ) . While product managers are viewed as powerful actors by employees in other roles, we know little about how they conceptualize their own agency and what factors drive them to take actions prioritizing ethics. This is particularly the case for generative AI (genAI), a form of AI that generates new outputs from existing data, including text, code, images, and patterns of behavior. ( Kenthapadi et al., 2023 ) What is the role of product managers in integrating, promoting, and challenging ethical use of genAI technologies in the workplace? What constrains or enables product managers in the types of actions they choose to take towards ethical use of genAI? In this paper, we examine the persistent research gap in understanding how responsibility is incorporated in organizational decision-making among product managers, and the conditions that constrain or support responsible development and use of generative AI (genAI). Drawing from our empirical data, we find two categories of actions product managers take to “recouple” ethical commitments and practices regarding genAI. The first category includes low resource, individual actions product managers can take without explicit organizational incentives, which include individual or team-wide reviews and safeguarding standards around data privacy. The second category includes high resource, collective actions that require organizational incentives, which include conducting audits, fairness/bias testing, or adversarial testing, such as red teaming. Our research highlights both the possibilities and limitations of such actions, suggesting that even in environments that are not conducive to higher-resource actions, there remain ways for employees to exhibit agency in promoting ethical genAI use. At the same time, many actions are realistically off-limits without broader organizational buy-in. We counter the existing conception of PMs as gatekeepers within the AI ethics and governance literature (individuals who determine critical decisions in the product development cycle) and suggest that the more realistic view is of product managers as guardrails (individuals who uphold organizational standards and principles set by higher-level leadership rather than exercising independent ethical authority). Rather than calling for reduced accountability for PMs, these findings call for greater organizational support to create the structures and conditions that make ethical robust action at the product team level genuinely possible. 2. Background Research on AI ethics and governance in organizations has documented the challenges of translating high-level ethical principles and policies into everyday organizational practice. This background section outlines prior work on organizational decoupling and recoupling in AI ethics, and situates product managers as critical, yet understudied, actors navigating gaps between ethical principles and practices. 2.1. Convergence on principle,

Read the complete text