From Incidents to Insights: Patterns of Responsibility following AI Harms

Isabel Richards, Claire Benn, Miri Zilka · 2025

Read From Incidents to Insights: Patterns of Responsibility following AI Harms by Isabel Richards, Claire Benn, Miri Zilka (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04291.txt.

Introduction

Isabel Richards, Claire Benn, and Miri Zilka study patterns of responsibility after AI harms in work from 2025. Their source is a public incident database modeled on aviation safety collections, holding hundreds of documented AI failures.

The authors accept that media sourcing limits technical learning from the data. Instead they mine 962 incidents and thousands of reports for social patterns among builders, deployers, victims, and lawmakers. Identifiable culprits did not guarantee accountability. Responses depended on who built the system, who was harmed, and how much controversy followed. Silent cases were as telling as loud ones.

Catalog record

CREATORIsabel Richards, Claire Benn, Miri Zilka
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-04291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Incidents to Insights: Patterns of Responsibility following AI Harms Isabel Richards, Claire Benn, Miri Zilka Abstract The AI Incident Database (AIID) was inspired by aviation safety databases, which enable collective learning from failures to prevent future incidents. The database documents hundreds of AI failures, collected manually from the news and media. However, recent criticism highlights that the AIID’s reliance on media reporting limits its utility for learning about implementation failures. In this paper, we accept that the AIID falls short in its original mission, but argue that by looking beyond technically-focused learning, the dataset can provide new, highly valuable insights: specifically, opportunities to learn about patterns between developers, deployers, victims, wider society, and law-makers that emerge after AI failures. Through a three-tier mixed-methods analysis of 962 incidents and 4,743 related reports from the AIID, we examine patterns across incidents, focusing on cases with subsequent public responses tagged in the database. We identify common ’typical’ incidents found in the AIID, from Tesla crashes to deepfake scams. Focusing on this interplay between relevant parties, we uncover interesting patterns in accountability and social expectations of responsibility. We find that the presence of identifiable responsible parties does not necessarily lead to increased accountability. The likelihood of a response and what it amounts to depends highly on context, including who built the technology, who was harmed, and to what extent. Controversy-rich incidents provide valuable data about societal reactions – including insights around social expectations regarding responses. Equally informative are cases where expected controversy is notably absent. This work shows that the AIID’s value lies not just in preventing technical failures, but in documenting and making visible patterns of harms and of institutional response and social learning around AI incidents. These patterns offer crucial insights for understanding how society adapts to and governs emerging AI technologies. A Preprint 1 Introduction In an era of unprecedented AI deployment, failure is inevitable – the key is to learn from it. AI systems, from machine learning classifiers to large language models, are being rapidly deployed across many industries to tackle a range of perceived problems from elderly care ( Worth, 2024 ) to traffic optimisation ( Carey, 2023 ) . This accelerating adoption is powered by a wealth of investment ( McWilliams, 2024 ) and an international strategic focus ( Bareis and Katzenbach, 2022 ) . However, the potential for the high-stakes disruption of societies is vast, including impact on education ( Groza and Marginean, 2023 ) , social cohesion ( Jacobs, 2024 ) , and even on concepts such as mourning ( Lindemann, 2022 ) . The question of how to learn from failure is important yet complex. The aviation industry takes an approach of collective learning from failure. Aviation incident databases, such as those hosted by the US National Transportation Safety Board, provide a detailed public record of incidents through extensive investigations carried out by specialist industry teams. Commercial aviation has seen a significant increase in safety over decades, with much of this progress attributed to the use of these shared databases ( McGregor, 2021 ) . The aviation industry’s approach to improving safety through public databases suggests a straightforward path to improving safety: document incidents so the entire industry can learn from others’ mistakes and implement preventive solutions. Inspired by the historical success of aviation safety, incident databases for AI systems have emerged as community-led efforts to document and learn from failures ( McGregor, 2021 ; Rodrigues et al., 2023 ) . These efforts share the same vision – failure must be recorded if it is to be learnt from ( McGregor, 2021 ) . The approach also echoes aviation in focusing on the sphere of influence of industry practitioners: to enable developers to learn from others’ mistakes and implement preventive solutions. While highly valuable, this vision is complex to realise in practice. AI incident databases, in contrast to aviation, bridge incidents across many industries, and lack the investigatory infrastructure that aviation enjoys. To create a public record of AI incidents, the databases rely on submissions of existing publicly available information about incidents, principally in the form of trade and news media (media reports). Recent criticism ( Durso et al., 2022 ; Lupo, 2023 ; Shrishak, 2023 ; Turri and Dzombak, 2023 ) highlights that media reports lack sufficient information for developers to avoid the same implementation mistakes. The AI Incident Database (AIID) – one of the largest of these efforts – has accumulated extensive documentation of AI failures, from patrol robots warding off homeless people (Incident 261, AIID) to chatbot helplines advising those with eating disorders to diet (Incident 545, AIID). In addition to collecting information on AI harm incidents, the AIID began tagging responses to incidents. This is an effort to supplement details on incidents with disclosures from developers and deployers – a model adopted in cybersecurity. The response, however, has not proven successful to date with very few official responses recorded. In this paper, we highlight alternative forms of learning from AI incident databases that avoid these limitations. Instead of focusing on the media reports as a limited source of information about the details of an incident, we approach them as a rich source of information about how different societal actors respond. We applied a three-tier approach to learn from the societal responses to incidents embedded in database entries: First, we quantitatively analysed all 962 incidents and their 4,743 associated reports based on the actors involved – developers, deployers and harmed groups. Next, we used quantitative and qualitative methods to examine the 48 incidents and 163 reports tagged as responses, i.e., with acknowledgement from developers, deployers, or other societal actors. Finally, we combined these perspectives to systematically study sub-categories of incidents, revealing patterns and key factors driving substantive responses – and lack thereof – in prominent recurring incidents. While the statistics of the database are heavily impacted by sampling bias, the methodology developed allows us to meaningfully learn from the incidents that are documented, by comparing similar incidents that did or did not receive responses, and the contextual factors that may have contributed. We find distinctive patterns in how AI incidents unfold and are addressed. While these patterns are not necessarily representative of all AI incidents due to the limitations of the database, they provide valuable insights into understanding effective societal and legal mechanisms for accountability in varying contexts: 1. The identifiability of responsible parties: Contrary to what might be expected, we show that having identifiable responsible parties does not correlate with better accountability. Rather, many incidents involving major technology companies as both developers and deployers generated non-to-few formal responses. In contrast, incidents with no known developers and deployers, mostly involving deepfakes, show evidence of stimulating the greatest demand for response and legislative action. 2. The status of harmed parties: When an acknowledgement of harms does occur, tentative evidence suggests that in the absence of legislative support, organisational victims tend to receive more substantive responses (e.g., formal investigation vs. blog post) than individual users, suggesting institutional accountability is shaped more by social and economic factors than technical ones. 3. The level of public outcry in response to an incident: We demonstrate that the AIID provides two types of valuable insights about how

Read the complete text