The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms

Pooja S. B. Rao, Sanja Šćepanović, Dinesh Babu Jayagopi, Mauro Cherubini, Daniele Quercia · 2025

Read The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms by Pooja S. B. Rao, Sanja Šćepanović, Dinesh Babu Jayagopi, Mauro Cherubini, Daniele Quercia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-16672.txt.

Introduction

Pooja Rao and co-authors catalog AI risks in 2025 from the words of model builders themselves. They mine nearly half a million model cards from a public hub and extract about three thousand distinct risk mentions. The resulting catalog is then set against researcher-compiled risks and a database of real-world AI incidents.

Builders mostly flag technical issues like bias and safety, while researchers stress wider social harms. Both groups barely mention fraud and manipulation, though such harms are common in practice. The authors call for structured risk reporting that pushes builders to weigh human and system-level harms early in design.

Catalog record

CREATORPooja S. B. Rao, Sanja Šćepanović, Dinesh Babu Jayagopi, Mauro Cherubini, Daniele Quercia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-16672

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The AI Model Risk Catalog: What Developers and Researchers Miss About Real-World AI Harms Pooja S. B. Rao, Sanja Šćepanović, Dinesh Babu Jayagopi, Mauro Cherubini, Daniele Quercia Abstract We analyzed nearly 460,000 AI model cards from Hugging Face to examine how developers report risks. From these, we extracted around 3,000 unique risk mentions and built the AI Model Risk Catalog . We compared these with risks identified by researchers in the MIT Risk Repository and with real-world incidents from the AI Incident Database. Developers focused on technical issues like bias and safety, while researchers emphasized broader social impacts. Both groups paid little attention to fraud and manipulation, which are common harms arising from how people interact with AI. Our findings show the need for clearer, structured risk reporting that helps developers think about human-interaction and systemic risks early in the design process. The catalog and paper appendix are available at: https://social-dynamics.net/ai-risks/catalog . Introduction Risk and harm have been central concerns in AI safety and ethics research. Researchers have worked to identify and organize these concepts into taxonomies ( Weidinger et al. 2022 ; Yampolskiy 2016 ) . Some focus on Large Language Models and Generative AI ( Weidinger et al. 2021 ; Weidinger et al. 2022 ; Stahl and Eke 2024 ) , while others address broader AI systems ( Yampolskiy 2016 ; Wirtz, Weyerer, and Sturm 2020 ) or Artificial General Intelligence ( McLean et al. 2023 ) . These efforts often use different definitions of risk and harm. We follow the OECD ( Perset and Aranda 2024 ) , which defines risk as the chance of harm, and harm as a risk that has caused damage. In some contexts, the terms are used interchangeably. The MIT Risk Repository ( Slattery et al. 2024 ) compiles risks from academic work on AI frameworks and is the largest collection of researcher-identified AI risks to date. The AI Incident Database ( McGregor 2021 ) catalogs real-world harms, which have been further classified by Velázquez et al. 2024 . Existing research has not explored how developers describe the risks of specific models or how those models might fail in typical user scenarios. This perspective is critical, as developers can offer grounded insight into the behavior and limitations of the systems they build. Figure 1: Overview of our methodology. Our methodology has three steps: extracting, classifying, and consolidating risks from model cards to build the AI Model Risk Catalog. We then compared the developer-reported risks in the catalog to those identified by researchers in the MIT Risk Repository ( Slattery et al. 2024 ) and to real-world harms from the AI Incident Database ( McGregor 2021 ) . The catalog is based on 64,116 model cards with risk sections, from which we extracted 37,401 risk mentions (including duplicates). After consolidation, we identified 2,863 unique risks from 2,672 model cards, making this the largest collection of AI risks across the three sources. To address the lack of risk data linked to specific models and described by their developers, we analyzed nearly half a million model cards ( Mitchell et al. 2019 ) from Hugging Face as of July 2024. Model cards are a widely adopted standard for documenting AI models, adopted by both major technology companies and individual developers. This work makes two main contributions (Figure 1 ): 1. Analyzing risks identified by developers to create the AI Model Risk Catalog. We collected all available model cards on Hugging Face and using established taxonomies ( Weidinger et al. 2022 ; Slattery et al. 2024 ) , grouped similar risks within each category, and selected representative examples. The result is the AI Model Risk Catalog, which includes 2,863 categorized risks tied to specific models. The catalog is publicly available at: https://social-dynamics.net/ai-risks/catalog . 2. Comparing developer and researcher risks with real-world harms. We compared the types of risks emphasized in three sources: developer-described risks from model cards, researcher-identified risks from the MIT Risk Repository, and real-world harms from the AI Incident Database. Developers tend to report technical risks such as model limitations, safety issues, and bias. These account for over half the harms recorded in real-world incidents. Researchers focus on governance, societal impacts, and threats to human agency, which together explain fewer than 15% of incidents. Notably, the largest share of incidents involves malicious use and misinformation. These risks are underrepresented by both developers and researchers, likely because they depend on unpredictable human behavior: hard for researchers to anticipate, and even harder for developers to account for ( Velázquez et al. 2024 ) . We close by discussing how our catalog complements existing efforts to map AI risk and what it means for developers, researchers, journalists, policymakers, and the public. Related Work We start by reviewing research on AI risk taxonomies, followed by that on collating AI risks into databases. AI risk taxonomies The taxonomization of AI risks has grown significantly in recent years ( Yampolskiy 2016 ; Liu et al. 2024b ; Stahl and Eke 2024 ; Shelby et al. 2023 ) . One of the most widely used taxonomies proposed by DeepMind ( Weidinger et al. 2022 ) , categorizes risks into six primary groups: representation and toxicity, misinformation, information and safety, malicious use, human autonomy and integrity, and socioeconomic and environmental harms. Each category is further divided into a total of 21 subcategories, such as toxic content under the first category and environmental damage under the last. Weidinger et al. 2023 also explore the context of harm origins, identifying three layers: capability (related to the system’s technical features), human interaction (linked to users’ experiences with the system), and systemic impact (including the broader context in which the system operates). Other notable taxonomies include that by Shelby et al. 2023 , which identifies five key harm categories: presentation, allocative, quality of service, interpersonal, and social system harms. Yampolskiy 2016 proposes a classification based on the causes of risks, organized along three axes: entity (human or AI), intentionality (intentional or unintentional), and timing (pre-deployment or post-deployment). Arguing that many existing taxonomies, including those mentioned above, are primarily designed for researchers and policymakers, making them less accessible to broader audience, Abercrombie et al. 2024 introduced the AI, Algorithmic, and Automation Harms Taxonomy, which aims to be more comprehensible to the general public. This framework adds categories such as physical and psychological harms. AI risks data sources Another important area of research focuses on collating AI risks into data sources. Risks envisioned by researchers. In the most comprehensive effort to date, Slattery et al. 2024 analyzed 43 scholarly and industry frameworks proposing various AI risk taxonomies and compiled them into the MIT AI Risk Repository. This continuously updated repository currently lists 967 risks, categorized using two taxonomies: a causal taxonomy adapted from Yampolskiy 2016 and a domain taxonomy extended from the DeepMind taxonomy ( Weidinger et al. 2021 ) . Using a best-fit approach ( Carroll et al. 2013 ) , Slattery et al. 2024 refined the DeepMind taxonomy by adding a seventh category, AI system safety, failures, and limitations , and making minor adjustments to existing categories. Derczynski et al. 2023 introduced “Risk Cards” providing definitions, categorizations, and examples of harms tied to specific language models

Read the complete text