The Quest for Reliable Metrics of Responsible AI
\name Theresia Veronika Rampisela \addr University of Copenhagen \email [email protected] · 2025
Read The Quest for Reliable Metrics of Responsible AI by \name Theresia Veronika Rampisela \addr University of Copenhagen \email [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26007.txt.
Introduction
Theresia Veronika Rampisela, Maria Maistro, Tuukka Ruotsalo, and Christina Lioma reflect on how to judge the judges of responsible AI. Responsible AI asks that systems be fair, safe, and accountable, and progress is often tracked with evaluation metrics. Yet the metrics themselves can be fragile, and far less work checks their robustness than uses them to score models.
Rampisela and the team draw on prior studies of fairness metrics for recommender systems, a concrete case where metric weakness has been probed. They distill the lessons into a set of practical, non exhaustive guidelines for building reliable responsible AI metrics. The guidelines are meant to apply broadly, including to AI used in science, where bad metrics can mislead high stakes work.
Catalog record
| CREATOR | \name Theresia Veronika Rampisela \addr University of Copenhagen \email [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 13 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-26007
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Quest for Reliable Metrics of Responsible AI \name Theresia Veronika Rampisela \addr University of Copenhagen \email [email protected] Abstract The development of Artificial Intelligence (AI), including AI in Science (AIS), should be done following the principles of responsible AI. Progress in responsible AI is often quantified through evaluation metrics, yet there has been less work on assessing the robustness and reliability of the metrics themselves. We reflect on prior work that examines the robustness of fairness metrics for recommender systems as a type of AI application and summarise their key takeaways into a set of non-exhaustive guidelines for developing reliable metrics of responsible AI. Our guidelines apply to a broad spectrum of AI applications, including AIS. Keywords: responsible AI, fairness evaluation, recommender systems 1 Introduction Recent legislation, such as the EU Artificial Intelligence (AI) Act and the EU Digital Services Act, has increasingly emphasised the responsible development of AI applications [ European Parliament and Council 2022 , European Parliament and Council 2024 ] to protect their users and minimise harms. One such AI application is Recommender Systems (RSs). RSs are highly useful in finding relevant items that match user needs, preferences, or interests [ Aggarwal 2016 ] . These systems are widely used for both high- and low-stakes activities in professional and personal settings. In professional settings, among other use cases, RSs can be used to suggest scientific papers and citations for researchers [ Kreutz and Schenkel 2022 , Färber and Jatowt 2020 ] , or job positions for jobseekers [ Siting et al. 2012 ] . Prior work has investigated aspects of responsible AI in RSs, such as fairness and bias [ Wang et al. 2023 , Klimashevskaia et al. 2024 ] . Biases may exist in the data, the algorithm, or the evaluation pipeline of an RS; all these biases can contribute to unfairness. While there is no universal definition, fairness in RSs is commonly understood as providing equal treatment or equitable outcomes to the RS stakeholders [ Wang et al. 2023 ] , ensuring that no individuals or groups are systematically discriminated. Unfair RSs can have severe real-life consequences. In job recommendations, for example, unfair RSs may contribute to the exacerbation of gender pay gaps. This could happen if historically marginalised groups (e.g., women) are only shown recommendations for lower-paying jobs, while highly-paid positions are only recommended to the historically dominant group. In the AIS context, an unfair paper/citation RS could perform extremely well in recommending relevant work for one discipline (e.g., computer science) but perform poorly for other disciplines (e.g., Nordic studies) due to training data imbalance between various fields of study, hindering the development of sciences. Similarly, if an unfair RS overpromotes articles by researchers from economically developed countries [ Biega et al. 2020 ] , it may simultaneously provide less exposure to researchers from other countries. This imbalance could lead to a less inclusive understanding of science [ Mihalcea et al. 2025 ] , especially in social sciences and humanities, where cultural context in particular matters. Hence, the development of fair RSs is not only scientifically relevant but also societally important. Many evaluation approaches and metrics have been proposed to measure RS fairness. Yet, few studies have investigated the reliability and robustness of these approaches. This leads to the metric score being potentially misleading or unstable, and thus unreliable. Having reliable metrics is critical as they primarily guide the responsible development of AI(S) applications, especially in the early stages, as they provide a scalable way of evaluating system performance without significant cost (e.g., opposed to user studies). In this paper, we condense insights from our previous work on RS fairness evaluation [ Rampisela et al. 2024a , Rampisela et al. 2024b , Rampisela et al. 2025a , Rampisela et al. 2025b ] into a set of guidelines for developing reliable metrics of responsible AI, based on the limitations that we uncovered in existing approaches. As these guidelines are general, they would also be relevant for evaluating AI in science (AIS). 2 Fairness Evaluation In this part, we explain how fairness, an aspect of responsible AI, is evaluated in RSs ( Section 2.1 ). We summarise our findings and contributions from previous work on RS fairness evaluation ( Section 2.2 ). 2.1 Evaluation of Fairness in Recommender Systems At a high level, Recommender System (RS) fairness evaluation is often categorised in terms of the subject (user/item fairness) and the evaluation granularity (group/individual fairness) [ Wang et al. 2023 ] . User fairness concerns the disparity in the recommendation effectiveness, where an RS that performs equally well for all users would be deemed fairer than an RS that can perform very well for some users but fails to provide relevant recommendations for others. In contrast, item fairness is usually defined in terms of how much exposure is received by the item, in comparison to other items in the RS. In terms of granularity, group fairness concerns the difference in utility (recommendation effectiveness or item exposure) between subject groups, where the groups are typically formed based on user or item attributes, e.g., socio-demographic attributes. Meanwhile, individual fairness is frequently operationalised as the variation of utility across all users or items. RS practitioners often rely on computing metrics as a proxy for fairness. Over 30 metrics have been used to measure fairness for different subjects and at different granularity levels [ Wang et al. 2023 ] . Some metrics are based on well-known inequality measures, such as standard deviation, entropy, and Gini Index. Others are newly developed based on existing fairness notions, e.g., Rawlsian fairness and envy-freeness. With the sheer amount of metrics, it is often difficult to understand what and how they actually quantify fairness. Even worse, some metrics were proposed without further analysis of their limitations. Consequently, the measure limitations are largely unknown. 2.2 Our Contributions to RS Fairness Evaluation We analysed several RS fairness metrics and uncovered limitations within the measures. To resolve the limitations, we proposed novel fairness evaluation approaches. Finally, we provided practical guidelines for evaluating RS fairness. Analysis on metric limitations. Our investigation on metric limitations was done both theoretically and empirically. In our theoretical studies, we show that some fairness metrics are mathematically flawed [ Rampisela et al. 2024a , Rampisela et al. 2025a ] . Firstly, some metrics would either crash upon computation due to invalid mathematical operations (e.g., division by 0), resulting in the metric outputting no scores that can be used for fairness assessment, rendering it unusable. Secondly, a large number of the metrics have an unknown score range, or if it is known, the max/min score is not reachable. Suppose that in theory, a metric score ranges between 0 and 1, where 0 is the fairest score and 1 is the unfairest score. In some cases, there is no input to the metric that would output 0 or 1. Instead, for example, its score could only range between 0.3 and 0.6 (i.e., the minimum and maximum reachable scores, respectively). This causes difficulty in interpreting the metric score, as one would interpret a score of 0.5 as somehow fair if the range is [0,1] , but unfair if the range is [0.3,0.6] . Thirdly, in some cases, it