Underrepresentation, Label Bias, and Proxies: Towards Data Bias Profiles for the EU AI Act and Beyond

Marina Ceccon, Giandomenico Cornacchia, Davide Dalle Pezze, Alessandro Fabris, Gian Antonio Susto · 2025

Read Underrepresentation, Label Bias, and Proxies: Towards Data Bias Profiles for the EU AI Act and Beyond by Marina Ceccon, Giandomenico Cornacchia, Davide Dalle Pezze, Alessandro Fabris, Gian Antonio Susto (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-08866.txt.

Introduction

Marina Ceccon and co-authors sort three data biases that bend AI outcomes in a 2025 paper. Biased training data drive much algorithmic discrimination, yet each bias kind is rarely told apart. The team tested underrepresentation, label bias, and proxies alone and in pairs across sets, models, and fairness gauges.

Scarce groups in training data proved less harmful than lore claims. Proxy traits joined with skewed labels, by contrast, did the deep damage. Sensors for each bias kind were then merged into a data bias profile. Case trials show the profile flags discrimination risk early.

Catalog record

CREATORMarina Ceccon, Giandomenico Cornacchia, Davide Dalle Pezze, Alessandro Fabris, Gian Antonio Susto
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-08866

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Underrepresentation, Label Bias, and Proxies: Towards Data Bias Profiles for the EU AI Act and Beyond Marina Ceccon, Giandomenico Cornacchia, Davide Dalle Pezze, Alessandro Fabris, Gian Antonio Susto Abstract Undesirable biases encoded in the data are key drivers of algorithmic discrimination. Their importance is widely recognized in the algorithmic fairness literature, as well as legislation and standards on anti-discrimination in AI. Despite this recognition, data biases remain understudied, hindering the development of computational best practices for their detection and mitigation. In this work, we present three common data biases and study their individual and joint effect on algorithmic discrimination across a variety of datasets, models, and fairness measures. We find that underrepresentation of vulnerable populations in training sets is less conducive to discrimination than conventionally affirmed, while combinations of proxies and label bias can be far more critical. Consequently, we develop dedicated mechanisms to detect specific types of bias, and combine them into a preliminary construct we refer to as the Data Bias Profile (DBP) . This initial formulation serves as a proof of concept for how different bias signals can be systematically documented. Through a case study with popular fairness datasets, we demonstrate the effectiveness of the DBP in predicting the risk of discriminatory outcomes and the utility of fairness-enhancing interventions. Overall, this article bridges algorithmic fairness research and anti-discrimination policy through a data-centric lens. K eywords AI Act \cdot algorithmic fairness \cdot anti-discrimination \cdot bias detection \cdot data bias Correct reference for this work : Marina Ceccon, Giandomenico Cornacchia, Davide Dalle Pezze, Alessandro Fabris, Gian Antonio Susto. “Underrepresentation, label bias, and proxies: Towards Data Bias Profiles for the EU AI act and beyond”. (2025). Expert Systems with Applications . https://doi.org/10.1016/j.eswa.2025.128266 1 Introduction Algorithmic anti-discrimination is a relatively young field, rapidly moving from niche research to market readiness ( Álvarez et al. 2024 ) . Several years of work carried out by a growing research community have convincingly shown that algorithms developed without attention to fairness put vulnerable groups at a systematic disadvantage ( Angwin et al. 2016 ; Obermeyer et al. 2019 ; Glazko et al. 2024 ) . Recognizing the critical implications of this research, policymakers and standards organizations have published regulations and norms on the topic ( Schwartz et al. 2022 ; Parliament 2024 ; ISO 2021 ) . These documents require standardization to apply across many domains where fairness work is critical, including medicine ( Obermeyer et al. 2019 ) , finance ( Gillis et al. 2024 ) , employment ( Fabris et al. 2024 ) , and education ( Baker and Hawn 2022 ) . Evaluations of data bias are key computational tools for anti-discrimination work. Since biases in the data are fundamental drivers of algorithmic discrimination ( Vetrò et al. 2021 ; Brzezinski et al. 2024 ) , bias management is mentioned in every recent standard and regulation on algorithmic anti-discrimination ( Schwartz et al. 2022 ; Parliament 2024 ; ISO 2021 ) . Policy formulations on this topic are rather vague, favouring flexibility on one hand, but leaving the contours of law-abiding bias management undefined for practitioners, contributing to legal risk and uncertainty. For example, recent regulation requires providers of AI systems in high-risk domains to signal sufficient efforts of bias detection and mitigation. How this should be done in practice, however, is left completely undefined ( Deck et al. 2024 ) . Defining precise criteria for bias management requires answering several important questions, left mostly unaddressed in the fairness literature. First, different data biases are associated with algorithmic discrimination. Which combinations of biases are more critical for fairness? Second, data biases are typically described qualitatively but vastly lack a quantitative characterization. Is it possible to monitor distinct data biases unambiguously? Third, while a plethora of fairness interventions exist in the literature, a set of guidelines to prioritize them is prominently lacking. In sum, how can practitioners and researchers make decisions about bias mitigation in a principled manner? Contributions . In this work, we tackle the above questions, providing several contributions. • We study three types of data bias widely recognized for their negative influence on algorithmic fairness, namely underrepresentation, label bias, and proxies. Through extensive experiments on diverse datasets, algorithms, and fairness measures we analyze their joint influence on algorithmic discrimination. Our experiments show that underrepresentation in training data is overemphasized in the literature while label bias is more critical. This novel result challenges conventional wisdom held in the algorithmic fairness community. • We propose a principled mechanism for bias detection that is widely applicable in practice. More in detail, we develop a suite of measures to detect specific data biases without auxiliary information from external sources. We integrate these measures into a preliminary construct, the Data Bias Profile (DBP) , which provides a quantitative foundation for identifying and communicating data biases, as well as assessing the risk of algorithmic discrimination. This framework serves as a proof of concept, offering a concrete starting point that the research community can build upon to develop a more robust and systematic approach to bias-aware data documentation. • We discuss the far-reaching implications of our work for researchers and practitioners. We recommend that researchers use DBPs to select datasets with complementary properties for their experiments, overcoming the present limitations of fairness benchmarks. Finally, we make recommendations for practitioners on the curation and utilization of anti-discriminatory datasets. Structure . Section 2 presents related work. Section 3 introduces data bias and experimental protocols to analyze it. Section 4 studies the effect of data bias. Section 5 describes bias detection, introducing and demonstrating DBPs. Sections 6 and 7 discuss our results in the broader context of algorithmic fairness work concluding with recommendations for researchers and practitioners. 2 Related Work Algorithmic fairness research keeps contributing new approaches to measure the risk of discrimination ( Cornacchia et al. 2023 ; Chen et al. 2024 ; Fabris et al. 2023 ; Cooper et al. 2024 ) and to mitigate it ( Yin et al. 2023 ; Cruz and Hardt 2024 ; Zhang et al. 2024 ; Fajri et al. 2024 ) . Moving fairness toward market readiness requires research on operationalizing algorithmic anti-discrimination policy ( section 2.1 ), on its close connection with data bias ( section 2.2 ), and on principled documentation practices to support anti-discrimination ( section 2.3 ). 2.1 AI policy on anti-discrimination Influential legislation and standards on anti-discrimination in AI, such as the EU AI Act ( Parliament 2024 ) and the NIST report on bias in AI ( Schwartz et al. 2022 ) require multi-disciplinary research to translate policy into computational best practices. Drukker et al. 2023 , for example, complement NIST guidelines with a list of domain-specific biases that arise in the medical domain. Borgesius et al. 2024 focus on the AI Act, summarizing the main requirements for mandatory evaluation of data biases and their documentation. Deck et al. 2024 compile a list of practical challenges for compliance with anti-discrimination requirements outlined in the AI Act. By highlighting the need for model owners to examine “possible biases that are likely to […] lead to discrimination prohibited under Union law”, they stress the pressing questions of which biases lead to discriminatory models and what kind of

Read the complete text