Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks

Valentin Lafargue, Adriana Laurindo Monteiro, Emmanuelle Claeys, Laurent Risser, Jean-Michel Loubes · 2025

Read Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks by Valentin Lafargue, Adriana Laurindo Monteiro, Emmanuelle Claeys, Laurent Risser, Jean-Michel Loubes (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-20708.txt.

Introduction

Valentin Lafargue and co-authors show how a firm could fake a fair audit result for an AI classifier, in work from 2025. European law treats many AI systems as high risk, and audits often check a fairness ratio on a sample the firm itself supplies. A supervisor may check that the sample looks representative, but that check has limits.

The team frames the attack as a constrained projection problem: shift the data as little as possible until the fairness test passes. They give mathematically grounded strategies based on entropy and transport methods, and measure the smallest shift needed. They then test how well statistical distance checks catch the faked samples. Some manipulations stay undetected under set conditions. The authors offer guidelines to strengthen supervisory checks.

Catalog record

CREATORValentin Lafargue, Adriana Laurindo Monteiro, Emmanuelle Claeys, Laurent Risser, Jean-Michel Loubes
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE112 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-20708

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks Valentin Lafargue, Adriana Laurindo Monteiro, Emmanuelle Claeys, Laurent Risser, Jean-Michel Loubes Abstract Proving the compliance of AI algorithms has become an important challenge with the growing deployment of such algorithms for real-life applications. Inspecting possible biased behaviors is mandatory to satisfy the constraints of the regulations of the EU Artificial Intelligence’s Act. Regulation‑driven audits increasingly rely on global fairness metrics, with Disparate Impact being the most widely used. Yet such global measures depend highly on the distribution of the sample on which the measures are computed. We investigate first how to manipulate data samples to artificially satisfy fairness criteria, creating minimally perturbed datasets that remain statistically indistinguishable from the original distribution while satisfying prescribed fairness constraints. Then we study how to detect such manipulation. Our analysis (i) introduces mathematically sound methods for modifying empirical distributions under fairness constraints using entropic or optimal transport projections, (ii) examines how an auditee could potentially circumvent fairness inspections, and (iii) offers recommendations to help auditors detect such data manipulations. These results are validated through experiments on classical tabular datasets in bias detection. Our code is available on Github 1 1 1 github.com/ValentinLafargue/Inspection . 1 Introduction PUC ADULT INC EMP MOB BAF TRA 0.30 0.37 0.67 0.32 0.45 0.30 0.35 0.69 0.35 1 0.54 0.85 0.93 0.52 Datasets 0.2 0.4 0.6 0.8 Impact Disparate Figure 1: Evading Bias Audits: Undetectable Maximal Fairness Manipulation in Seven Benchmark Datasets Fairness auditing has emerged as a critical practice to ensure that machine learning models comply with ethical and legal standards by not exhibiting discriminatory bias. We refer to ( Barocas, Hardt, and Narayanan 2019 ; Besse et al. 2022 ; Oneto and Chiappa 2020 ; Wang, Zhang, and Zhu 2022 ) and references therein. High-profile investigative audits, such as the ProPublica analysis of the COMPAS recidivism risk tool, have exposed significant biases against certain demographic groups ( Angwin et al. 2016 ) . These findings underscored the societal harms of unverified AI systems and prompted calls for regular fairness audits by independent parties ( Raji et al. 2020 ) . In response, regulators have begun instituting fairness compliance requirements. For instance, the EU’s proposed AI Act mandates bias monitoring, and in the U.S., the Disparate Impact DI doctrine (the “80% rule”) is used to quantify indirect discrimination in algorithms ( Feldman et al. 2015 ) . This doctrine requires that the selection rate for a protected group be at least 80% of that of the most favored group. Consequently, the demographic parity metric (also known as statistical parity) has become a standard global fairness criterion which has inspired many mitigation methods, for example, in ( Hardt, Price, and Srebro 2016a ; Gouic, Loubes, and Rigollet 2020 ; Chzhen et al. 2020 ) . A small demographic parity gap or a ratio above 0.8 is expected for fairness under this rule, along with other metrics such as equalized odds and predictive parity. These metrics provide quantifiable targets for auditors and have been integrated into various auditing toolkits ( Bellamy et al. 2018 ; Bird, Dimmery, and Walker 2020 ) . We consider an auditing framework in which the auditee submits a sub-sample of their dataset to a regulatory authority, either by providing the algorithm’s outputs on that sample or by sharing the sample itself along with API access to the model, allowing the auditor to compute outputs independently. The supervisory body is then responsible for verifying that the submitted sample is sufficiently representative of the auditee’s complete dataset. This framework is particularly relevant for high-risk systems, where rigorous oversight is required. The supervisory authority may be internal (e.g., a general inspection body) or external, such as the Cour des Comptes for public administration or the ACPR for banking supervision, Ensuring the good faith of the auditee is critical, as exemplified by the Volkswagen emissions scandal ( Jacobs and Kalbers 2019 ) , in which vehicles were engineered to detect auditing conditions and deceive regulators by temporarily producing compliant behavior. Similar concerns arise in machine learning, where inconsistencies in model behavior during auditing have been documented, such as in the case of Facebook’s models discussed in ( Bourrée et al. 2025 ) . In summary, the auditing framework (similar to the one proposed in ( Fukuchi, Hara, and Maehara 2020 ) ) involves three distinct entities: 1. The audited entity , which provides a subset of its data and, in the context of the audit, grants access to run its algorithm on this data. 2. The auditor , who applies standardized procedures to assess whether the dataset and corresponding model outputs satisfy a prescribed fairness criterion, in this case, Disparate Impact. 3. The supervisory authority , a higher-level body that oversees the integrity of the entire auditing process. It ensures that the audited entity submits a dataset that is representative of the full underlying data distribution, and that the auditor adheres to established auditing protocols. In this work, our objective is to support supervisory authorities by identifying potential strategies that audited entities might use to circumvent fairness audits, and by providing tools to detect such attempts. Building on the notion of manipulation-proof introduced in ( Yan and Zhang 2022 ) , we show how a dataset that initially violates a fairness criterion, such as Disparate Impact, can be minimally altered to appear compliant, with limited distributional shift as measured by the Kullback–Leibler (KL) divergence or the Wasserstein distance. By systematically analyzing these plausible manipulations, our aim is to raise awareness of audit vulnerabilities and to equip oversight bodies with methods to detect suspicious modifications, thereby strengthening the reliability and robustness of fairness auditing processes. Our contributions are the following: • We introduce an entropic projection under constraint tool to a new field that is fairness application and auditing, we also build upon this tool to enable constraints on DI. • We provide mathematical foundations for Wasserstein projection under constraint and implement its application, as well as other Wasserstein-minimizing algorithm, to control distribution shift under fairness constraint. • We assess whether–and to what extent– a sample from the projected distribution can significantly increase, without being detected by distributional-based statistical tests, the Disparate Impact from 7 unfair tabular datasets, as illustrated in Fig. 1 . These tests would be used by the supervisory authority to assert the representativeness of the sample. 2 Related Works Bias mitigation Achieving fairness under these metrics has has prompted extensive research. One line of work proposes pre-processing techniques that alter the training data to remove bias. ( Kamiran and Calders 2009 ) introduced a “data massaging” approach, flipping class labels of a few selected instances to reduce discrimination. ( Feldman et al. 2015 ) proposed repairing datasets by adjusting feature values to remove disparate impact, while ( Calmon et al. 2017 ) framed the problem as a convex optimization for probabilistic data transformation. More recently, ( Celis, Straszak, and Vishnoi 2019 ) introduced a maximum entropy approach to learn fair distributions under statistical constraints. ( Gordaliza et al. 2019 ) or ( Chakraborty, Oudot, and Vayatis 2024 ) applied optimal transport (OT) to modify datasets by respectively reweighting the training dataset or reducing the relationship between

Read the complete text