Quantitative Auditing of AI Fairness with Differentially Private Synthetic Data

Chih-Cheng Rex Yuan, Bow-Yaw Wang · 2025

Read Quantitative Auditing of AI Fairness with Differentially Private Synthetic Data by Chih-Cheng Rex Yuan, Bow-Yaw Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-21634.txt.

Introduction

Chih-Cheng Rex Yuan and Bow-Yaw Wang propose a safer way to audit AI systems for bias, in a paper published in 2025. Audits normally need real personal data, which makes auditors targets for attack and risks leaks even in careful hands. Their framework swaps real data for fake data made under a privacy shield, which mimics the real set's patterns without exposing anyone.

They tested the idea on three public sets covering income, court risk scores, and diabetes. Fairness scores from the fake data tracked those from real data closely enough to spot the same broad problems, though some gaps appeared. The authors argue the method lets outsiders audit sensitive systems, in fields like health and justice, without holding the keys to private records.

Catalog record

CREATORChih-Cheng Rex Yuan, Bow-Yaw Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-21634

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Quantitative Auditing of AI Fairness with Differentially Private Synthetic Data Chih-Cheng Rex Yuan, Bow-Yaw Wang Abstract . Fairness auditing of AI systems can identify and quantify biases. However, traditional auditing using real-world data raises security and privacy concerns. It exposes auditors to security risks as they become custodians of sensitive information and targets for cyberattacks. Privacy risks arise even without direct breaches, as data analyses can inadvertently expose confidential information. To address these, we propose a framework that leverages differentially private synthetic data to audit the fairness of AI systems. By applying privacy-preserving mechanisms, it generates synthetic data that mirrors the statistical properties of the original dataset while ensuring privacy. This method balances the goal of rigorous fairness auditing and the need for strong privacy protections. Through experiments on real datasets like Adult, COMPAS, and Diabetes, we compare fairness metrics of synthetic and real data. By analyzing the alignment and discrepancies between these metrics, we assess the capacity of synthetic data to preserve the fairness properties of real data. Our results demonstrate the framework’s ability to enable meaningful fairness evaluations while safeguarding sensitive information, proving its applicability across critical and sensitive domains. CCS: General and reference Metrics CCS: Security and privacy Usability in security and privacy CCS: Information systems Data mining CCS: Computing methodologies Machine learning CCS: Computing methodologies Artificial intelligence CCS: Security and privacy privacy-preserving protocols CCS: Information systems Information systems applications CCS: Mathematics of computing Contingency table analysis 1. Introduction Fairness in machine learning has become an increasingly important topic as AI systems are widely used across various sectors, including healthcare and criminal justice. These AI systems can significantly influence outcomes that impact individuals and communities. However, AI systems can exhibit bias, which may lead to increased unfairness. Biased AI systems have the potential to reinforce historical injustices, erode trust in technology, and exacerbate inequality. Therefore, ensuring fairness in AI systems is crucial to prevent the perpetuation of these biases. For instance, investigative journalists at ProPublica uncovered bias in the COMPAS system, which is used in the United States criminal justice system to predict recidivism. Their analysis revealed that the COMPAS system disproportionately discriminated against certain demographic groups. These findings have raised significant concerns about the fairness and ethical implications of using such AI systems in critical decision-making processes. Addressing these biases is essential to ensure that AI systems do not perpetuate inequalities. One effective approach to achieving fairness in AI systems is to conduct audits. Quantitative fairness auditing involves evaluating these systems using various fairness metrics to identify potential biases and inequalities. Some commonly proposed metrics for assessing AI system fairness include demographic parity, equalized odds, and accuracy equality. These metrics provide a quantitative basis for addressing fairness concerns in AI models. A well-documented example of fairness auditing is the investigation of the COMPAS system. Journalists from ProPublica revealed that COMPAS exhibited bias against certain demographic groups, particularly African Americans. The system was more likely to falsely classify African American defendants as high risk compared to their Caucasian counterparts. This finding not only underscored the necessity of fairness auditing but also highlighted the potential harm of deploying biased AI systems in critical areas. This case emphasizes the importance of independent audits in ensuring fairness. Evaluations conducted by third parties, such as those performed by ProPublica, are vital because they operate free from the influence of those who develop or deploy the systems. By offering an objective perspective, third-party audits can identify and address biases that internal evaluations might overlook. Without such oversight, AI systems risk perpetuating systemic inequalities, making independent auditing a crucial aspect of accountability in AI development. While fairness auditing is essential, it presents significant challenges, particularly when it involves sensitive data. To conduct audits, auditors may rely on access to real datasets containing sensitive information. Auditors often require access to real-world datasets containing sensitive attributes, such as race, gender, or income, to evaluate an AI system’s fairness. However, this reliance on sensitive data raises concerns about data security and privacy. This reliance invites data security attacks on auditors, making them targets of unauthorized access. The use of real datasets for fairness auditing creates a point of vulnerability, making auditors potential targets for data breaches and unauthorized access. If sensitive data falls into the wrong hands, it could lead to severe consequences, including identity theft, discrimination, and reputational damage. Moreover, analyses of real data run the risk of data inference attacks, where confidential information may be deduced from seemingly harmless or aggregated data. Even if the real data is not directly exposed, the results of analyses can sometimes reveal sensitive information through inference attacks. For example, attackers could combine publicly available audit results with external datasets to infer private details about individuals in the original dataset. Various technologies have been developed to mitigate privacy risks, such as differential privacy. Differential privacy is a mathematical framework that ensures the inclusion or exclusion of any individual in a dataset has a minimal and quantifiable impact on the overall results of data analysis. This is achieved by introducing carefully calibrated random noise into the computations derived from the data, ensuring that individual records remain indistinguishable while preserving aggregate patterns and trends. Differential privacy is especially useful in situations that involve sensitive data. For example, a differential privacy mechanism could allow researchers to analyze patterns in patient health data without exposing any single patient’s medical history. This approach strikes a balance between data utility and privacy, enabling meaningful analysis while safeguarding individuals against privacy breaches or inference attacks. The key idea is that no matter how much background knowledge an attacker has about the data, they cannot determine with high confidence whether a specific individual is included in the dataset. This is quantified by some parameters which control the amount of noise introduced; smaller parameters provide stronger privacy but can reduce the accuracy of the analysis, while larger parameters allow for more precise results but with weaker privacy guarantees. The challenge is to carefully calibrate the parameters to ensure that privacy is maintained without overly compromising the utility of the data. One way of auditing while preserving privacy is to use synthetic data. Synthetic data are fake data that is generated from real data. Synthetic data is artificially generated to replicate the statistical properties of real data while protecting privacy. The principle of this technology is to preserve some statistical properties of the real data while protecting the privacy of the individuals in the real data. Unlike anonymized data, synthetic data breaks the link to the original data, reducing the risk of re-identification. Among the generation techniques, differentially private synthetic data offers a way to investigate the data properties under differential privacy guarantees. It stands out for it strong differential privacy guarantees when generating synthetic data. This method combines data synthesis with the principles of differential privacy, ensuring that individuals’ sensitive information cannot be inferred from the synthetic data. By introducing controlled randomness during generation, it provides strong privacy guarantees while preserving the overall utility of the data. Differentially private synthetic data is particularly valuable in sensitive domains like healthcare and criminal justice, enabling researchers to analyze patterns or identify trends without exposing personal details. It also supports fairness

Read the complete text