Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise DOI: XXXXXXX.XXXXXXX ISBN: TBD
Paweł Borsukiewicz, Fadi Boutros, Iyiola E. Olatunji, Charles Beumier, Wendkûuni Christian OUEDRAOGO, Jacques Klein, Tegawendé F. Bissyandé · 2025
Read Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise DOI: XXXXXXX.XXXXXXX ISBN: TBD by Paweł Borsukiewicz, Fadi Boutros, Iyiola E. Olatunji, Charles Beumier, Wendkûuni Christian OUEDRAOGO, Jacques Klein, Tegawendé F. Bissyandé (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-17372.txt.
Introduction
Face recognition needs huge photo sets, yet collecting real faces without consent has led to retracted data sets and legal risk. Paweł Borsukiewicz, with six co-authors, asks whether computer made faces can replace real ones without giving up accuracy. Earlier work had not tested that claim broadly enough to settle the question, so the team combined a review of past synthetic sets with fresh experiments.
Across more than ten million synthetic samples, the best sets scored above ninety-five percent on standard face tests, passing a well known real data set. Borsukiewicz and his co-authors also find that synthetic faces can leak less private data and allow bias to be adjusted at creation time. Gaps remain, but the case for synthetic training data looks strong.
Catalog record
| CREATOR | Paweł Borsukiewicz, Fadi Boutros, Iyiola E. Olatunji, Charles Beumier, Wendkûuni Christian OUEDRAOGO, Jacques Klein, Tegawendé F. Bissyandé |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-17372
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise DOI: XXXXXXX.XXXXXXX ISBN: TBD Paweł Borsukiewicz, Fadi Boutros, Iyiola E. Olatunji, Charles Beumier, Wendkûuni Christian OUEDRAOGO, Jacques Klein, Tegawendé F. Bissyandé Abstract . The deployment of facial recognition systems has created an ethical dilemma: achieving high accuracy requires massive datasets of real faces collected without consent, leading to dataset retractions and potential legal liabilities under regulations like GDPR. While synthetic facial data presents a promising privacy-preserving alternative, the field lacks comprehensive empirical evidence of its viability. This study addresses this critical gap through extensive evaluation of synthetic facial recognition datasets. We present a systematic literature review identifying 25 synthetic facial recognition datasets (2018-2025), combined with rigorous experimental validation. Our methodology examines seven key requirements for privacy-preserving synthetic data: identity leakage prevention, intra-class variability, identity separability, dataset scale, ethical data sourcing, bias mitigation, and benchmark reliability. Through experiments involving over 10 million synthetic samples, extended by a comparison of results reported on five standard benchmarks, we provide the first comprehensive empirical assessment of synthetic data’s capability to replace real datasets. Best-performing synthetic datasets (VariFace, VIGFace) achieve recognition accuracies of 95.67% and 94.91% respectively, surpassing established real datasets including CASIA-WebFace (94.70%). While those images remain private, publicly available alternatives Vec2Face (93.52%) and CemiFace (93.22%) come close behind. Our findings reveal that they ensure proper intra-class variability while maintaining identity separability. Demographic bias analysis across four ethnic groups shows that, even though synthetic data inherits limited biases, it offers unprecedented control for bias mitigation through generation parameters. These results establish synthetic facial data as a scientifically viable and ethically imperative alternative for facial recognition research. By eliminating consent violations while maintaining competitive performance, synthetic datasets provide a sustainable path forward for the field. † † authorsaddresses: Authors’ addresses: {pawel.borsukiewicz, emmanuel.olatunji, wendkuuni.ouedraogo, jacques.klein, tegawende.bissyande}@uni.lu, [email protected], [email protected] Correspondence: [email protected] 1. Introduction Facial recognition technology processes billions of faces daily across security, commerce, and personal devices. Its capability largely depends on training with large-scale images, which enable the development of robust models. Yet, this reliance on vast data collection poses fundamental privacy, ethical, and bias concerns, as the images used are often gathered without an individual’s explicit consent ( Smith and Miller 2022 ) . Consequently, researchers have retracted many well-established datasets ( Parkhi et al. 2015 ; Cao et al. 2018 ; Guo et al. 2016 ) in response to these legal and ethical challenges ( Wang et al. 2024 ; Beltrán and Calvo 2023 ) . To address this privacy compromise, synthetic facial recognition datasets have emerged as a promising alternative, garnering significant interest in recent years ( Boutros et al. 2023e ) . Early approaches often augmented real datasets with synthetic data ( Boutros et al. 2022 ; Qiu et al. 2021 ) , yet yielded limited performance gains. However, continual methodological improvements have enabled models to be trained entirely on synthetic data, achieving increasingly competitive benchmark results. Recent studies demonstrate that these fully synthetic models can now rival or even surpass ( Yeung et al. 2024 ) the performance of models trained on real data. This observed progress demonstrates that synthetic data can not only ensure reliable performance but also mitigate inherent ethical and legal concerns of real datasets, including demographic biases and label noise, while inherently preserving privacy. Despite the inherent privacy benefits of synthetic data, its adoption in both research and industry remains limited, primarily due to the absence of standardized benchmarks and comprehensive validation. This work addresses this gap by presenting a systematic review and extensive evaluation of current synthetic facial recognition datasets. We critically assess the state-of-the-art to benchmark its current capabilities and highlight specific weaknesses that require further research. The contributions of this work are as follows. • Identification of the core ethical and legal limitations of real-face datasets and how synthetic datasets resolve them. • A systematic literature review that identifies and benchmarks 25 synthetic facial recognition datasets and provides a historical analysis of the evolution of synthetic dataset generation methodologies. • A novel framework and practical guidelines for synthetic facial recognition datasets, establishing key requirements and a standardized protocol for their evaluation. • An in-depth evaluation of existing datasets, yielding novel quantitative insights on intra-class variance, identity separability, and identity leakage. • A discussion of compelling future research directions, informed by our analysis of the current landscape. The remainder of this paper is structured as follows. Section 2 details the critical limitations of real-face datasets that motivate this review. Section 3 outlines our systematic methodology for literature collection, scope, and dataset selection. Section 4 introduces key requirements for privacy-preserving synthetic datasets and surveys their primary generation methods and relevant competitive benchmarks. Section 5 provides a comprehensive evaluation of the field’s progress against these requirements, highlighting both significant advancements and persistent challenges. Finally, Section 6 concludes the paper and discusses avenues for future work. 2. Motivation Consider the foundation upon which modern facial recognition is built: massive datasets scraped from the web, enabling models that process billions of faces daily. This paradigm is now collapsing. As shown in Table 1 , a majority of the most prominent real-face datasets, including VGGFace ( Parkhi et al. 2015 ) , VGGFace2 ( Cao et al. 2018 ) , and MS-Celeb-1M ( Guo et al. 2016 ) , are no longer available from their original sources. This systemic retraction is not merely an inconvenience; it is a direct consequence of irreconcilable ethical, legal, and technical limitations that render the real-data paradigm unsustainable. The core weaknesses of real-face datasets are threefold: • Ethical and legal limitations: The scale required for state-of-the-art performance has historically been achieved by scraping images from the web ( Parkhi et al. 2015 ; Cao et al. 2018 ; Taigman et al. 2015 ; Wang et al. 2018a ) without explicit consent . This practice is ethically questionable ( Smith and Miller 2022 ) and clashes directly with a tightening global regulatory landscape, including the EU’s GDPR ( European Parliament and Council of the European Union 2016 ) and laws like Illinois’ BIPA ( Assembly 2008 ) , which treat facial data as a protected biometric identifier ( Wang et al. 2024 ; Beltrán and Calvo 2023 ) . • Inherent demographic biases: Web-scraped datasets often reflect and amplify real-world demographic imbalances, leading to documented biases in gender ( Albiero et al. 2020 ; Buolamwini and Gebru 2018 ) , race ( Wang et al. 2019 ) , and age. While synthetic datasets are not immune to bias ( Huber et al. 2024 ; Whitney and Norman 2024 ) , their generation process offers a more direct and controllable path to mitigation ( Kortylewski et al. 2018a ; Kortylewski et al. 2019 ) . • Pervasive label noise and volatility: The mass collection of images introduces significant label noise that is costly to rectify ( Wang et al. 2018a ) . Even benchmark datasets like LFW contain mislabeled images and inconsistent identity groupings ( Bae et al. 2023 ) . Furthermore, distribution methods that rely on volatile web links lead to broken URLs and corrupted data integrity over time ( Wang et al. 2018a ) . Table 1 . Real datasets used for