Measuring the Accuracy and Effectiveness of PII Removal Services

Jiahui He, Peter Snyder, Hamed Haddadi, Fabián E. Bustamante, Gareth Tyson · 2025

Read Measuring the Accuracy and Effectiveness of PII Removal Services by Jiahui He, Peter Snyder, Hamed Haddadi, Fabián E. Bustamante, Gareth Tyson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06989.txt.

Introduction

Jiahui He and four co-authors measured how well paid privacy services scrub personal data from broker databases, in a 2025 paper. Such services promise to find and delete users' records automatically. Volunteers bought subscriptions to four popular services. The team tracked what each service found and claimed. Removal success and record accuracy were checked by hand.

Results fell short of the marketing. Under half of found records were actually removed. Most records flagged were not even about the subscriber. Only about four in ten matched the person who paid. Coverage claims across brokers also diverged from practice. The authors call the services' usefulness limited as privacy tools. Buyers may overestimate the protection they purchased. Independent audits like this one help calibrate expectations, they argue.

Catalog record

CREATORJiahui He, Peter Snyder, Hamed Haddadi, Fabián E. Bustamante, Gareth Tyson
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06989

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measuring the Accuracy and Effectiveness of PII Removal Services Jiahui He, Peter Snyder, Hamed Haddadi, Fabián E. Bustamante, Gareth Tyson Abstract . This paper presents the first large-scale empirical study of commercial personally identifiable information (PII) removal systems — commercial services that claim to improve privacy by automating the removal of PII from data broker’s databases. Popular examples of such services include DeleteMe , Mozilla Monitor , Incogni , among many others. The claims these services make may be very appealing to privacy-conscious Web users, but how effective these services actually are at improving privacy has not been investigated. This work aims to improve our understanding of commercial PII removal services in multiple ways. First, we conduct a user study where participants purchase subscriptions from four popular PII removal services, and report ( i ) what PII the service find, ( ii ) from which data brokers, ( iii ) whether the service is able to have the information removed, and ( iv ) whether the identified information actually is PII describing the participant. And second, by comparing the claims and promises the services makes ( e.g. which and how many data brokers each service claims to cover). We find that these services have significant accuracy and coverage issues that limit the usefulness of these services as a privacy-enhancing technology. For example, we find that the measured services are unable to remove the majority of the identified PII records from data broker’s (48.2% of the successfully removed found records) and that most records identified by these services are not PII about the user (study participants found that only 41.1% of records identified by these services were PII about themselves). Volume: YYYY X DOI: XXXXXXX.XXXXXXX 1. Introduction Personal Identifiable Information (PII) has become a core of the global information economy. PII refers to any data that can be used to identify a specific individual, including information such as names, addresses, emails, phone numbers, and even biometric data like fingerprints ( 43 ) . With the continuous growth in the value of PII, the risk and frequency of data exposure has also increased ( 13 , 4 , 16 ) . Such incidents not only violate personal privacy, but may also lead to identity theft, financial fraud, and long-term damage to the reputation of individuals and organizations. In this digital ecosystem, data brokers play an important and controversial role by treating PII as a commodity. They collect and trade vast amounts of PII –— often without individuals’ knowledge or consent —– from sources such as public records, online activities, social media, and retail transactions ( 7 , 22 , 24 ) . The data broker market, projected to reach $382.16 billion by 2030 ( 40 ) , fuels concerns about consent, ownership, and misuse of PII. For instance, the 2017 Equifax breach compromised the PII of 147 million people, highlighting the systemic risks posed by centralized repositories of sensitive data ( 49 ) . These risks underscore the urgent need for mechanisms to mitigate the unchecked proliferation of PII. In response to the risks posed by the poorly regulated collection of personal PII by data brokers, various laws have been issued. F GDPR, implemented in 2018, established strong data protection standards for EU citizens, requiring organizations to be transparent about data collection and usage ( 23 , 2 , 48 , 50 ) . Similarly, the CCPA, implemented in 2020, grants California residents the right to know about their personal data, its collection purpose, and sharing practices, while also allowing them to opt-out of data sales ( 6 , 9 , 5 ) . However, enforcement gaps and the opaque nature of data brokerage persist, leaving individuals with limited practical control over their PII ( 22 , 11 ) . In order for individuals to better protect personal privacy, a new industry has emerged: PII removal services . These services act on behalf of individuals (typically for a small fee). They actively seek to remove users’ PII from data brokers and other online platforms to reduce the risk of misuse of PII and data breaches. The process typically involves identifying which data brokers hold an individual’s data, submitting formal removal requests, and monitoring for compliance. PII removal services often employ a combination of automated tools and manual processes to efficiently manage data removal requests. By intervening in the data broker ecosystem, the PII removal service arguably strengthens individual’s control over their own data and also contributes positively to the promotion of privacy protection. Despite the growing role these PII removal services play in the privacy ecosystem, they remain critically understudied. To bridge this gap, we conduct the first large-scale empirical study on PII removal services. Based on an initial survey of 10 major PII removal services and the 2,024 data brokers they cover, we recruit 71 participants to use 4 different services. Through this, we study what PII the services discover on each data broker, alongside whether the service is able to (correctly) remove the PII. We explore the following research questions: • RQ1 : What are the characteristics of PII removal services in terms of the data brokers they cover and the information they require to pursue data removals? • RQ2 : How effectively do these PII removal services discover user records on data brokers, in terms of the number and accuracy of records retrieved? • RQ3 : What is the efficacy of the PII removal service in removing the discovered records? Specifically, what is the percentage of records successfully removed from the data brokers, and what is the time required to achieve this? Through studying these RQs, our findings include: (1) The data broker coverage of the PII removal services varies widely, with a low overlap between them (average Jaccard similarity of 0.21). There are only 10 data brokers common to all services, indicating they may target different industries or user groups. We find that 71.7% of data brokers covered are not government-registered, showing a lack of regulation. ((§ 4.1 ) (2) The removal services vary in terms of their requested user PII (required to facilitate the data removals). Full Name , Email , and City Address are mandatory across all of them. However, services with larger data broker coverage tend to request more PII from users during the subscription stage. (§ 4.2 ) (3) Data brokers operate across various industries, predominantly in business (19.0%) and information technology (16.0%). However, aside from those in the economy and finance industry, most data brokers are not registered with the required government authorities, highlighting insufficient regulatory practices. (§ 4.3 ) (4) Removal services that boast of covering a larger number of brokers, do not necessarily have high success rates. Notably, Kanary has the worst performance, despite having the widest data broker coverage. In fact, it identifies the lowest number (average of 14.6) of records for the users from its broker coverage. This indicates that larger data broker coverage does not necessarily mean that PII can be successfully removed from all listed data brokers. (§ 5.1 ) (5) The accuracy of the PII removal service in retrieving records is also low, with an average of 41.1% of records being correctly linked to the participant

Read the complete text