Efficiently Attacking Memorization Scores
Tue Do, Varun Chandrasekaran, Daniel Alabi · 2025
Read Efficiently Attacking Memorization Scores by Tue Do, Varun Chandrasekaran, Daniel Alabi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-20463.txt.
Introduction
Scores that rate how strongly a model memorized a training example can themselves be gamed. Tue Do, Varun Chandrasekaran, and Daniel Alabi demonstrate the attack in a 2025 arXiv paper. Memorization scores guide data attribution and dataset curation. If the scores bend under pressure, decisions built on them bend too.
The attack crafts inputs that look highly memorized while being highly sensitive queries. It needs only black-box access to model outputs and modest computing power, using a pseudoinverse calculation at its core. Across a wide suite of image classification tasks, even state-of-the-art proxy scores fell to targeted manipulation. The authors back the experiments with theory on when influence estimates are inherently fragile. Their findings argue that attribution tools need defenses of their own before high-stakes use.
Catalog record
| CREATOR | Tue Do, Varun Chandrasekaran, Daniel Alabi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-20463
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficiently Attacking Memorization Scores Tue Do, Varun Chandrasekaran, Daniel Alabi Abstract Influence estimation tools—such as memorization scores—are widely used to understand model behavior, attribute training data, and inform dataset curation. However, recent applications in data valuation and responsible machine learning raise the question: can these scores themselves be adversarially manipulated? In this work, we present a systematic study of the feasibility of attacking memorization-based influence estimators. We characterize attacks for producing highly memorized samples as highly sensitive queries in the regime where a trained algorithm is accurate. Our attack (calculating the pseudoinverse of the input) is practical, requiring only black-box access to model outputs and incur modest computational overhead. We empirically validate our attack across a wide suite of image classification tasks, showing that even state-of-the-art proxies are vulnerable to targeted score manipulations. In addition, we provide a theoretical analysis of the stability of memorization scores under adversarial perturbations, revealing conditions under which influence estimates are inherently fragile. Our findings highlight critical vulnerabilities in influence-based attribution and suggest the need for robust defenses. All code can be found at https://github.com/tuedo2/MemAttack 1 Introduction Online data market platforms, such as AWS Data Exchange ( AWS ) , Dawex ( Dawex ) , Xignite ( XIgnite ) , WorldQuant ( WorldQuant ) , are spaces where data is bought and sold. Concretely, there are three major entities in a data market: platforms, buyers, and sellers/providers ( Kennedy et al. 2022 ; Alabi et al. 2025 ) . The data market platform performs data valuation based on the acquired data from data sellers ( Azcoitia and Laoutaris 2022 ; Mehta et al. 2021 ; Agarwal et al. 2019 ; Fan et al. 2020 ; Wang and Jia 2023 ) . Sellers in data markets offer datasets (collections of information) that are valuable for businesses, researchers, or governments. Buyers look for data that can help them make better decisions, run more effective marketing campaigns, or improve products and services. Data valuation is released to buyers who use that information to buy data ( Jung and Park 2019 ; Ray et al. 2020 ) . Influence functions such as Shapley values are commonly used to price data (proportional to its valuation) in a data market ( Yan and Procaccia 2021 ; Song et al. 2019 ; Wang et al. 2020 ) . A series of recent papers ( Feldman and Zhang 2020 ; Feldman 2020 ; Brown et al. 2021 ) propose the label memorization score for supervised classification (in machine learning settings): in large datasets, a small subset of highly influential (memorized) training examples disproportionately affects the model’s predictions and generalization capabilities, while the majority of examples have little to no impact. Clearly, this concept is relevant in data valuation, where the goal is to identify which training data points contribute most to a model’s decision-making (i.e., samples with high memorization scores are more valuable). While the original proposal ( Feldman and Zhang 2020 ) is computationally expensive, various studies ( Garg et al. 2023 ; Ravikumar et al. 2024 ; Jiang et al. 2020 ; Zhao and Triantafillou 2024 ) propose efficient proxies. Given the difficulty of obtaining high-quality data in data market platforms (and consequently their high value), there is a clear economic incentive for some sellers to manipulate the valuation scores ( Alabi et al. 2025 ) . We aim to study how malicious data sellers can alter influence valuations in data markets. In particular, we look at the robustness of memorization-based approaches, by proposing multiple attacks based on distribution shift, stability notions, and decision boundary proximity. Basu et al. 2021 previously study the robustness of influence functions, but do not provide theoretical guarantees and restrict their empirical analyses to neural networks. Lai and Bayraktar 2020 provide an approach for adversarial robustness of general estimators (not influence estimators). Yadav et al. 2024 address the robustness of influence functions on adversaries that control the valuation algorithm, a strong assumption. In contrast to these works, we present the first theoretical analysis for vulnerability of memorized-based valuation functions. We particularly assert that when a trained algorithm exhibits high accuracy, we can characterize highly memorized samples with highly sensitivity. We take advantage of the inverse operation’s naturally high sensitivity to motivate a simple, computationally efficient, and effective Pseudoinverse attack to produce highly memorized samples in image classification tasks. In comparison to prior work, we assume a much weaker adversarial model, one where the (malicious) data sellers only change the provided data, require no collusion, and are unaware of the exact (memorization-based) valuation algorithm. In our most effective attack, the adversary need not be aware of knowledge of the underlying data distribution! Figure 1 illustrates the threat model. The simplicity, yet effectiveness, of both our theory and attack demonstrate a fundamental vulnerability in data valuation for supervised learning. Figure 1: An overview of how data valuation functions can be attacked We summarize our contributions below: • We develop a theoretical framework for analyzing vulnerability of memorization scores to adversarial manipulation and demonstrate its implications on potential data pricing (§ 4 ). • We provide a simple and efficient Pseudoinverse attack that successfully alters memorization-based scoring of image classification tasks agnostic of underlying dataset and model architecture (§ 5 ). • We verify our theoretical guarantees with experiments on image classification tasks across various convolutional network architectures on the MNIST, SVHN, and CIFAR-10 datasets (§ 6 ), and present additional support over more complex transformer architecture, higher resolution image datasets, and other data modalities in (§ C ). 2 Related Work Data Markets: Azcoitia and Laoutaris 2022 give a survey on existing commercial data markets and their business models. Mehta et al. 2021 introduce pricing policies for enabling data economies. Agarwal et al. 2019 introduce a mathematical model for data marketplaces. Also, recent work surveys privacy and security vulnerabilities in data markets and offers possible design solutions ( Alabi et al. 2025 ) . Our work introduces a realistic threat model and calls for robust algorithmic solutions for future data markets. Data Valuation: Shapley values are a classical concept in game theory which were first employed for the problem of data valuation in machine learning by Ghorbani and Zou 2019 , termed Data Shapley Value, which aims to measure the influence of individual data samples on some specified performance metric. Due to computational overhead of computing Shapley Value, various influence functions have been proposed and adopted, including finding a core set of data examples ( Yan and Procaccia 2021 ) , gradient-based approximations to the Shapley Value ( Song et al. 2019 ) , adaptations to federated learning setting ( Wang et al. 2020 ) , as well as non-Shapley based influence functions such as gradient tracing ( Pruthi et al. 2020 ) and low-rank kernel approximation ( Park et al. 2023 ) . However, it is not always clear what target for which the influence of data samples should be measured on. Common settings involve a specific test set on which the effects of influence are measured, but a definitive target test set might not be available or might change over time, or might not even be known in certain applications. Label memorization, the main metric of