Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods
Dennis Wei, Inkit Padhi, Soumya Ghosh, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Maria Chang · 2024
Read Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods by Dennis Wei, Inkit Padhi, Soumya Ghosh, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Maria Chang (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-03906.txt.
Introduction
Dennis Wei and five co-authors rethink how to credit training data for a model's behavior. Their 2024 paper works in the final-model-only setting: the trained model is all you have. No training recipe or checkpoints are available. The question is which training examples shaped the model most.
They frame credit as sensitivity: how much would extra training on an example move the model? That further-training test becomes the gold standard. Popular gradient methods, they show, are all shortcuts approximating that test. On table, image, and text data, quick first-order shortcuts sometimes matched well. Influence-function shortcuts were steadier but, oddly, less accurate overall.
Catalog record
| CREATOR | Dennis Wei, Inkit Padhi, Soumya Ghosh, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Maria Chang |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-03906
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods Dennis Wei, Inkit Padhi, Soumya Ghosh, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Maria Chang Abstract Training data attribution (TDA) is concerned with understanding model behavior in terms of the training data. This paper draws attention to the common setting where one has access only to the final trained model, and not the training algorithm or intermediate information from training. We reframe the problem in this “final-model-only” setting as one of measuring sensitivity of the model to training instances. To operationalize this reframing, we propose further training , with appropriate adjustment and averaging, as a gold standard method to measure sensitivity. We then unify existing gradient-based methods for TDA by showing that they all approximate the further training gold standard in different ways. We investigate empirically the quality of these gradient-based approximations to further training, for tabular, image, and text datasets and models. We find that the approximation quality of first-order methods is sometimes high but decays with the amount of further training. In contrast, the approximations given by influence function methods are more stable but surprisingly lower in quality. 1 Introduction Training data attribution (TDA, or sometimes simply “data attribution”) refers to the attribution or explanation of ML model behavior in terms of its training data. Existing methods for TDA fall into several categories: re-training-based approaches ( ghorbani2019datashapley ; ghorbani202020distributional ; feldman2020memorize ; kwon2022beta ; ilyas2022datamodels ; lin2022measuring ; wang2023banzhaf ) , gradient-based approaches applied throughout training ( hara2019cleansing ; pruthi2020tracin ; chen2021hydra ; bae2024unrolled ) , and gradient approaches applied only at the end ( koh2017inffunc ; yeh2018representer ; koh2019group ; barshan2020relatif ; basu2020second ; guo2021fastif ; schioppa2022scalingarnoldi ; park2023trak ; kwon2024datainf ) . We refer to the survey by hammoudeh2024tdasurvey for a deeper look at these categories. In this paper, rather than proposing another TDA method, we take a more reflective approach. First, we draw attention to the fact that there exist multiple problem settings for TDA, alongside the multiple categories of methods. These problem settings differ in the level of access assumed. In particular, we focus on what we call the “final-model-only” (FiMO) setting in which we have access only to the final trained model, and not the algorithm used to train the model or intermediate information from training (for example checkpoints). The FiMO setting is motivated by the common scenario in which TDA is performed by a different party than the one who developed the model. Models published on platforms such as HuggingFace have now made this scenario ubiquitous. We find that since the TDA literature has not clearly differentiated these problem settings, it is also not clear on what should be the goal for TDA in the FiMO setting, and accordingly, what could be an ideal “gold standard” method. Having such a goal and gold standard (as opposed to proxy tasks such as mislabelled example detection) facilitates the development and evaluation of more practical methods. We thus reframe the problem in the FiMO setting as one of quantifying sensitivity of the model to training instances. We call the reframed problem “FiMODA” (DA for data attribution). We then propose further training , starting from the given final model, as a gold standard measure of sensitivity. Notably, our proposal adjusts for the effect of further training non-converged models and accounts for the randomness of neural network training algorithms. These refinements have consequences as we show in our experiments. Like re-training 1 1 1 Throughout the paper, we reserve the term “re-training” to mean re-training from scratch. , further training a model multiple times can be computationally prohibitive. We thus consider approximations based on first- and second-order Taylor expansions of the further training objective. In doing so, we unify several existing gradient-based methods for TDA. These methods include gradient similarity ( charpiat2019similarity ) (a final-checkpoint-only case of TracIn ( pruthi2020tracin ) ) and methods based on influence functions ( koh2017inffunc ; schioppa2022scalingarnoldi ; park2023trak ; grosse2023studyingllmgen ; kwon2024datainf ) . Given that these gradient-based methods approximate further training as we show, we submit that they are more suited to the FiMO setting, rather than as approximations to re-training where their effectiveness has been questioned ( basu2021inffragile ; nguyen2023bayesian ) . Our derivation of gradient-based methods also does not make assumptions of convexity or stationarity that are common in the TDA literature, and we obtain generalized influence function expressions as a result. We investigate empirically the quality of the approximations to further training provided by different gradient-based methods. Our experiments span the modalities of tabular, image, and text data. Overall, we find that first-order gradient-based methods can give good initial approximations to further training, but the quality of approximation decays with the amount of further training. In contrast, the approximation quality of influence function methods is more persistent, but somewhat surprisingly, never as high as first-order methods at their peak. We provide code to help reproduce our experiments at https://github.com/IBM/fimoda . Table 1 : Comparison with other retrospective works on TDA plus koh2017inffunc (FT = Further training) Category Attribute Ours koh2017inffunc basu2021inffragile nguyen2023bayesian zhang2022rethinking bae2022infanswer schioppa2023theoretical Setting FiMO (explicit) ✓ FT refinements Training randomness ✓ Non-convergence ✓ ✓ ✓ Derivations Non-convexity/stationarity ✓ ✓ ✓ Generalized influence functions ✓ Scope # TDA methods evaluated 8 2 2 3 2 2 2 Our contributions are summarized as follows and in Table 1 : 1) We highlight the FiMO setting for TDA (Section 2 ). 2) We reframe the TDA problem in the FiMO setting as one of quantifying sensitivity. We articulate and refine a further training gold standard for this reframed problem (called FiMODA, Section 3 ). 3) We show how several gradient-based TDA methods approximate further training, theoretically (Section 4 ) and numerically (Section 6 ). 2 Problem Settings for Training Data Attribution Preliminaries In all problem settings that we consider for TDA, we are given access to the training dataset \mathcal{D}=\{{\bm{z}}_{i}\}_{i=1}^{n} for the model, consisting of n pairs {\bm{z}}_{i}=({\bm{x}}_{i},y_{i}) of inputs {\bm{x}}_{i}\in{\mathcal{X}} and targets y_{i}\in{\mathcal{Y}} . A model f({\bm{x}};{\bm{\theta}}) is a function f:{\mathcal{X}}\to{\mathcal{F}} , parameterized by {\bm{\theta}}\in\mathbb{R}^{p} , that maps to an output space {\mathcal{F}} (e.g., predicted logits for a classification model). The quality of each model output f({\bm{x}}_{i};{\bm{\theta}}) with respect to target y_{i} is measured by a loss function \ell(f({\bm{x}}_{i};{\bm{\theta}}),y_{i}) ; we adopt the more compact notation L({\bm{z}}_{i};{\bm{\theta}})\triangleq\ell(f({\bm{x}}_{i};{\bm{\theta}}),y_{i}) . We use A to denote the training algorithm used to train f({\bm{x}};{\bm{\theta}}) on dataset \mathcal{D} . We view A as a function that takes \mathcal{D} and initial model parameters {\bm{\theta}}^{(0)} (which may come from a pre-trained model) as input and outputs final model parameters {\bm{\theta}}^{f}=A(\mathcal{D},{\bm{\theta}}^{(0)}) . Given a test instance {\bm{z}}=({\bm{x}},y) and an evaluation function g({\bm{z}},{\bm{\theta}}) that depends in general on both f({\bm{x}};{\bm{\theta}}) and y , the task of TDA is to assign scores a_{i} quantifying the importance of each training instance {\bm{z}}_{i} to the output g({\bm{z}},{\bm{\theta}}) . We refer to a_{i} interchangeably as an attribution or influence score. Commonly, the evaluation function is the loss on {\bm{z}} , g({\bm{z}},{\bm{\theta}})=L({\bm{z}};{\bm{\theta}}) , or the output of the model alone, g({\bm{z}},{\bm{\theta}})=f({\bm{x}};{\bm{\theta}}) . One may also consider evaluation functions that sum over test instances {\bm{z}} . Three problem settings We distinguish three problem settings for TDA, based on the level of access to