Interaction-Aware Influence Functions for Group Attribution
Jaeseung Heo, Kyeongheung Yun, Youngbin Choi, Sehyun Hwang Jungseul Ok Dongwoo Kim · 2026
Read Interaction-Aware Influence Functions for Group Attribution by Jaeseung Heo, Kyeongheung Yun, Youngbin Choi, Sehyun Hwang Jungseul Ok Dongwoo Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15675.txt.
Introduction
Jaeseung Heo and five colleagues add interaction terms to influence functions for group credit. Influence functions gauge how dropping one case shifts a target loss. Group credit is usually the sum of single credits, which misses effects: pairs may repeat or complement each other. The team expands the target to second order, adding a pair term that tracks how two cases line up. On six data-model pairs it tracks retraining better than first-order influence. Used to pick tuning data for Llama, it beats past baselines on most tasks.
Catalog record
| CREATOR | Jaeseung Heo, Kyeongheung Yun, Youngbin Choi, Sehyun Hwang Jungseul Ok Dongwoo Kim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 86392 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15675
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interaction-Aware Influence Functions for Group Attribution Jaeseung Heo, Kyeongheung Yun, Youngbin Choi, Sehyun Hwang Jungseul Ok Dongwoo Kim Abstract Influence functions approximate how removing a training example changes a quantity of interest, called the target function, such as a held-out loss. To estimate the influence of a group of examples, the standard practice is to sum the individual influences of its members. However, this sum does not capture how examples jointly affect the target: a pair of examples may be redundant or complementary, but the sum cannot distinguish these cases. We propose an interaction-aware influence function that characterizes how interactions between examples influence the target. By expanding the target to second order around the trained parameters, we obtain an estimator that augments the standard sum with a pairwise interaction term that captures the alignment between two examples’ effects on the target. We empirically evaluate our estimator in two settings. First, on six dataset–model pairs spanning logistic regression, MLPs, and ResNet-9, our estimator tracks leave-group-out retraining substantially better than first-order influence across all settings. Second, when used as a greedy selection rule for instruction-tuning data on Llama-3.1-8B, it beats prior influence-based and representation-similarity baselines on five of seven downstream tasks, in a regime where standard influence-based selection underperforms random selection. 1 Introduction Quantifying how training examples shape a trained model is a foundational problem in machine learning, with applications including selecting valuable training data [ 18 , 59 , 64 ] , understanding what kinds of data benefit a model [ 22 , 24 ] , and fairly compensating data providers [ 10 , 20 ] . The conceptual gold standard is leave-one-out retraining: remove a training example, retrain the model, and measure how some quantity of interest has changed. Since retraining for every example is prohibitive, influence functions [ 29 ] estimate this counterfactual in closed form by modeling two things in turn: how reweighting a training example would shift the trained parameters, and how those shifted parameters would change the quantity of interest. The quantity of interest, called the target function , is left to the user. It can be the loss on a held-out example [ 29 ] , an average over a validation set, the likelihood of a downstream task answer [ 10 , 59 ] , or any other smooth function of the model. This flexibility is what lets a single approximation serve so many applications. Most of these applications, including subset selection, group-robustness analysis, and data valuation, ultimately ask about groups of examples rather than single ones. The textbook extension scores a group as the sum of its members’ individual influences [ 30 ] , implicitly assuming examples contribute independently. This assumption is rarely true. Two near-duplicate examples each look highly influential on their own, yet adding both has roughly the same effect as adding one, so additive scoring double-counts the overlap and overestimates the group’s true effect [ 25 , 26 ] . The same blind spot makes top- k influence-based data selection produce subsets that are highly redundant rather than collectively useful [ 67 ] , since the most individually influential examples tend to be similar. 6 previously addressed this with a second-order correction that makes the estimator non-additive, but their construction refines only how the trained parameters shift, missing how the target function responds to those shifts. We argue that group attribution is fundamentally about how training examples affect the target. Therefore, interactions between examples should be characterized in terms of their joint effect on the target, not just on the parameter shift. To this end, we propose an interaction-aware influence function that augments the standard sum of individual influences with a pairwise interaction term over examples in the group. The pairwise term captures the joint effect on the target: it is positive when the two examples push the model in similar directions and negative when they push in opposite directions. We further derive a closed-form factorization of the interaction term across classification settings, proving that two similar examples are jointly complementary when their labels differ and redundant when their labels agree, formally recovering classical principles in supervised learning [ 7 , 15 , 39 , 46 ] . As an application, we use the interaction term to discount candidates whose effects overlap with already selected examples, addressing the redundancy in top- k influence-based selection. We evaluate the method in two settings. First, we test whether our estimator faithfully tracks leave-group-out retraining across six dataset–model pairs, by assessing Spearman rank correlation with ground-truth retraining effects. It substantially outperforms prior influence-based training data attribution methods, demonstrating the importance of accounting for interactions in group attribution. Second, we evaluate the data selection method at scales ranging from small MLPs to instruction-tuning of Llama-3.1-8B [ 21 ] . On small-scale models, our method selects subsets with class diversity comparable to random selection, whereas baselines collapse onto a few classes and fall below random. In instruction-tuning data selection, our method consistently improves over random selection and achieves the strongest overall performance on five of seven downstream tasks. 2 Preliminary We first review the classical influence function and its standard extension to group attribution. We summarize the notation used throughout the paper in Table 2 of Appendix A . Let \mathcal{D}=\{z_{i}\}_{i=1}^{N} be the training set of size N , where each example z_{i}=(x_{i},y_{i}) consists of an input x_{i} and a label y_{i} , and let \ell(z_{i},\theta) denote the loss of example z_{i} at parameter \theta\in\Theta . For any subset A\subseteq\mathcal{D} , define the empirical risk and its minimizer as \mathcal{R}_{A}(\theta)\coloneqq\frac{1}{|A|}\sum_{z_{i}\in A}\ell(z_{i},\theta),\qquad\hat{\theta}_{A}\coloneqq\arg\min_{\theta}\mathcal{R}_{A}(\theta). (1) We write \hat{\theta}\coloneqq\hat{\theta}_{\mathcal{D}} for the parameter trained on the full dataset. For a subset S\subseteq\mathcal{D} , the parameter obtained by retraining after removing S is \hat{\theta}_{\mathcal{D}\setminus S} . Let f:\Theta\to\mathbb{R} be a differentiable scalar target function of the trained parameters that we treat as a quantity to be minimized, such as the loss on a test example or the average loss on a validation set. 1 1 1 A target to be maximized can be handled by negating f . The exact leave-group-out effect of removing S is \mathcal{I}^{-}(S)\coloneqq f(\hat{\theta}_{\mathcal{D}\setminus S})-f(\hat{\theta}). (2) For a singleton S=\{z_{i}\} , this reduces to the usual leave-one-out effect. Computing \mathcal{I}^{-}(S) exactly requires retraining the model without S , which is expensive even for a single example and infeasible for many candidate groups. Influence functions approximate this retraining effect through infinitesimal reweighting [ 29 ] . Instead of removing S directly, consider the parameter obtained after adding a small weight \epsilon to the losses of examples in S : \hat{\theta}(\epsilon;\,S)\coloneqq\arg\min_{\theta}\left\{\frac{1}{N}\sum_{z_{i}\in\mathcal{D}}\ell(z_{i},\theta)+\epsilon\sum_{z_{i}\in S}\ell(z_{i},\theta)\right\}. (3) At \epsilon=0 , this recovers the original parameter \hat{\theta} . Since each example has weight 1/N in the empirical risk, setting \epsilon=-1/N removes the contribution of every example in S . Thus, leave-group-out retraining can be viewed as a finite step along the reweighting path \hat{\theta}(\epsilon;\,S) . The influence function replaces this finite step with two first-order approximations. Let H\coloneqq\tfrac{1}{N}\sum_{z_{i}\in\mathcal{D}}\nabla^{2}_{\theta}\ell(z_{i},\hat{\theta}) be the Hessian of the training objective. Assuming that \ell(z_{i},\cdot) is twice continuously differentiable for every z_{i}\in\mathcal{D} and that H is nonsingular at \hat{\theta} , the first approximation linearizes the reweighting path \hat{\theta}(\epsilon;\,S) in \epsilon at \epsilon=0 , yielding the parameter shift induced by removing S as \hat{\theta}_{\mathcal{D}\setminus