Guda : Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning
Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji · 2026
Read Guda : Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning by Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-22651.txt.
Introduction
Naoki Murata and a team assign credit for a diffusion model's output to whole groups of training data. Attribution asks which data shaped a given image. Their 2026 preprint, Guda, notes that users often need group answers — a style, a class — not one image at a time.
Group credit is a counterfactual: how would the output change if the group were gone? Their method answers by way of unlearning. Fair group credit can guide pay, consent, and data curation. The work serves artists and data holders most. It makes a vague debt concrete.
Catalog record
| CREATOR | Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 84 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-22651
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Guda : Counterfactual Group-wise Training Data Attribution for Diffusion Models via Unlearning Naoki Murata, Yuhta Takida, Chieh-Hsin Lai, Toshimitsu Uesaka, Bac Nguyen, Stefano Ermon, Yuki Mitsufuji Abstract Training-data attribution for vision generative models aims to identify which training data influenced a given output. While most methods score individual examples, practitioners often need group-level answers (e.g., artistic styles or object classes). Group-wise attribution is counterfactual: how would a model’s behavior on a generated sample change if a group were absent from training? A natural realization of this counterfactual is Leave-One-Group-Out (LOGO) retraining, which retrains the model with each group removed; however, it becomes computationally prohibitive as the number of groups grows. We propose Guda (Group Unlearning-based Data Attribution) for diffusion models, which approximates each counterfactual model by applying machine unlearning to a shared full-data model instead of training from scratch. Guda quantifies group influence using differences in a likelihood-based scoring rule (ELBO) between the full model and each unlearned counterfactual. Experiments on CIFAR-10 and artistic style attribution with Stable Diffusion show that Guda identifies primary contributing groups more reliably than semantic similarity, gradient-based attribution, and instance-level unlearning approaches, while achieving ∼ \sim 100 × \times speedup on CIFAR-10 over LOGO retraining. 1 Introduction Figure 1 : Overview of the Guda (Group Unlearning-based Data Attribution) framework. Instead of retraining N+1 models from scratch (LOGO), Guda starts from the all-group model \theta^{\mathrm{full}} and applies machine unlearning to obtain approximate counterfactual models \theta^{\mathrm{ul}}_{-k} for each group k . Attribution is computed by comparing the ELBO of generated samples under the all-group model versus each unlearned model, providing an efficient approximation to the counterfactual target validated against LOGO. Consider an AI-generated artwork that blends multiple artistic styles ( 13 ; 25 ) . A fundamental question arises: which training data groups contributed most to this output? This question is critical for copyright assessment, fair compensation for data providers, and debugging ( 4 ; 10 ; 12 ) . We address the core problem of group-wise data attribution: estimating the counterfactual influence of training data groups on generated outputs. Instance-level vs. group-level attribution. While instance-level training data attribution methods ( 17 ; 23 ; 22 ; 7 ; 19 ; 36 ) estimate which specific training samples influenced a generation, group-level attribution asks a fundamentally different question: how would generation change if an entire group (e.g., an artistic style, object class) were absent from training? Simply summing instance-level scores fails due to (i) scalability : evaluating per-instance contributions becomes prohibitive as datasets grow, and (ii) nonlinearity : group influences interact through shared representations, so collective effects are generally non-additive ( 14 ; 2 ; 16 ) . Counterfactual models as the estimation target. The natural gold standard for group attribution is the counterfactual model retrained without the target group. The quantity we seek to estimate is the difference in model behavior between the full-data model \theta^{\mathrm{full}} and the counterfactual model \theta^{\mathrm{logo}}_{-k} trained without group k , for which Leave-One-Group-Out (LOGO) retraining provides an oracle implementation. We measure group attribution by comparing the model’s likelihood-based explanatory power (via ELBO as a tractable surrogate) on a sample under the full-data model versus the counterfactual model. However, LOGO retraining is prohibitively expensive: training N+1 models from scratch (where N is the number of groups) scales poorly as the number of groups increases, making it impractical when applied to large-scale models. Unlearning as a natural approximation. Our key insight is that group attribution inherently asks for the effect of data removal : the counterfactual question “what if group k were absent from training?” is precisely a data removal query. This is exactly the operation that machine unlearning addresses ( 11 ; 3 ; 21 ) . This makes unlearning the natural solution class for approximating \theta^{\mathrm{logo}}_{-k} : starting from \theta^{\mathrm{full}} and selectively removing a group’s influence through fine-tuning, rather than expensive retraining from scratch. Importantly, using unlearning for attribution is not a plug-in choice: deletion-oriented objectives do not target an explicit LOGO counterfactual. We therefore define a likelihood-based LOGO estimand for per-generation group influence and design redirection-based unlearning (including anchor conditioning) whose fidelity is validated against LOGO oracles. We propose Guda as a general framework that uses unlearning to approximate counterfactual models and estimates group influence by comparing their likelihood-based measures (ELBO difference). To achieve this, the unlearning loss consists of two terms: a shared preservation term to prevent catastrophic forgetting, and a setting-specific forget term that varies by generation setting. For unconditional generation, we adopt ReTrack-based redirection ( 27 ) that trains toward importance-weighted targets from the retain set. For conditional text-to-image attribution, we instead redirect forget-condition responses toward retain-style anchors, yielding a conditional analogue of ReTrack-style redirection. We summarize the overall Guda pipeline in Fig. 1 . This paper makes the following contributions: • Framework: We formalize per-generation group attribution in diffusion models around a LOGO-style counterfactual estimand, and provide an unlearning-based approximation that makes it practical at scale. Guda defines group influence as an ELBO difference between the full-data model and the counterfactual model \theta^{\mathrm{logo}}_{-k} (Sec. 3.3 ), and approximates \theta^{\mathrm{logo}}_{-k} by machine unlearning from a shared initialization rather than expensive from-scratch retraining (Sec. 4 ), with approximation quality validated against LOGO in experiments. • Two instantiations: We demonstrate Guda in two settings: (i) unconditional generation via ReTrack-based importance-weighted unlearning (Sec. 4.2.1 ), and (ii) conditional text-to-image style attribution via CLIP-weighted style-selection anchors for redirection (Sec. 4.2.2 ) for a conditional analogue of ReTrack. • Empirical validation: We demonstrate on CIFAR-10 and artistic style attribution (Sec. 5 ) that Guda substantially outperforms semantic similarity baselines (CLIPA) and instance-level attribution ( 19 ; 31 ) in identifying the most influential groups, validating that group-wise counterfactual approximation captures causal influence more effectively. 2 Related Work 2.1 Instance-Level Attribution and Group Effects Influence functions ( 17 ) , TracIn ( 23 ) , and TRAK ( 22 ) provide instance-level attribution with improved scalability. However, extending these to group effects poses challenges: 16 show that aggregating instance-level influences often fails to capture nonlinear group interactions, while 2 address this through second-order approximations. From a data valuation perspective, Data Shapley ( 9 ) and CS-Shapley ( 26 ) provide principled frameworks based on cooperative game theory, and 20 apply Shapley-style crediting to diffusion models for global contributor valuation using pruning/fine-tuning to accelerate subset evaluation. Unlike these approaches, which target specific value functions, our work directly estimates the counterfactual model resulting from group removal during training. 2.2 Data Attribution for Diffusion Models Recent methods address attribution specifically for diffusion models: 7 , 19 , and 36 develop instance-level attribution techniques for these settings. 30 establish evaluation protocols, and 31 connect unlearning to attribution for text-to-image models. Our work differs from 31 in two key aspects: (i) Estimand : we approximate the group-removal counterfactual model \theta^{\mathrm{logo}}_{-k} , not instance-level attribution for a synthesized output; (ii) Method : we unlearn the target group to approximate \theta^{\mathrm{logo}}_{-k} , rather than unlearning a synthesized image to obtain loss-based ranking signals. 2.3 Machine Unlearning for Diffusion Models Machine unlearning removes training data influence without full retraining, with foundational work by 11 and 3 . For diffusion models, representative approaches include ESD ( 8 ) , Forget-Me-Not (