Counterfactual Influence as a Distributional Quantity
Matthieu Meeus, Igor Shilov, Georgios Kaissis, Yves-Alexandre de Montjoye · 2025
Read Counterfactual Influence as a Distributional Quantity by Matthieu Meeus, Igor Shilov, Georgios Kaissis, Yves-Alexandre de Montjoye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20481.txt.
Description
Matthieu Meeus and co-authors present a 2025 paper on a blind spot in how memorization by language models is measured. The usual gauge asks how one training sample changes the model's memory of itself. Near-duplicate samples, though, can carry the same content, so a sample can leak even when its self-influence looks tiny.
The authors compute the full web of influences among training samples for a small language model. Duplicates indeed cut self-influence while staying nearly extractable from the model. The same pattern surfaced in image classification, where influence maps exposed hidden duplicates in a famous data set. Treating influence as a distribution, not a single score, changes the privacy math. Auditors should hunt for duplicate clusters, not just eager samples.
Catalog record
| CREATOR | Matthieu Meeus, Igor Shilov, Georgios Kaissis, Yves-Alexandre de Montjoye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-20481