Step-resolved data attribution for looped transformers
Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou · 2026
Read Step-resolved data attribution for looped transformers by Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10097.txt.
Description
Training-data influence scores usually collapse a whole recurrent computation into one number, hiding when an example mattered. A 2026 paper led by Georgios Kaissis, with four co-authors, studies looped transformers, where one shared block repeats for many iterations to enable latent reasoning. The team introduces Step-Decomposed Influence, which unrolls the recurrent graph and attributes influence to specific loop iterations.
A TensorSketch implementation makes the method practical at transformer scale without ever materializing per-example gradients. Tests on looped GPT-style models and algorithmic puzzles show the method scales well. It matches full-gradient baselines closely and supports varied attribution and interpretability tasks with per-step insight into latent reasoning.
Catalog record
| CREATOR | Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 125 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-10097