Step-resolved data attribution for looped transformers

Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou · 2026

Read Step-resolved data attribution for looped transformers by Georgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10097.txt.

Description

Training-data influence scores usually collapse a whole recurrent computation into one number, hiding when an example mattered. A 2026 paper led by Georgios Kaissis, with four co-authors, studies looped transformers, where one shared block repeats for many iterations to enable latent reasoning. The team introduces Step-Decomposed Influence, which unrolls the recurrent graph and attributes influence to specific loop iterations.

A TensorSketch implementation makes the method practical at transformer scale without ever materializing per-example gradients. Tests on looped GPT-style models and algorithmic puzzles show the method scales well. It matches full-gradient baselines closely and supports varied attribution and interpretability tasks with per-step insight into latent reasoning.

Catalog record

CREATORGeorgios Kaissis, David Mildenberger, Juan Felipe Gomez, Martin J. Menten, Eleni Triantafillou
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE125 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-10097

Browse the full library · About the Tower of Babel