Stability and Generalization in Looped Transformers

Asher Labovich · 2026

Read Stability and Generalization in Looped Transformers by Asher Labovich (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15259.txt.

Description

Labovich studies looped transformers, networks that repeat one layer to spend extra compute on harder problems, and asks which design choices let them generalize instead of memorizing training tricks. His fixed-point framework tracks reachability, input sensitivity, and geometry, showing that memory plus outer normalization yields stable, reachable fixed points. Tests on chess, sudoku, and prefix sums match the theory, and a new internal memory placement beats standard placement on sudoku.

Catalog record

CREATORAsher Labovich
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82020
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15259

Browse the full library · About the Tower of Babel