Simply Stabilizing the Loop via Fully Looped Transformer
Rao Fu, Zixuan Yang, Jiankun Zhang, Jing Ma, Hechang Chen, Yu Li, Yi Chang, [0.5em] Hong Kong Baptist University, Jilin University · 2026
Read Simply Stabilizing the Loop via Fully Looped Transformer by Rao Fu, Zixuan Yang, Jiankun Zhang, Jing Ma, Hechang Chen, Yu Li, Yi Chang, [0.5em] Hong Kong Baptist University, Jilin University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-18797.txt.
Description
Rao Fu and six colleagues stabilize the Looped Transformer, which reuses the same blocks to gain power without adding weights. Reuse lets users trade compute for quality at test time, but training grows unstable as loop counts rise. The authors trace the trouble to gradient swings and residual growth. Their Fully Looped Transformer spreads signals across all layers and reuses attention to damp swings, with no new weights. Training then stays stable at higher loop counts.
Catalog record
| CREATOR | Rao Fu, Zixuan Yang, Jiankun Zhang, Jing Ma, Hechang Chen, Yu Li, Yi Chang, [0.5em] Hong Kong Baptist University, Jilin University |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53407 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.18797