From Growing to Looping: A Unified View of Iterative Computation in LLMs

Ferdinand Kapl Emmanouil Angelis, [0.25em] Kaitlin Maile Johannes von Oswald Stefan Bauer, [0.4em] Technical University of Munich Helmholtz AI, Munich Google, Paradigms of Intelligence Team · 2026

Read From Growing to Looping: A Unified View of Iterative Computation in LLMs by Ferdinand Kapl Emmanouil Angelis, [0.25em] Kaitlin Maile Johannes von Oswald Stefan Bauer, [0.4em] Technical University of Munich Helmholtz AI, Munich Google, Paradigms of Intelligence Team (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-16490.txt.

Description

Ferdinand Kapl leads four coauthors on looped and depth-grown language models. Looping reuses one block of layers across depth, while depth growing trains shallow-to-deep models by copying middle layers. Both tricks have been tied to stronger reasoning, but how they relate was unclear.

The authors find the two share depth-wise signatures, such as heavier use of late layers and repeating patterns tied to the looped or grown block. These point to a shared form of iterative computation. The tricks also combine. Looping the middle blocks of a depth-grown model at run time lifts accuracy on some reasoning tasks by up to twofold, even though the model never trained to loop. Both adapt better than baselines with more examples or tuning data, and depth-grown models gain most from high-quality, math-heavy training mixes.

Catalog record

CREATORFerdinand Kapl Emmanouil Angelis, [0.25em] Kaitlin Maile Johannes von Oswald Stefan Bauer, [0.4em] Technical University of Munich Helmholtz AI, Munich Google, Paradigms of Intelligence Team
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-16490

Browse the full library · About the Tower of Babel