LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation

Ahmadreza Jeddi, Marco Ciccone, Babak Taati, University of Toronto · 2026

Read LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation by Ahmadreza Jeddi, Marco Ciccone, Babak Taati, University of Toronto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11451.txt.

Description

Looped models reason by iterating, but fixed loop counts waste or starve compute. Ahmadreza Jeddi and two co-authors offer LoopFormer, a looped model trained on factor-length paths for budget-conditioned thinking. A shortcut-consistency training scheme aligns paths of other lengths.

Shorter loops thus yield informative forms while longer loops keep refining them. Each loop is conditioned on current time and step size, so forms evolve steadily rather than drifting or stagnating. LoopFormer holds up on language-model and thinking benchmarks under aggressive compute limits and scales gracefully with extra budget. The work opens a path toward controllable, budget-aware language models.

Catalog record

CREATORAhmadreza Jeddi, Marco Ciccone, Babak Taati, University of Toronto
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11451

Browse the full library · About the Tower of Babel