Tiny Autoregressive Recursive Models

Paulius Rauba, Claudio Fanconi, Mihaela van der Schaar · 2026

Read Tiny Autoregressive Recursive Models by Paulius Rauba, Claudio Fanconi, Mihaela van der Schaar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08082.txt.

Description

Rauba and colleagues ask whether the refinement trick behind Tiny Recursive Models can move into autoregressive models. Tiny Recursive Models have shone on ARC-AGI, with very small models rivaling large models through two-step refinement of an internal reasoning state and the predicted output. Yet they lack causal structure and carry lasting hidden states, so gains are hard to pin down. The authors build an Autoregressive TRM, test it on small autoregressive tasks, and compare it with baselines that morph a standard Transformer toward it step by step.

Catalog record

CREATORPaulius Rauba, Claudio Fanconi, Mihaela van der Schaar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46505
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.08082

Browse the full library · About the Tower of Babel