Tiny Autoregressive Recursive Models
Paulius Rauba, Claudio Fanconi, Mihaela van der Schaar · 2026
Read Tiny Autoregressive Recursive Models by Paulius Rauba, Claudio Fanconi, Mihaela van der Schaar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-08082.txt.
Description
Rauba and colleagues ask whether the refinement trick behind Tiny Recursive Models can move into autoregressive models. Tiny Recursive Models have shone on ARC-AGI, with very small models rivaling large models through two-step refinement of an internal reasoning state and the predicted output. Yet they lack causal structure and carry lasting hidden states, so gains are hard to pin down. The authors build an Autoregressive TRM, test it on small autoregressive tasks, and compare it with baselines that morph a standard Transformer toward it step by step.
Catalog record
| CREATOR | Paulius Rauba, Claudio Fanconi, Mihaela van der Schaar |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46505 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.08082