Thinking into the Future: Latent Lookahead Training for Transformers
Lorenzo Noci*, Gregor Bachmann, Seyed-Mohsen Moosavi-Dezfooli, Moin Nabi · 2026
Read Thinking into the Future: Latent Lookahead Training for Transformers by Lorenzo Noci*, Gregor Bachmann, Seyed-Mohsen Moosavi-Dezfooli, Moin Nabi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20219.txt.
Description
Noci and co-authors teach transformers to think in latent space before committing to the next token. Next token training forces a choice at every step, with equal compute for easy and hard tokens. Latent lookahead pauses at chosen positions for extra internal steps. Hidden states feed back into the context for several rounds, refining the pending prediction. Those latent predictions are supervised against the coming ground truth tokens. The method beat autoregressive and non autoregressive baselines on planning tasks.
Catalog record
| CREATOR | Lorenzo Noci*, Gregor Bachmann, Seyed-Mohsen Moosavi-Dezfooli, Moin Nabi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50367 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20219