Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference
Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti · 2026
Read Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference by Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26099.txt.
Description
Lee, McLeish, Goldstein and Fanti give language models a sleep-like consolidation step. Transformer attention scales poorly with long context, so recent context is periodically turned into persistent fast weights, and the key-value cache is cleared. In sleep, offline recurrent passes over stored context update fast weights in state-space blocks by a learned local rule. Wake-time prediction keeps normal latency, with extra work shifted to sleep. Gains peak where deeper reasoning is needed, including a math task where baselines fail.
Catalog record
| CREATOR | Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42966 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.26099