Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference

Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti · 2026

Read Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference by Sangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26099.txt.

Description

Lee, McLeish, Goldstein and Fanti give language models a sleep-like consolidation step. Transformer attention scales poorly with long context, so recent context is periodically turned into persistent fast weights, and the key-value cache is cleared. In sleep, offline recurrent passes over stored context update fast weights in state-space blocks by a learned local rule. Wake-time prediction keeps normal latency, with extra work shifted to sleep. Gains peak where deeper reasoning is needed, including a math task where baselines fail.

Catalog record

CREATORSangyun Lee, Sean McLeish, Tom Goldstein, Giulia Fanti
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42966
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26099

Browse the full library · About the Tower of Babel