Towards Compressive and Scalable Recurrent Memory
Yunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute · 2026
Read Towards Compressive and Scalable Recurrent Memory by Yunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11212.txt.
Description
Focus costs explode with context length, and recurrent memory often trades principle for scale. Yunchong Song and four co-authors propose Elastic Memory, a design grounded in the HiPPO frame for online function approximation. History is treated as samples from continuous signals, compressed into a fixed-size state.
A polynomial sampling step rebuilds history summaries from that state for recall. On long-context sets across three domains, Elastic Memory beat rivals, including a memorizing model by a wide memory margin. It stayed ahead as model size grew and ran faster than a rival at larger scale. Its decoupled design also allows test-time biases to boost scores further.
Catalog record
| CREATOR | Yunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-11212