Towards Compressive and Scalable Recurrent Memory

Yunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute · 2026

Read Towards Compressive and Scalable Recurrent Memory by Yunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-11212.txt.

Description

Focus costs explode with context length, and recurrent memory often trades principle for scale. Yunchong Song and four co-authors propose Elastic Memory, a design grounded in the HiPPO frame for online function approximation. History is treated as samples from continuous signals, compressed into a fixed-size state.

A polynomial sampling step rebuilds history summaries from that state for recall. On long-context sets across three domains, Elastic Memory beat rivals, including a memorizing model by a wide memory margin. It stayed ahead as model size grew and ran faster than a rival at larger scale. Its decoupled design also allows test-time biases to boost scores further.

Catalog record

CREATORYunchong Song 2 , Jushi Kai 1 , Liming Lu 1 , Kaixi Qiu 1 , Zhouhan Lin 1,2,3‡ 1 LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University 2 Shanghai Artificial Intelligence Laboratory 3 Shanghai Innovation Institute
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-11212

Browse the full library · About the Tower of Babel