EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts

Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel · 2025

Read EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts by Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14280.txt.

Description

Subhajit Chaudhury and co-authors give language models a memory for long texts in 2025. Models struggle to use very long contexts well, even when the text fits in memory. Their method stores the context in an episodic memory module that attends to relevant chunks as a whole.

The memory output then reweights the model's own attention to its stored context. That happens both in training and while generating replies. Across recall and question tasks from sixteen thousand to a quarter million tokens, the method beat plain models. It also beat popular retrieval-based frames on robustness across lengths.

Catalog record

CREATORSubhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-14280

Browse the full library · About the Tower of Babel