EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel · 2025
Read EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts by Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-14280.txt.
Description
Subhajit Chaudhury and co-authors give language models a memory for long texts in 2025. Models struggle to use very long contexts well, even when the text fits in memory. Their method stores the context in an episodic memory module that attends to relevant chunks as a whole.
The memory output then reweights the model's own attention to its stored context. That happens both in training and while generating replies. Across recall and question tasks from sixteen thousand to a quarter million tokens, the method beat plain models. It also beat popular retrieval-based frames on robustness across lengths.
Catalog record
| CREATOR | Subhajit Chaudhury, Payel Das, Sarathkrishna Swaminathan, Georgios Kollias, Elliot Nelson, Khushbu Pahwa, Tejaswini Pedapati, Igor Melnyk, Matthew Riemer [email protected], [email protected], [email protected],[email protected], [email protected], [email protected], [email protected],igor.mel |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-14280