MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning Conference: Proceedings of the 23rd ACM International Conference on Computing Frontiers; May 19–21, 2026; Catania, Italy Proceedings of the 23rd ACM International Conference on Computing Frontiers (CF ’26), May 19–21, 2026, Catania, Italy DOI: 10.1145/3801487.3801812 ISBN: 979-8-4007-2568-5/2026/05
Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu · 2026
Read MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning Conference: Proceedings of the 23rd ACM International Conference on Computing Frontiers; May 19–21, 2026; Catania, Italy Proceedings of the 23rd ACM International Conference on Computing Frontiers (CF ’26), May 19–21, 2026, Catania, Italy DOI: 10.1145/3801487.3801812 ISBN: 979-8-4007-2568-5/2026/05 by Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20586.txt.
Description
Liu and co-authors propose Memory-Keyed Attention for long context language models. Large key value caches make training and use costly. Their design stacks three cache tiers: local, session, and long term. A learned router picks where attention should look. A fused variant, FastMKA, merges sources before the attention step. It matched MLA on perplexity in their tests. Training ran up to five times faster, with lower latency at test time.
Catalog record
| CREATOR | Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40272 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20586