MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning Conference: Proceedings of the 23rd ACM International Conference on Computing Frontiers; May 19–21, 2026; Catania, Italy Proceedings of the 23rd ACM International Conference on Computing Frontiers (CF ’26), May 19–21, 2026, Catania, Italy DOI: 10.1145/3801487.3801812 ISBN: 979-8-4007-2568-5/2026/05

Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu · 2026

Read MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning Conference: Proceedings of the 23rd ACM International Conference on Computing Frontiers; May 19–21, 2026; Catania, Italy Proceedings of the 23rd ACM International Conference on Computing Frontiers (CF ’26), May 19–21, 2026, Catania, Italy DOI: 10.1145/3801487.3801812 ISBN: 979-8-4007-2568-5/2026/05 by Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20586.txt.

Description

Liu and co-authors propose Memory-Keyed Attention for long context language models. Large key value caches make training and use costly. Their design stacks three cache tiers: local, session, and long term. A learned router picks where attention should look. A fused variant, FastMKA, merges sources before the attention step. It matched MLA on perplexity in their tests. Training ran up to five times faster, with lower latency at test time.

Catalog record

CREATORDong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40272
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.20586

Browse the full library · About the Tower of Babel