Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers

Kazuki Irie Morris Yau Samuel J. Gershman · 2025

Read Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers by Kazuki Irie Morris Yau Samuel J. Gershman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00744.txt.

Description

Kazuki Irie, Morris Yau, and Samuel J. Gershman propose a 2025 design that blends two kinds of machine memory. One kind stores pairs and finds them with great care, but its cost grows fast with length. The other kind changes its own links as data flows in, handles long runs well, but recalls past facts with less care.

The authors set out three ways to join the two kinds into one shared store. Each way sends new facts to the two stores on a distinct plan. Their tests weigh how well the blends recall facts and do hard steps, and they show how a mixed store can gain from both parents at once.

Catalog record

CREATORKazuki Irie Morris Yau Samuel J. Gershman
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00744

Browse the full library · About the Tower of Babel