Hyperparameter Transfer for Dense Associative Memories

Roi Holtzman, Dmitry Krotov, Boris Hanin · 2026

Read Hyperparameter Transfer for Dense Associative Memories by Roi Holtzman, Dmitry Krotov, Boris Hanin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10164.txt.

Description

Roi Holtzman, Dmitry Krotov, and Boris Hanin address a practical headache in training large AI memory systems. Tuning settings on a huge model is far too costly. Their 2026 paper develops hyperparameter transfer for Dense Associative Memories. These are energy-based networks that store patterns as low points on an energy landscape and generalize the classic Hopfield model.

Dense Associative Memories share weights across layers and use sharply peaked activation functions. Those traits break the transfer recipes built for ordinary feed-forward networks. The authors derive explicit rules for carrying settings tuned on small models up to large ones. Their experiments agree closely with the theory. Readers get a principled shortcut for scaling these memory networks without paying full tuning costs at every size.

Catalog record

CREATORRoi Holtzman, Dmitry Krotov, Boris Hanin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE91 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10164

Browse the full library · About the Tower of Babel