Hopfield Networks as Models of Emergent Function in Biology Journal: Xxxx. Xxx. Xxx. Xxx. DOI: 10.1146/((please add article doi))

Maria Yampolskaya, 1∗ Pankaj Mehta 1,2,3,4,† · 2025

Read Hopfield Networks as Models of Emergent Function in Biology Journal: Xxxx. Xxx. Xxx. Xxx. DOI: 10.1146/((please add article doi)) by Maria Yampolskaya, 1∗ Pankaj Mehta 1,2,3,4,† (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-13076.txt.

Introduction

Maria Yampolskaya and Pankaj Mehta wrote a 2025 review on Hopfield networks as models of how function emerges in biology. Hopfield models began as a theory of memory in neural networks, where stored patterns are retrieved from partial cues. Their math has since proved handy far beyond brains.

The review teaches the models from a biophysics angle. Three readings build intuition: the dynamics as noise filtering, as a geometry that gives pattern space natural coordinates, and as a slide down an energy landscape. The authors then survey recent uses across biology. Applications include cell differentiation and epigenetic memory, molecular self assembly, and the brain's spatial maps. The piece is written as a pedagogical entry point for biologists and physicists alike.

Catalog record

CREATORMaria Yampolskaya, 1∗ Pankaj Mehta 1,2,3,4,†
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-13076

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hopfield Networks as Models of Emergent Function in Biology Journal: Xxxx. Xxx. Xxx. Xxx. DOI: 10.1146/((please add article doi)) Maria Yampolskaya, 1∗ Pankaj Mehta 1,2,3,4,† Abstract Hopfield models, originally developed to study memory retrieval in neural networks, have become versatile tools for modeling diverse biological systems in which function emerges from collective dynamics. In this review, we provide a pedagogical introduction to both classical and modern Hopfield networks from a biophysical perspective. After presenting the underlying mathematics, we build physical intuition through three complementary interpretations of Hopfield dynamics: as noise discrimination, as a geometric construction defining a natural coordinate system in pattern space, and as gradient-like descent on an energy landscape. We then survey recent applications of Hopfield networks a variety of biological setting including cellular differentiation and epigenetic memory, molecular self-assembly, and spatial neural representations. \jvol AA YYYY keywords Hopfield model, modern Hopfield networks, cellular differentiation, self-assembly 1 Introduction From the millions of proteins in a single cell to the billions of neurons in the human brain, collective behavior is responsible for much of the complex phenomena observed in biology [ 20 , 4 , 45 , 48 , 49 , 9 ] . Across scales, biological systems perform specific functions as a result of many parts working together. Modeling the emergence of function from these components is part of what makes biophysics so difficult [ 26 ] . One promising approach for meeting this challenge is to recast this problem in the language of high-dimensional dynamical systems. The most famous and fruitful example of this approach is the Hopfield model [ 24 ] . Originally conceived as a model of memory in neural networks, the Hopfield model has now found broad applicability across biophysics, including understanding the epigenetics of cellular identity and differentiation, modeling the self-assembly of heterogeneous structures, and thinking about neural representations of spatial environments in the hippocampus. The Hopfield model has a long pedigree, and many papers have applied analytical tools from statistical physics to calculate properties of these models, including phase diagrams and storage capacities [ 7 , 6 , 5 , 41 , 24 , 35 , 14 , 37 , 3 ] . Recently, Hopfield models have attracted renewed interest due to their close relationship with modern machine learning techniques such as transformers [ 52 ] (see [ 33 ] for a recent review that highlights these developments). Here, we review the Hopfield networks from a biophysical perspective, focusing on Hopfield models as exemplars of complex systems exhibiting emergent function [ 5 , 25 , 35 , 14 , 69 , 34 , 52 , 32 ] . We explain the intuition behind Hopfield dynamics, why and how these models work, and how they can be used to understand disparate biophysical phenomena. The review is organized as follows. In Section 2 , we present mathematical descriptions of classic and modern Hopfield networks. In Section 3 , we provide three biophysically inspired perspectives for interpreting the dynamics of Hopfield models: (i) as a system that discriminates between signal and noise; (ii) as a geometric operation that projects onto a subspace; and (iii) as gradient-like dynamics on a landscape. We also include Section 3.4 , which provides an overview of other, more commonly, approaches for thinking about Hopfield models. In Section 4 , we turn to applications and describe how Hopfield models have been used to study a variety of biophysical phenomena including cell fate transitions, molecular self-assembly, and neural representations. 2 The Mathematics of Hopfield models 2.1 Overview In this section, we provide a brief self-contained introduction to the mathematics of Hopfield models. We start by introducing the basic mathematical objects used to describe Hopfield models. We then specialize our discussion to three commonly used variants of the Hopfield model: the classic network introduced by Hopfield [ 24 ] , the projection method [ 47 , 31 ] , and the exponential “Modern Hopfield Network” [ 35 , 14 , 52 ] . Despite the many implementational differences between traditional and modern networks, they share a common underlying logic, namely: (i) a dynamical update for the state of the system (ii) and a set of stored patterns that act as attractors (fixed points) of the dynamics. When the dynamics are initialized sufficiently close to a stored pattern, the system flows to the corresponding fixed point, allowing the network to “retrieve” the stored pattern. Hopfield models share a set of common ingredients (see Fig. 1 ): • A N -dimensional state vector \vec{x}(t)=(x_{1}(t),\ldots,x_{N}(t)) whose i -th component x_{i}(t) encodes the state of the variable i -th dynamical variable at a time t . The x_{i} can be binary-valued (e.g., x_{i}=\pm 1 ) or continuous (e.g., x_{i}\in\mathbb{R} ). In the computational neuroscience and physics literatures, these variables are often called “neurons” or “spins”. • A matrix \xi_{\mu i} with dimensions P\times N describing the P patterns we wish to store. By construction, when the system is in a fixed point corresponding to the \nu -th stored pattern, x_{i}=\xi_{\nu i} . In the spin glass literature, these stored patterns are often assumed to be random and independent. However, in many biophysical applications we discuss below, it will be important to consider the case where stored patterns are correlated. • An update rule or differential equation governing the dynamics, e.g. x_{i}(t+1)=f_{i}(\vec{x}(t)) for discrete time or \frac{dx_{i}}{dt}=f_{i}(\vec{x}(t))-x_{i} for continuous time. In the Hopfield model, the functions f_{i}(\vec{x}(t)) are chosen in such a way as to ensure that all P patterns \{\vec{\xi}_{\nu}\} are fixed points of the dynamics when the number of patterns P is sufficiently small compared to the number of neurons N . • A P -dimensional “order parameter” vector \mathbf{m}(t)=(m_{1}(t),\ldots,m_{P}(t)) that measures the alignment of the state of the system \vec{x}(t) at a time t with each of the P stored patterns. As discussed below, describing the systems in terms of order parameter offers a rich lens for interpreting Hopfield dynamics. • An energy function, or Lyapunov function, which decreases with time as a result of the update rule. What distinguishes model variants from each other is how these basic ingredients are combined to construct the detailed network dynamics. Some models consider discrete variables where x_{i}=\pm 1 , whereas others use continuous x_{i} . Another major difference between model variants is how the stored patterns \xi_{\mu i} are constructed. The classic Hopfield model assumes these are binary random vectors whereas other constructions relax this assumption. The biggest difference between model variants is the choice of non-linear function f_{i}(\vec{x}(t)) that defines the update rule. Here, we focus on three choices for this function corresponding to the classical Hopfield model, the projection method, and Modern Hopfield networks. Figure 1: Interpretations of Hopfield models. A. Hopfield models involve networks of neurons, and the update rule for each neuron depends on the state of every other neuron. In classic Hopfield networks, each neuron has a binary value, and neurons are connected all-to-all with pairwise interactions. B. Hopfield models are also called associative memory models because they compare the system state, \vec{x} , to a list of memories, or stored patterns, and retrieve the memory that has the most association with the initial state. States are vectors in neuron space. In

Read the complete text