Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks

Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele Natale, [1ex] COATI, CNRS, Inria, I3S, Université Côte d’Azur, France · 2026

Read Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks by Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele Natale, [1ex] COATI, CNRS, Inria, I3S, Université Côte d’Azur, France (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24611.txt.

Introduction

Aakash Kumar and three colleagues build a classical asymmetric Hopfield network with binary neurons and synchronous updates that stores vastly more strings. With n neurons, the construction supports a superpolynomial number of distinct limit-cycle attractors. Each has superpolynomial period, robust to random noise flipping bits with probability approaching one half. The result combines combinatorics, number theory, and opinion dynamics, and is the first such feat for asymmetric Hopfield networks. It shows robust sequence memory can arise from coarse architectural motifs rather than complex nonlinearities.

Catalog record

CREATORAakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele Natale, [1ex] COATI, CNRS, Inria, I3S, Université Côte d’Azur, France
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81418
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.24611

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele Natale, [1ex] COATI, CNRS, Inria, I3S, Université Côte d’Azur, France Abstract Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows n n neurons to support exp ⁡ ( Ω ⁡ ( n / ( log ⁡ n ) 2 ) ) \exp\!\big(\Omega(n/(\log n)^{2})\big) distinct limit-cycle attractors, each with period exp ⁡ ( Ω ⁡ ( n / log ⁡ n ) ) \exp\!\big(\Omega(\sqrt{n}/\log n)\big) and robust to random noise with flip probability up to 1 2 − o ⁡ ( 1 ) \frac{1}{2}-o(1) , yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities. 1 Introduction Associative memory, the ability to retrieve complete patterns from partial or noisy cues, is a fundamental concept in both neuroscience and artificial intelligence. The Hopfield network, a recurrent neural network model, stands as a canonical example of how collective neuronal dynamics can give rise to associative memory ( 8 ) . In its classic formulation, the network is characterized by a symmetric weight matrix. This symmetry guarantees the existence of a Lyapunov energy function, ensuring that the network’s state always converges to a stable fixed point. 1 1 1 If the update rule is synchronous, the network may exhibit limit cycles of period two; however, limit cycles of higher period cannot occur. These fixed points act as stored memories, making the symmetric Hopfield network an effective model for static pattern recognition. However, the constraint of symmetry, while ensuring stability, fundamentally limits the network’s dynamical repertoire to point attractors (see Section 3 ). Many cognitive processes, such as motor control, central pattern generation, and sequential reasoning, are inherently dynamic and involve recalling and generating temporal sequences rather than static patterns. By allowing for asymmetric connections, asymmetric Hopfield networks break the limitations imposed by a global energy function. This asymmetry opens the door to a richer set of dynamics, including the emergence of limit cycles, i.e., periodic sequences of states that can serve as attractors. These limit-cycle attractors, formalized in Section 3 (Definition 2 ), provide a natural mechanism for storing and retrieving temporal sequences. For instance, in biological systems, high-capacity limit cycles can act as robust Central Pattern Generators (CPGs) for complex rhythmic motor control, while in artificial intelligence, they can serve as high-dimensional “clocks” for temporal reasoning and sequence timing. While the potential for asymmetric (or directed) networks to store sequences is known, and various construction methods have been proposed ( 15 ; 13 ) , the question of their storage capacity remains a critical area of investigation. 13 demonstrated that asymmetric Hopfield networks with n neurons can support limit-cycle attractors of length 2^{n} , thereby realizing dynamics that traverse the entire state space. However, this result relies on a highly specialized construction in which the weights are carefully engineered on a case-by-case basis. As a consequence, while theoretically striking, the model is unlikely to provide a realistic account of memory storage in biological or practical settings. A further challenge for such constructions is robustness. In practical settings, one would expect the dynamics to reliably return to the same limit-cycle attractor after perturbations induced by noise. However, if the dynamics traverse the entire state space, robustness is fundamentally unattainable. Recent advances have pushed the capacity of Hopfield-type networks by modifying the standard model with nonlinear interaction terms or continuous states ( 11 ; 14 ; 4 ) . This raises a key question: How can one construct, within the classical synchronous asymmetric Hopfield framework, a model that not only encodes temporal sequences but also supports the reliable storage of a large number of long sequences, while remaining robust to noise and simple enough to be compatible with what could be realized through biological developmental processes? This paper addresses this challenge directly. We introduce a novel construction for asymmetric Hopfield networks based on a very simple network topology, presented in Section 5 . We show that this architecture supports a superpolynomial number of distinct limit-cycle attractors (with respect to the number of neurons), with periods that can themselves be superpolynomial in the network size. Moreover, the proposed construction is inherently robust to even extreme levels of noise, admitting non-trivial basins of attraction around each cycle. Finally, we argue in Section 6 that the model satisfies natural biological desiderata: beyond coarse architectural constraints, the result is provably preserved under highly sparsified network topologies and addition of adversarial connections. Throughout, we work with the parallel update rule introduced by Little ( 12 ) ; we do not analyze the asynchronous single-neuron dynamics more commonly associated with the original symmetric Hopfield model, and extending comparable guarantees to that regime remains open. This should be viewed as a classical variant of the same binary-threshold architecture rather than as a modern modification of the model: Little’s rule uses the same linear local fields and sign activation, differing only in the update schedule. Since our goal is to study deterministic sequence storage through periodic attractors, the synchronous regime is also the natural classical setting in which those limit cycles appear as first-class dynamical objects. We now present an informal statement of our main results. Informal Theorem (Informal statement of Theorems 3 and 4 ) . There exists an asymmetric Hopfield network with n neurons that supports \exp\big(\Omega\big(\frac{n}{(\log n)^{2}}\big)\big) distinct limit-cycle attractors, each with period \exp\big(\Omega(\sqrt{n}/\log n)\big) . Moreover, these limit cycles remain robust under independent flips with probability up to \frac{1}{2}-\Omega\big(\frac{1}{\sqrt{\log n}}\big) , and hence up to \frac{1}{2}-o(1) , (Theorem 3 ) . Furthermore, this robustness extends to highly sparsified network topologies, with only a modest reduction in capacity and period length, both of which remain superpolynomial in n (Theorem 4 ) ; the corresponding quantitative noise thresholds are made explicit in the proofs. We further show in Section 6 (Theorem 5 ) that the construction is robust to addition of adversarial connections in the graph, which strengthens the case for biological plausibility. In Appendix D , we also record complementary global convergence statements: in the dense construction every initial configuration rapidly reaches some limit cycle deterministically, in the sparse construction the same holds with high probability, and under adversarial block perturbations the trajectory reaches the corresponding weak limit cycle. We also provide empirical support to our results through experiments in Section 7 . Our main contributions are summarized below. • Classical Synchronous Asymmetric Hopfield Model: We demonstrate that it is possible to achieve very large sequence memory capacity without altering

Read the complete text