Overcoming Quadratic Hardware Scaling for a Fully Connected Digital Oscillatory Neural Network
Bram Haverkort, Aida Todri-Sanial · 2025
Read Overcoming Quadratic Hardware Scaling for a Fully Connected Digital Oscillatory Neural Network by Bram Haverkort, Aida Todri-Sanial (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20680.txt.
Introduction
Networks of coupled oscillators can compute in parallel while sipping energy. Bram Haverkort and Aida Todri-Sanial study digital versions of these oscillatory neural networks. The standard design links every oscillator to every other. That full wiring makes the coupling hardware grow with the square of the network size. Their hybrid design mixes serial and parallel coupling to tame the growth. Hardware then scales almost linearly, at about 1.2 times per added oscillator.
On a common FPGA board the design fit 506 fully connected oscillators. That is a tenfold jump over the recurrent baseline at the same precision. The authors call it the largest such digital network built so far.
Catalog record
| CREATOR | Bram Haverkort, Aida Todri-Sanial |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-20680
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Overcoming Quadratic Hardware Scaling for a Fully Connected Digital Oscillatory Neural Network Bram Haverkort, Aida Todri-Sanial Abstract . Computing with coupled oscillators or oscillatory neural networks (ONNs) has recently attracted a lot of interest due to their potential for massive parallelism and energy-efficient computing. However, to date, ONNs have primarily been explored either analytically or through analog circuit implementations. This paper shifts the focus to the digital implementation of ONNs, examining various design architectures. We first report on an existing digital ONN design based on a recurrent architecture. The major challenge for scaling such recurrent architectures is the quadratic increase in coupling hardware with the network size. To overcome this challenge, we introduce a novel hybrid architecture that balances serialization and parallelism in the coupling elements that shows near-linear hardware scaling, on the order of about 1.2 with the network size. Furthermore, we evaluate the benefits and costs of these different digital ONN architectures in terms time to solution and resource usage on FPGA emulation. The proposed hybrid architecture allows for a 10.5 × \times increase in the number of oscillators while using 5-bits to represent the coupling weights and 4-bits to represent the oscillator phase on a Zynq-7020 FPGA board. The near-linear scaling is a major step towards implementing large scale ONN architectures. To the best of our knowledge, this work presents the largest fully connected digital ONN architecture implemented thus far with a total of 506 fully connected oscillators. 1. Introduction With the growing computational demands and the rise of AI, we are witnessing a significant increase in the power consumption of computing systems, and this growth is becoming unsustainable, particularly with the widespread deployment of smart systems. One of the main reasons for this trend is the computing architecture based on the von Neumann paradigm, which has inherent limitations due to the separation of memory and processing units, leading to memory and power wall bottlenecks. To overcome these limitations, significant efforts in recent years are focused on neuromorphic computing architectures, such as spiking neural networks (SNNs) ( Maass 1997 ; Yao et al. 2024 ) . Inspired by biological neural networks for their energy efficiency, these efforts aim to develop architectures that eliminate the separation between memory and processing, moving towards an in-memory computing paradigm and thereby reducing the need for data shuffling between memory and processing units. In this work, we present a novel physics-inspired computing paradigm based on coupled oscillators. This approach not only eliminates the separation between memory and processing but also enables energy-efficient computing by harnessing complex dynamics to achieve massive parallelism, distinguishing it from other neuromorphic computing paradigms ( Todri-Sanial et al. 2024 ) . Coupled oscillator networks are typically structured as recurrent networks that perform computations in phase and frequency domain contrary to other neural networks that compute in time domain. Coupled oscillators or Oscillatory Neural Networks (ONNs) originate from Hopfield neural networks (HNNs) ( Hopfield 1982 ; Ramsauer et al. 2021 ) , illustrated in Figure 1 . Figure 1. Illustration of oscillatory neural network for pattern retrieval. Pattern retrieval process where each pixel in the pattern is represented with an oscillator. The coupling weights in the network are determined using a learning rule. A corrupted pattern is injected into a network as initial condition. The oscillator phases naturally evolve in parallel to the memorized pattern. Unlike Hopfield Neural Networks (HNNs) that use binary neurons (e.g., spin-up +1, spin-down -1), ONNs can surpass binary limitations. In ONNs, information is encoded in phase, providing additional degrees of freedom to represent data within a phase space. For instance, a spin-up state can be represented by a 0° phase, while a spin-down state corresponds to a 180° phase. Additionally, finer phase differences can encode even greater amounts of information. Like HNNs, ONNs are energy-minimizing networks that inherently perform associative memory tasks, such as pattern recognition ( Hoppensteadt and Izhikevich 2000 ; Abernot et al. 2021 ; Abernot et al. 2023 ; Delacour et al. 2021 ) . ONNs have also been adapted and shown promising performance for image classification ( Abernot and Todri-Sanial 2023 ; Sabo and Todri-Sanial 2024 ) . Typically, these applications are tackled using convolutional neural networks ( Stanojevic et al. 2024 ; Syed et al. 2021 ) and spiking neural networks ( Byerly et al. 2021 ; Wu and He 2021 ) . However, these networks often feature high complexity, using many layers and parameters, with numbers in the tens and millions, respectively ( Simonyan and Zisserman 2015 ) . Additionally, the energy minimization of coupled networks of oscillators can be described by the Ising model. This model applied to ONNs has garnered significant interest in recent years for designing oscillatory Ising machines, which can be utilized to solve various complex optimization problems, such as max-cut, graph coloring, and more ( Vaidya et al. 2022 ; Delacour et al. 2023 ; Wang and Roychowdhury 2019 ; Wang et al. 2021 ; Bashar and Shukla 2023 ; Bashar et al. 2024 ; Sundara Raman et al. 2024 ; Cılasun et al. 2024 ; Nikhar et al. 2024 ; Bashar et al. 2021 ) . Since the number of oscillators in a network required to solve a certain problem is determined by the dataset or exact problem that needs to be embedded, this motivates the need for larger ONNs. For example, solving the max-cut problem on a graph requires each graph node to be represented by one oscillator, or in the case of associative memory tasks, each oscillator is typically used to represent one pixel in a pattern. Additionally, the network topology is also related to the ease of problem embedding. Sparse network topologies such as nearest-neighbor or King’s graphs do not support the embedding of all problems and can require an additional computational step to map a problem. However, architectures using sparse networks typically have a higher number of nodes compared to architectures with fully-connected networks since sparse networks require fewer connections compared to a fully-connected topology. Typically, ONNs are designed in the analog domain to emulate the complex dynamics of coupled oscillators. Recently, the first digital ONN implementation was reported ( Abernot et al. 2021 ; Abernot and Aida 2023 ; Abernot and Todri-Sanial 2023 ) , featuring a recurrent ONN architecture that represents the dynamics of digital square-wave oscillators. This recurrent architecture has been implemented in Field Programmable Gate Arrays (FPGA) and demonstrated ONNs for performing tasks such as pattern retrieval and edge detection. While ONNs have been successfully demonstrated on FPGA for their potential in edge computing, a significant limitation is the quadratic increase in the number of coupling elements as the network size grows. Fully connected small ONNs are feasible, but scaling up to larger networks for solving realistic problems remains a challenge. In this work, for the first time, we propose a novel ONN computing architecture that is not recurrent but a hybrid architecture that enables near-linear network scalability instead of quadratic scalability. This hybrid architecture can be used to accelerate computing on the edge, or scaled up to solve large combinatorial optimization problems. The rest of this paper is structured as follows. Section 2 provides background on the computing principles of ONNs, state-of-the-art implementations