Traveling Waves Integrate Spatial Information Through Time

Mozes Jacobs Roberto C. Budzinski Lyle Muller Demba Ba T. Anderson Keller · 2025

Read Traveling Waves Integrate Spatial Information Through Time by Mozes Jacobs Roberto C. Budzinski Lyle Muller Demba Ba T. Anderson Keller (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-06034.txt.

Introduction

Mozes Jacobs, Roberto Budzinski, Lyle Muller, Demba Ba, and Anderson Keller show, in a 2025 preprint, that traveling waves can give artificial networks a sense of the whole picture. In real brains, waves of activity sweep across tissue, but their job is unclear. One guess is that they carry information across space over time. The authors borrow a hint from acoustics: a drum's tone modes encode its shape. Wave patterns can hold global layout in a readable form.

Their networks learn to send waves through their hidden states when shown an image. Those wave patterns then serve as the representation for the task. On image labeling jobs that demand global context, wave models beat local feed-forward ones. They rivaled a strong non-local design while using fewer parameters. The waves widen each neuron's effective view, the authors argue. The work also hints at a bridge to brain recordings.

Catalog record

CREATORMozes Jacobs Roberto C. Budzinski Lyle Muller Demba Ba T. Anderson Keller
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE66 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-06034

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Traveling Waves Integrate Spatial Information Through Time Mozes Jacobs Roberto C. Budzinski Lyle Muller Demba Ba T. Anderson Keller Abstract Abstract Traveling waves of neural activity are widely observed in the brain, but their precise computational function remains unclear. One prominent hypothesis is that they enable the transfer and integration of spatial information across neural populations. However, few computational models have explored how traveling waves might be harnessed to perform such integrative processing. Drawing inspiration from the famous “ Can one hear the shape of a drum? ” problem – which highlights how normal modes of wave dynamics encode geometric information – we investigate whether similar principles can be leveraged in artificial neural networks. Specifically, we introduce convolutional recurrent neural networks that learn to produce traveling waves in their hidden states in response to visual stimuli, enabling spatial integration. By then treating these wave-like activation sequences as visual representations themselves, we obtain a powerful representational space that outperforms local feed-forward networks on tasks requiring global spatial context. In particular, we observe that traveling waves effectively expand the receptive field of locally connected neurons, supporting long-range encoding and communication of information. We demonstrate that models equipped with this mechanism solve visual semantic segmentation tasks demanding global integration, significantly outperforming local feed-forward models and rivaling non-local U-Net models with fewer parameters. As a first step toward traveling-wave-based communication and visual representation in artificial networks, our findings suggest wave-dynamics may provide efficiency and training stability benefits, while simultaneously offering a new framework for connecting models to biological recordings of neural activity. Keywords: Traveling Waves; Oscillation; Information Integration Introduction The propagation of traveling waves of neural activity has been measured on the surface of the brain from the earliest neural recordings ( Adrian & Matthews, (1934) ; Goldman et al., (1949) ; Lilly, (1949) ; Mickle & Ades, (1953) ) . Such waves have been measured to travel both locally and globally across cortical regions with a range of velocities ( Reimer et al., (2010) ; Muller et al., (2016) ; Zhang et al., (2018) ) . Stimulus-evoked traveling waves have been directly measured in visual cortex ( Cowey, (1964) ) with increasingly sophisticated methodology from penetrating electrodes ( Ebersole & Kaplan, (1981) ) to voltage-sensitive dye imaging ( Muller et al., (2014) ) , including in awake behaving primates ( Davis et al., (2020) ) . Driven by these observations, many theoretical arguments have been put forth to explain the functional roles of these dynamics. Examples include that they are relevant to predictive coding ( Alamia & VanRullen, (2019) ) , the representation of symmetries ( Keller et al., (2024) ) , the consolidation of long-term memories ( Muller et al., (2018) ) , and the encoding of motion ( Heitmann & Ermentrout, (2020) ) . Figure 1: Overview of traveling wave-based spatial information integration . An input stimulus triggers an initial condition and sets the response properties of a lattice of neurons with both local input receptive fields and recurrent connectivity. This initial condition evolves over time under the recurrent wave dynamics, and the resulting timeseries at each neuron becomes a globally integrated representation of the visual stimuli. Most relevant to this study, one often hypothesized role is that traveling waves serve as a mechanism for integration and transfer of information over long distances – a mechanism that is believed to play an important role specifically within visual cortex ( Sato et al., (2012) ) . For example, Kitano et al. ((1994)) demonstrated early on that local-field potential (LFP) responses of neurons in primary visual cortex could be elicited by stimuli far outside their classic retinotopic receptive fields with increased latency as a function of distance, implying a long-range distance-delayed integration of information. These findings were later reinforced by the intracellular subthreshold membrane potential recordings of Bringuier et al. ((1999)) , denoting this extended receptive field the ‘visually evoked synaptic integration field’. However, the hypothesized role of information transfer and integration extends beyond visual stimuli. For example, Rubino et al. ((2007)) found that beta frequency oscillations propagated spatially across the motor cortices of monkeys in preparation for movement, and that information about the visual target was directly encoded in these waves. Similarly, Besserve et al. ((2015)) used direction-specific causal information transfer metrics to demonstrate that traveling waves in the gamma frequency band are correlated with information transfer between different cortical regions; while Bhattacharya et al. ((2022)) showed that waves change direction during information retrieval and processing in a working memory task. Despite these promising observations however, it remains challenging to investigate these ideas computationally due to a lack of task-trainable artificial neural network models which exhibit traveling wave dynamics. In modern artificial neural networks, information is integrated and transmitted over spatial distances of an input (e.g. an image) or between ‘tokens’ of a sequence either via extremely deep convolutional neural networks ( He et al., (2015) ) , bottleneck/pooling layers ( Ronneberger et al., (2015) ) , or all-to-all connectivity as-in Transformers ( Vaswani et al., (2023) ) . Each of these approaches comes with its own computational complexity and expressivity limitations, and it is therefore of great interest to explore alternative methods to integrate disparate information in neural systems. In this paper, we aim to make progress towards understanding the causal role of wave dynamics in the transfer of information by filling this modeling gap, and exploring the computational potential of wave-based models in task-relevant settings. To begin, we take inspiration from the famous mathematical question “ Can one hear the shape of a drum ", and explore if the techniques underlying this problem, namely the representation of global information through stationary solutions to wave-based dynamical systems, can be equivalently applied to extract global information from locally-connected recurrent neural network hidden states over time. In first part of this paper, we begin with a review of this problem and the associated formalism, outlining how we may construct trainable recurrent neural networks to leverage these ideas as a computational principle. On toy tasks, we demonstrate that these simple models do indeed match theoretical predictions (Figure 2 ), and that when slightly relaxed to allow for more flexible input encoding, they generate wave dynamics which enable the disentanglement of simple shapes in frequency space (Figures 3 & 4 ). In the second part of this paper, we use this intuition as motivation to build a suite of further relaxed, yet more computationally capable, convolutional recurrent neural networks (conv-RNNs), with inherently limited receptive field sizes in both their initial encoders and recurrent connections, and test them on more complex global-information processing (semantic segmentation) tasks. We demonstrate how by using the timeseries of each neuron’s recurrent neural activity as our primary neural representation during training (schematized in Figure 1 ), such models are able to outperform other locally constrained models, and even rival the performance of some deeper convolutional models such as U-Nets, while often converging more consistently to favorable solutions – ultimately suggesting that traveling wave-based information integration may be an efficient and stable alternative to existing deep neural network spatial integration techniques. Motivation: Hearing the Shape of a Drum To build intuition for how traveling waves may integrate information over space, we take

Read the complete text