Sufficient conditions for offline reactivation in recurrent neural networks

Nanda H Krishna, Colin Bredenberg, Daniel Levenstein, Blake Aaron Richards, Guillaume Lajoie, [2pt] Mila -- Quebec AI Institute, [2pt] {nanda.harishankar-krishna,colin.bredenberg,guillaume.lajoie}@mila.quebec · 2025

Read Sufficient conditions for offline reactivation in recurrent neural networks by Nanda H Krishna, Colin Bredenberg, Daniel Levenstein, Blake Aaron Richards, Guillaume Lajoie, [2pt] Mila -- Quebec AI Institute, [2pt] {nanda.harishankar-krishna,colin.bredenberg,guillaume.lajoie}@mila.quebec (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17003.txt.

Introduction

Nanda Krishna and four co-authors ask when brain-like networks replay their activity during rest, in this 2025 paper. During sleep and quiet moments, real brain circuits revisit patterns they used while awake. Modelers call this offline reactivation, and the conditions that let it emerge were poorly understood.

The team built a mathematical frame for recurrent networks trained to track smoothly changing stimuli from noisy change signals. They prove such networks naturally develop denoising dynamics: left alone, with no input, the same dynamics pull the network back through the states it used online. Computer tests on two classic tasks, tracking position from self-motion and head direction from turning speed, matched the theory. The work backs the idea that replay falls out of good task design, not special machinery.

Catalog record

CREATORNanda H Krishna, Colin Bredenberg, Daniel Levenstein, Blake Aaron Richards, Guillaume Lajoie, [2pt] Mila -- Quebec AI Institute, [2pt] {nanda.harishankar-krishna,colin.bredenberg,guillaume.lajoie}@mila.quebec
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17003

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sufficient conditions for offline reactivation in recurrent neural networks Nanda H Krishna, Colin Bredenberg, Daniel Levenstein, Blake Aaron Richards, Guillaume Lajoie, [2pt] Mila -- Quebec AI Institute, [2pt] {nanda.harishankar-krishna,colin.bredenberg,guillaume.lajoie}@mila.quebec Abstract During periods of quiescence, such as sleep, neural activity in many brain circuits resembles that observed during periods of task engagement. However, the precise conditions under which task-optimized networks can autonomously reactivate the same network states responsible for online behavior is poorly understood. In this study, we develop a mathematical framework that outlines sufficient conditions for the emergence of neural reactivation in circuits that encode features of smoothly varying stimuli. We demonstrate mathematically that noisy recurrent networks optimized to track environmental state variables using change-based sensory information naturally develop denoising dynamics, which, in the absence of input, cause the network to revisit state configurations observed during periods of online activity. We validate our findings using numerical experiments on two canonical neuroscience tasks: spatial position estimation based on self-motion cues, and head direction estimation based on angular velocity cues. Overall, our work provides theoretical support for modeling offline reactivation as an emergent consequence of task optimization in noisy neural circuits. 1 Introduction It has been widely observed that neural circuits in the brain recapitulate task-like activity during periods of quiescence, such as sleep ( Tingley & Peyrache 2020 ) . For example, the hippocampus “replays” sequences of represented spatial locations akin to behavioral trajectories during wakefulness ( Nádasdy et al. 1999 ; Lee & Wilson 2002 ; Foster 2017 ) . Furthermore, prefrontal ( Euston et al. 2007 ; Peyrache et al. 2009 ) , sensory ( Kenet et al. 2003 ; Xu et al. 2012 ) , and motor ( Hoffman & McNaughton 2002 ) cortices reactivate representations associated with recent experiences; and sleep activity in the anterior thalamus ( Peyrache et al. 2015 ) and entorhinal cortex ( Gardner et al. 2022 ) is constrained to the same neural manifolds that represent head direction and spatial position in those circuits during wakefulness. This neural reactivation phenomenon is thought to have a number of functional benefits, including the formation of long term memories ( Buzsáki 1989 ; McClelland et al. 1995 ) , abstraction of general rules or “schema” ( Lewis & Durrant 2011 ) , and offline planning of future actions ( Ólafsdóttir et al. 2015 ; Igata et al. 2021 ) . Similarly, replay in artificial systems has been shown to be valuable in reinforcement learning, when training is sparse or expensive ( Mnih et al. 2015 ) , and in supervised learning, to prevent catastrophic forgetting in continual learning tasks ( Hayes et al. 2019 ) . However, where machine learning approaches tend to save sensory inputs from individual experiences in an external memory buffer, or use external networks that are explicitly trained to generate artificial training data ( Hayes & Kanan 2022 ) , reactivation in the brain is autonomously generated in the same circuits that operate during active perception and action. Currently, it is unknown how reactivation can emerge in the same networks that encode information during active behavior, or why it is so widespread in neural circuits. Previous approaches to modeling reactivation in neural circuits fall into two broad categories: generative models that have been explicitly trained to reproduce realistic sensory inputs ( Deperrois et al. 2022 ) , and models in which replay is an emergent consequence of the architecture of network models with a particular connectivity structure ( Shen & McNaughton 1996 ; Azizi et al. 2013 ; Milstein et al. 2023 ) or local synaptic plasticity mechanism ( Hopfield 2010 ; Litwin-Kumar & Doiron 2014 ; Theodoni et al. 2018 ; Haga & Fukai 2018 ; Asabuki & Fukai 2023 ) . Other approaches have modeled replay and theta sequences as emergent consequences of firing rate adaptation ( Chu et al. 2024 ) or input modulation ( Kang & DeWeese 2019 ) in continuous attractor network models. Generative modeling approaches have strong theoretical guarantees that reactivation will occur, because networks are explicitly optimized to provide this functionality. However, modeling approaches that argue for emergent reactivation typically rely on empirical results, and lack rigorous mathematical justification. In this study, we demonstrate that a certain type of reactivation—diffusive reactivation—can emerge from a system attempting to optimally encode features of its environment in the presence of internal noise. We trained continuous-time recurrent neural networks (RNNs) to optimally integrate and track perceptual variables based on sensations of change (motion through space, angular velocity, etc.), in the context of two ethologically relevant tasks: spatial navigation and head direction integration. Critically, training in this manner has been shown to produce grid cell ( Cueva & Wei 2018 ; Sorscher et al. 2019 ) and head direction cell representations ( Cueva et al. 2020 ; Uria et al. 2022 ) , which correspond to neural systems in which reactivation phenomena have been observed ( Gardner et al. 2022 ; Peyrache et al. 2015 ) . We see that these networks exhibit reactivation during quiescent states (when subject to noise but in the absence of perceptual inputs), and we explain these phenomena by demonstrating that noise compensation dynamics naturally induce diffusion on task-relevant neural manifolds in optimally trained networks. Figure 1: Reactivation in a spatial position estimation task . a) Schematic of task. b) Test metrics as a function of training batches for the place cell tuning mean squared error loss used during training (gray) and for the position decoding spatial distance error (black). c) Explained variance as a function of the number of principal components (PCs) in population activity space during task engagement (blue) and during quiescent noise-driven activity (red). d) Sample decoded outputs during active behavior. Circles indicate the initial location, triangles indicate the final location. e) Same as (d), but for decoded outputs during quiescence. f) Neural activity projected onto the first two PCs during the active phase. Color intensity measures the decoded output’s distance from the center in space. g) Neural activity during the quiescent phase projected onto the same active PC axes as in (f). h) Two-dimensional kernel density estimate (KDE) plot measuring the probability of state-occupancy over 200 decoded output trajectories during active behavior. i-k) Same as (h), but for decoded outputs during quiescence (i), neural activity projected onto the first two PCs during the active phase (j), and during the quiescent phase (k). Error bars (b-c) indicate \pm 1 standard deviation over five networks. 2 Mathematical Results 2.1 Setup and variables In this study, we will consider a noisy discrete-time approximation of a continuous-time RNN, receiving change-based information \frac{\mathrm{d}\mathbf{s}(t)}{\mathrm{d}t} about an N_{s} -dimensional environmental state vector \mathbf{s}(t) . The network’s objective will be to reconstruct some function of these environmental state variables, f\big(\mathbf{s}(t)\big):\mathbb{R}^{N_{s}}\rightarrow\mathbb{R}^{N_{o}} , where N_{s} is the number of stimulus dimensions and N_{o} is the number of output dimensions (for a schematic, see Fig. 1 a). An underlying demand for this family of tasks is that path integration needs to be performed, possibly followed by some computations based on that integration. These requirements are often met in natural settings, as it is widely believed that animals are able to estimate their location in space \mathbf{s}(t) through path integration based exclusively on

Read the complete text