Teaching signal synchronization in deep neural networks with prospective neurons
Nicolas Zucchet, Qianqian Feng, Axel Laborieux, Friedemann Zenke, Walter Senn, João Sacramento, [1.5em] Computer Science Department, ETH Zürich · 2025
Read Teaching signal synchronization in deep neural networks with prospective neurons by Nicolas Zucchet, Qianqian Feng, Axel Laborieux, Friedemann Zenke, Walter Senn, João Sacramento, [1.5em] Computer Science Department, ETH Zürich (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14917.txt.
Introduction
Nicolas Zucchet, Qianqian Feng, and four co-authors show how brain-like networks can keep teaching signals in step with delayed activity. Working memory needs neurons that hold a trace of the recent past, but stacking such slow neurons layer on layer delays their signals. Feedback about right and wrong answers then arrives aimed at the wrong moment in the activity stream.
The authors give neurons an adaptive current that makes them respond ahead of time, predicting their coming inputs. With that prospective behavior, slow layers can still learn from late feedback. The mechanism mirrors a known trait of real neurons and offers a design rule for memory-capable artificial networks that learn online.
Catalog record
| CREATOR | Nicolas Zucchet, Qianqian Feng, Axel Laborieux, Friedemann Zenke, Walter Senn, João Sacramento, [1.5em] Computer Science Department, ETH Zürich |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 97 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-14917
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Teaching signal synchronization in deep neural networks with prospective neurons Nicolas Zucchet, Qianqian Feng, Axel Laborieux, Friedemann Zenke, Walter Senn, João Sacramento, [1.5em] Computer Science Department, ETH Zürich Abstract Working memory requires the brain to maintain information from the recent past to guide ongoing behavior. Neurons can contribute to this capacity by slowly integrating their inputs over time, creating persistent activity that outlasts the original stimulus. However, when these slowly integrating neurons are organized hierarchically, they introduce cumulative delays that create a fundamental challenge for learning: teaching signals that indicate whether behavior was correct or incorrect arrive out-of-sync with the neural activity they are meant to instruct. Here, we demonstrate that neurons enhanced with an adaptive current can compensate for these delays by responding to external stimuli prospectively – effectively predicting future inputs to synchronize with them. First, we show that such prospective neurons enable teaching signal synchronization across a range of learning algorithms that propagate error signals through hierarchical networks. Second, we demonstrate that this successfully guides learning in slowly integrating neurons, enabling the formation and retrieval of memories over extended timescales. We support our findings with a mathematical analysis of the prospective coding mechanism and learning experiments on motor control tasks. Together, our results reveal how neural adaptation could solve a critical timing problem and enable efficient learning in dynamic environments. \declaretheorem [name=Theorem]thm \declaretheorem [name=Remark]remark Understanding how the brain learns and adapts to new situations is a central question in neuroscience. Effective learning requires associating current neural activity with the delayed consequences of behavior, whether from inherent factors such as delayed rewards or processing delays within the brain. Such processing delays are accentuated by the brain’s hierarchical organization, where different areas specialize in various aspects of information processing [ 24 ] . Indeed, working memory requires neurons to slowly integrate their inputs over time to maintain information from the recent past, but these slow dynamics introduce processing delays that accumulate across hierarchical levels. This creates a fundamental problem for learning: when teaching signals – whether external rewards or internal error signals – arrive at output-related areas, the neural activity in earlier layers is already out of sync due to cumulative processing delays, creating a temporal mismatch between the activity that should be instructed and the teaching signals guiding learning. Error backpropagation [ 61 , 50 ] has emerged as the canonical algorithm for solving the spatial credit assignment problem in deep artificial neural networks, with numerous recent propositions exploring its potential biological implementations [ 31 , 32 , 52 , 62 , 48 , 35 , e.g.,] . However, these models typically assume instantaneous transmission, failing to account for the slow dynamics of the neurons responsible for working memory. To address this limitation, researchers have proposed solutions including forward models [ 37 , 64 , 16 ] , eligibility traces [ 21 , 15 , 65 , 8 ] , and prospective neurons [ 36 , 19 , 54 ] . In this work, we focus on the latter, examining how prospective neurons can mitigate delays, both theoretically and practically, and investigating how adaptive neurons can serve as practical implementations of such prospective mechanisms. We structure the paper as follows. First, we focus on the theoretical foundations of prospective neurons by formalizing delay as a deviation from an ideal trajectory that needs to be tracked – specifically, what neural activity would look like if no processing delays existed. Through this lens, we demonstrate that prospective neurons can effectively circumvent delays after an initial warm-up phase, whereas standard slowly integrating neurons prove insufficient. Next, we explore adaptive neurons as physically realistic implementations of such prospective neurons, analyzing how this physical implementation impacts both tracking and learning performance. Finally, we validate our theoretical insights on a range of behaviorally relevant motor control learning problems, including scenarios with delayed rewards that require tuning the memory storage process. Our findings reveal that the tracking ability of prospective neurons enables effective teaching signal synchronization, thus facilitating online learning in neural networks and providing insights into how biological systems might overcome internal signal propagation delays. 1 Theoretical properties of prospective neurons Biological neural networks comprise dynamical neurons that integrate information over time, with leaky integrators [ 1 ] standing as the archetypal model for such neurons. The temporal integration of neural signals presents a critical challenge, particularly for neurons responsible for propagating errors, as it results in a misalignment between neural activities and their corresponding learning signals. In this section, we first formalize such delays as deviations from a reference trajectory obtained under an idealized, instantaneous system. We then analyze how the network structure, neuronal characteristics, and external stimuli properties interplay to influence these deviations. Finally, we investigate prospective dynamics as a potential solution to mitigate temporal delays. 1.1 Teaching signal synchronization as a tracking problem While our ideas apply to a broad class of plasticity rules, we focus our exposition on gradient-based learning and the backpropagation algorithm, as it is the canonical algorithm for learning deep neural networks. In this context, teaching signals are error signals derived from an objective function. Let us consider a feedforward neural network whose neurons’ voltages u^{l} at layer l are determined by u^{l+1}=W^{l+1}\rho(u^{l})~~\mathrm{and}~u^{0}=x (1) where \rho is a nonlinear activation function, W^{l+1} the synaptic strength matrix, and x the input of the network, which we consider fixed for now. The voltage of the last layer, u^{L} , is taken to be the output of the network and is compared with a target y , through the loss function \ell(u^{L},y) . Error signals \delta are backpropagated through the network hierarchy with \delta^{l}=\rho^{\prime}(u^{l}){W^{l+1}}^{\top}\delta^{l+1}~~\mathrm{and}~\delta^{L}=\nabla\ell(u^{L},y). (2) Those equations follow from defining the gradient of the loss with respect to the postsynaptic voltages, \delta^{l}:=\nabla_{u^{l}}\,\ell , and recursively applying the chain rule to this quantity. When using the mean squared error loss, the error signal for the output units becomes \delta^{L}=u^{L}-y . Error signals can then be combined with voltages to obtain the gradient following update \Delta W^{l+1}\propto-\nabla_{W^{l}}\,\ell(u^{L},y)=-\delta^{l+1}{\rho(u^{l})}^{\top} . While these equations provide a theoretically grounded learning rule, they ignore temporal dynamics: voltages and errors are assumed to propagate instantaneously through the network hierarchy. It can be justified by a separation of timescales argument: the input x and target output y evolve at a slower rate than the neurons, so that they can be considered constant. Through the instantaneous nature of signal processing in this model, error signals always arrive on time. However, realistic models of learning should take into account scenarios in which external stimuli evolve faster than the response time of the considered neural network. The separation of timescales argument mentioned above thus no longer holds. We must take into account the temporal dynamics of neural activity. Here, we study a standard leaky integrator model, in which (u_{t},\delta_{t}) satisfies the differential equation \begin{split}\tau\dot{u}_{t}&=-u_{t}+W\rho(u_{t})~~\mathrm{and}~~u_{t}^{0}=x_{t}\\ \tau\dot{\delta}_{t}&=-\delta_{t}+\rho^{\prime}(u_{t})W^{\top}\delta_{t}~~\mathrm{and}~~\delta^{L}_{t}=\nabla\ell(u^{L}_{t},y_{t})\end{split} (3) where \dot{u} (resp. \dot{\delta} ) stands for the temporal derivative of u (resp. \delta ), and \tau is the neuronal integration time constant, which we consider to be the same for all u and \delta for simplicity. For conciseness, we have concatenated the voltages u^{l} (resp. \delta^{l} ) of all layers into a single vector u (resp. \delta ). The matrix W contains all the W^{l}