Grow-Prune-Freeze Networks: Adaptive & Continual Learning Technique for Olfactory Navigation
Kordel K. France, Ovidiu Daescu · 2026
Read Grow-Prune-Freeze Networks: Adaptive & Continual Learning Technique for Olfactory Navigation by Kordel K. France, Ovidiu Daescu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-25170.txt.
Introduction
Kordel K. France and Ovidiu Daescu introduce Grow-Prune-Freeze networks, which let a robot keep learning as its world changes. Early policy layers expand, shrink, or lock in place in response to complexity. The design extends single-hidden-layer random matrix theory to continual learning across many layers, preserving how weight eigenvalues combine as layers are added. On turbulent plume navigation, a version based on Expected SARSA succeeds 94 percent of the time, and evidence suggests the idea transfers to games, vision, and language models.
Catalog record
| CREATOR | Kordel K. France, Ovidiu Daescu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58622 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.25170
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Grow-Prune-Freeze Networks: Adaptive & Continual Learning Technique for Olfactory Navigation Kordel K. France, Ovidiu Daescu Abstract Training data for olfaction is scattered through disparate, non-standardized datasets that limit the ability to build representative world models. Olfactory navigation is a highly dynamic and non-stationary task that benefits from real-time continual learning. We introduce an adaptive framework called Grow-Prune-Freeze (GPF) networks that enable an agent to continually learn through growing, pruning, and freezing early layers of its policy in response to world complexity. Grounding GPFs in non-linear random matrix theory, we show that the work of 31 can be extended from single hidden layers to n n -layer continual-learning models, and that eigenvalue composition of network weights is preserved as successive layers are added. We show that GPFs based on Expected SARSA achieve a 94% success rate on turbulent plume navigation—a partially observable, non-stationary task representative of the “big world” challenges that motivate adaptive learning in robotics—and provide supporting methodology for applying GPFs in other world models. Further experiments amount evidence that GPFs may generalize well to other machine learning tasks such as reinforcement learning in Atari, image classification, and autoregressive language models. We open source all code and data to encourage improvements on and more research in olfactory robotics. 1 Introduction Figure 1 : Conceptualization of a Grow-Prune-Freeze (GPF) neural network. Modern deep learning architectures are typically static, limiting their applicability in dynamic and non-stationary tasks. In addition, many of these modern models are based on transformers trained on petabytes of internet data. The luxury of big data does not exist among the unstandardized and data-starved world of olfaction which discourages work in representative world models. Furthermore, molecular dynamics cause an environment to constantly evolve, and some environments are more dynamic than others which correspondingly warrant more compute in order to correlate molecular patterns within the air. Tasks like olfactory navigation significantly benefit from runtime learning. As a result, we became drawn toward machine learning methods that could continuously update their own priors based on batches of observed data, and adjust their own architectures according to the observed data complexity. These desires led us to develop Grow-Prune-Freeze (GPF) networks , grounded in known work from random matrix theory and scaled from first-principles mathematics. Our motivation is to develop a useful method for robotic runtime learning in partially observable world models, with a specific emphasis on olfactory navigation: a robot must locate a chemical source by following a turbulent odor plume in sparse-signal conditions using an edge-deployable policy-gradient model. Growing layer-by-layer enables a network to adapt capacity to plume complexity, pruning keeps the model compact for onboard inference, and freezing preserves temporal dynamics learned from early plume encounters to hedge against catastrophic forgetting. We selected foundational temporal difference learning and pruning methods to emphasize that the performance observed in our experiments is credited to GPFs rather than an exotic learning architecture. In further support of GPFs as a generalized method, we elaborate on their intuitive extension to other learning tasks in the Supplementary Material. 2 Grow-Prune-Freeze Networks Continual learning requires adapting indefinitely to non-stationary streams while maintaining stability (retention of prior knowledge) and plasticity (ability to learn new tasks). Standard deep learning suffers catastrophic plasticity loss in continual settings as shown by 6 , and while adaptive pruning methods such as those in 20 ; 14 show promise, they lack theoretical grounding for long-term spectral stability. Grow-Prune-Freeze neural networks offer a framework that automatically (1) grows hidden layers when validation loss plateaus; (2) prunes “soft” neurons according to temporal belief values; and (3) freezes stable layers to prevent forgetting while preserving plasticity. We root GPFs in random matrix theory (RMT) and analogize the mathematics to theories of deep learning architectures. Specifically, we show that the work of 31 can be extended from single-layer neural networks to n -layer continuous-learning deep neural networks that incrementally add layers as they learn while preserving eigenvalue spectrality. Through this, we also explore how a GPF’s low-rank perturbations maintain Marchenko-Pastur spectral densities with the appropriate shape, ensuring dynamical isometry and gradient propagation. We then show that principles from 37 ; 25 hold as a result. The final model partially addresses the three plasticity correlates identified by 6 : (1) weight magnitude growth, (2) dead units, and (3) effective rank collapse. 2.1 Relevant Background Random matrix theory is the bedrock from which GPFs were inspired and built. 31 derive spectral densities for non-linear activations in basic neural networks with a single hidden layer, and 4 link low-rank bias to generalization. 25 cover RMT as it pertains to deep learning by showing how model width and iid weight assumptions can be checked and controlled. We extend these works to non-equilibrium ensembles induced through the the continual adding of layers in multilayer perceptrons. Dynamic Sparse Training (DST) by 20 alternates pruning and regrowth of neural networks, while MorphNet from 15 optimizes sparsity via regularization. Growing methods like DEN from 38 add neurons according to loss plateaus. General principles for constructing adaptive neural networks are recently enumerated by 27 and backed by empirical support on recent model architectures. Elastic Weight Consolidation from 19 highlights weight-specific penalization during learning; Progressive Nets by 32 ; 33 expand layers laterally to support additional inputs. 6 demonstrate plasticity loss across optimizers and propose continual backpropagation (CBP) via selective reinitialization. Additional work from 23 and 24 denote other challenges in building adaptive architectures that continually learn. GPFs build on top of principles established in each of these works by using random matrix theory as the grounding ideology and Columnar Constructive Netwroks established by 16 as the inspiration for continually growing layers. 8 show grow-and-prune methods may be empirically effective but lack theoretical characterization of how structural adaptation affects network dynamics. Magnitude-based pruning from 21 , the Lottery Ticket Hypothesis by 14 , and parameter freezing from 9 all give supporting evidence that sparse, stabilized networks preserve critical knowledge. In GPFs , pruning and freezing are integrated with growth to form a continuous cycle of adaptive architecture. We select LeCun’s magnitude-based pruning as the specific method for our evaluation. 3 Theoretical Analysis The GPF framework assumes a very simple neural network to start: an input layer, an output layer, and a single hidden layer. Consider a feedforward network with L_{t}=1 layer at time t=0 , input dimension n_{0} , hidden widths n_{1},\dots,n_{L_{t}-1} , and output n_{L_{t}} . Weights W^{\ell}\in\mathbb{R}^{n_{\ell}\times n_{\ell-1}} are initialized from Kaiming distribution. GPFs grow depth only as their environment demands it, preserving the eigenvalue spectrality of previously learned weights. To facilitate this, we define a threshold \omega_{l} within the validation loss. As it learns, if the network experiences no change in loss above this threshold, a new hidden layer \ell_{i} is added, where i>0,i\in\mathbb{R} . At time k , there is a “patience” variable \rho_{k} such that there must be \rho_{k} epochs with validation loss all below this threshold in order to add a new layer. The above can be denoted as follows: Add layer \ell_{i}=L_{t}+1 if validation loss \mathcal{L}_{\text{val}} satisfies: \mathcal{L}_{\text{val}}^{(t-k)}-\mathcal{L}_{\text{val}}^{(t)}<\omega_{l},\quad\forall i\in[t-k+1,t], (1) where \omega_{l}>0 is the stagnation threshold. Each neuron’s input weight w_{i,j}^{\ell} has belief h_{i,j}^{\ell}\in[0,1) . This belief value is such that if the absolute value of a neuron’s weight is