Local Pheromone Network: Sparse Local Learning with Multi-Scale Synaptic Trails, Consolidation, and Replay
Fu Xingcheng, Chen Xianjun, Li Zhihao · 2026
Read Local Pheromone Network: Sparse Local Learning with Multi-Scale Synaptic Trails, Consolidation, and Replay by Fu Xingcheng, Chen Xianjun, Li Zhihao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30669.txt.
Introduction
Fu Xingcheng, Chen Xianjun and Li Zhihao describe the Local Pheromone Network, a small prototype that learns without backpropagation. Standard networks train all parameters together through global error signals, so new tasks can overwrite old associations. Their network is sparse and local instead. Each output unit reads only a neighbourhood of inputs, chosen by geometric distance and molecular-tag compatibility, like smells matching receptors.
Each synapse stores a weight plus short-term and long-term pheromone traces, chemical-style markers that fade or persist. Training uses pheromone-weighted Hebbian-style updates on a budgeted subset of synapses picked by local error. The budget adapts: it shrinks when loss improves and grows toward active neighbourhoods when loss worsens. Optional pieces add structural plasticity, replay and consolidation. In tests the prototype learned local linear rules, kept split memories separate through tags and masks, and forgot less under consolidation and replay.
Catalog record
| CREATOR | Fu Xingcheng, Chen Xianjun, Li Zhihao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30669
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Local Pheromone Network: Sparse Local Learning with Multi-Scale Synaptic Trails, Consolidation, and Replay Fu Xingcheng, Chen Xianjun, Li Zhihao Abstract Backpropagation-trained dense neural networks are powerful function approximators, but they couple learning across many parameters and can overwrite previous associations when tasks conflict. This paper describes Local Pheromone Network, a small research prototype for sparse, local, manually updated neural networks. In Local Pheromone Network, each output unit reads only a fixed local neighborhood of input units subject to geometric distance and molecular-tag compatibility. Each synapse stores a weight, a short-term pheromone trace, a long-term pheromone trace, and an optional consolidation state. Training does not call automatic differentiation. Instead, every layer performs a pheromone-weighted Hebbian-style update on a budgeted subset of local synapses selected from local error and co-activity. The update budget adapts online: it shrinks when loss improves and expands toward recently active neighborhoods when loss worsens. Optional mechanisms add structural plasticity, local replay, output masks for partitioned learning, and a target-free local contrastive step. We present the implementation, learning rule, and preliminary experiments on synthetic regression, partitioned memory, conflicting memory, consolidated conflict, structural plasticity, replay, and a synthetic long-context hybrid memory task. The prototype learns local linear rules, preserves partitioned memories through tags and masks, reduces forgetting under consolidation, and uses replay under conflict. Keywords: Sparse local learning; long-term memory; pheromone-based learning; synaptic consolidation; brain-region partitioning; continual learning; structural plasticity; replay. 1 Introduction Most contemporary neural networks use dense or structured differentiable modules trained by global backpropagation. This has produced remarkable results, but it also couples learning across wide parameter sets: a new task may adjust parameters needed by an older task, causing catastrophic forgetting [ 5 , 4 ] . Biological neural systems suggest a different collection of constraints: synapses are local, neuromodulatory signals alter plasticity, frequently used pathways become easier to reuse, and some memories become consolidated over time [ 2 , 1 ] . Local Pheromone Network explores a simple computational analogue of those ideas. The model is not intended as a biologically faithful account of neural learning. Instead, it is an engineering prototype that asks: what happens if a neural layer stores only local sparse synapses, routes learning through pheromone-like traces, updates only a small budget of synapses, and protects mature high-pheromone connections from being overwritten? The implementation studied here is contained in local_pheromone_network.py . It provides two main objects: • LocalPheromoneLayer , a sparse layer with tag-restricted local synapses and pheromone traces. • LocalPheromoneNetwork , a stack of such layers trained by local_train_step rather than loss.backward() . The central design choices are: 1. each output unit connects to at most max_neighbors compatible input units; 2. compatibility is determined by spatial locality and integer tags; 3. forward propagation uses a pheromone-gated effective weight; 4. learning selects only a budgeted subset of synapses per output unit; 5. short- and long-term pheromone traces evolve at different time scales; 6. consolidated synapses become less plastic; 7. replay can reapply old local examples when a new update appears to conflict with previous memory. The contributions of this manuscript are descriptive rather than benchmark-driven. We formalize the implemented model, document its mechanisms, and report preliminary behavior from the accompanying tests and evaluation scripts. 2 Related Work The local update rule is inspired by Hebbian learning [ 2 ] , but differs by using an explicit local error signal and a budgeted synapse-selection mechanism. Associative memory has a long history in neural computation, including Hopfield networks [ 3 ] . Local Pheromone Network models are not energy-based attractor networks, but they share the goal of storing persistent associations in weights. The consolidation mechanism is related in spirit to complementary learning systems and elastic synaptic models of memory preservation [ 4 , 1 ] . However, Local Pheromone Network consolidation is per-local-synapse and driven by pheromone maturity, not Fisher information or gradient-derived parameter importance. Local replay follows the general idea that old examples can reduce forgetting, but uses the same manual local update rule rather than gradient replay. The model also touches structural plasticity and sparse neural computation. Instead of training a dense matrix and later pruning, the layer is sparse by construction: each output unit stores only a local list of candidate synapses. Optional structural plasticity can prune low-pheromone low-consolidation synapses and sprout tag-compatible new synapses in active neighborhoods. 3 Model 3.1 Units, Geometry, and Tags Let a layer receive an input vector x\in\mathbb{R}^{d_{\mathrm{in}}} and produce an output y\in\mathbb{R}^{d_{\mathrm{out}}} . Each input and output unit has a normalized position in a one- or two-dimensional geometry. If a shape is provided, such as (r,c) , units are placed on a grid; otherwise they are placed on a one-dimensional line. Each unit also has an integer tag. An input unit i may be considered as a candidate for output unit j only if |t^{\mathrm{out}}_{j}-t^{\mathrm{in}}_{i}|\leq\Delta_{t}, (1) where \Delta_{t} is tag_distance . If a connection radius is provided, the candidate should also lie within that geometric radius. The implementation falls back to nearest tag-compatible inputs if no candidate exists inside the radius, but it does not fall back to tag-incompatible inputs. For each output j , the layer selects at most K=\texttt{max\_neighbors} nearest compatible inputs. These indices are stored in N_{j,k}\in\{1,\ldots,d_{\mathrm{in}}\}, (2) with a boolean mask M_{j,k} indicating valid slots. The trainable state is therefore sparse: W\in\mathbb{R}^{d_{\mathrm{out}}\times K}, (3) rather than a dense d_{\mathrm{out}}\times d_{\mathrm{in}} matrix. 3.2 Pheromone-Gated Forward Pass Each synapse has a short-term pheromone s_{j,k} , a long-term pheromone p_{j,k} , and a consolidation value c_{j,k} . The effective pheromone is \phi_{j,k}=\frac{\alpha_{s}s_{j,k}+\alpha_{p}p_{j,k}}{\alpha_{s}+\alpha_{p}}, (4) when \alpha_{s}+\alpha_{p}>0 , otherwise it defaults to the long-term pheromone. In the implementation these coefficients are short_pheromone_weight and long_pheromone_weight . The forward pass first gathers local inputs \tilde{x}_{j,k}=x_{N_{j,k}}. (5) The pheromone gate normalizes pheromone mass per output row: g_{j,k}=0.5+0.5\cdot\mathrm{clip}\left(\frac{\phi_{j,k}}{\sum_{k^{\prime}}\phi_{j,k^{\prime}}+\epsilon}\sum_{k^{\prime}}M_{j,k^{\prime}},0,2\right). (6) The layer output is then y_{j}=b_{j}+\sum_{k=1}^{K}M_{j,k}\,W_{j,k}\,g_{j,k}\,\tilde{x}_{j,k}. (7) Thus pheromone does not replace weights; it modulates which local pathways matter more during inference. 3.3 Network Stack A LocalPheromoneNetwork stacks one or more layers: h^{(0)}=x,\quad z^{(\ell)}=L^{(\ell)}(h^{(\ell-1)}),\quad h^{(\ell)}=\sigma(z^{(\ell)}) (8) for hidden layers, with a linear final output. Supported activations are tanh , relu , sigmoid , gelu , and identity . All parameters have requires_grad=False ; learning is manual. 4 Local Learning Rule 4.1 Loss and Output Mask The supervised local update uses mean squared error. An optional output mask m restricts the active output region: \mathcal{L}=\frac{\sum_{i}m_{i}(\hat{y}_{i}-y_{i})^{2}}{\sum_{i}m_{i}}. (9) This is useful for partitioned memory: a task can update only its assigned output region instead of treating unrelated regions as zero targets. 4.2 Budget Adaptation The network maintains a current update budget B , the maximum number of synapses per output unit to update. Let \mathcal{L}_{t} be the current loss and \mathcal{L}_{t-1} the previous loss. The mode is: • warmup : no previous loss exists; • exploit : \mathcal{L}_{t}<\mathcal{L}_{t-1}-\epsilon ; shrink B by shrink_factor ; • neighbor-follow : \mathcal{L}_{t}>\mathcal{L}_{t-1}+\epsilon ; grow B by grow_factor ; • steady : otherwise. The budget is clipped between configured minimum and maximum values. 4.3 Layer Error and Synaptic Signal At the output, the local error is e=y-\hat{y}. (10) For hidden layers the implementation propagates an approximate feedback signal through the effective local weights and activation derivatives. This is not automatic differentiation and not claimed to be a biologically