Spiking Neural Belief Propagation Decoder for LDPC Codes with Small Variable Node Degrees
Alexander von Bank, Eike-Manuel Edelmann, Jonathan Mandelbaum, and Laurent Schmalen · 2024
Read Spiking Neural Belief Propagation Decoder for LDPC Codes with Small Variable Node Degrees by Alexander von Bank, Eike-Manuel Edelmann, Jonathan Mandelbaum, and Laurent Schmalen (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-15897.txt.
Introduction
Alexander von Bank and colleagues present an improved decoder for error-correcting codes that runs on brain-like spiking networks. Their paper, from twenty twenty-four, concerns low-density parity-check codes, widely used to fix errors in transmitted data. An earlier decoder from the same research line used a single spiking network to handle part of the decoding math. That shortcut worked poorly when each variable node in the code connects to only a few others, because the rough estimate lost too much detail.
The new design runs several spiking networks side by side for the same step, giving finer resolution and a wider range of message values. On a large test code, the improved decoder matched the performance of a standard widely used decoder. The result keeps alive the hope of decoding with much lower energy use, since spiking hardware works only when events occur.
Catalog record
| CREATOR | Alexander von Bank, Eike-Manuel Edelmann, Jonathan Mandelbaum, and Laurent Schmalen |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-15897
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Spiking Neural Belief Propagation Decoder for LDPC Codes with Small Variable Node Degrees Thanks: This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No. 101001899). Parts of this work were carried out in the framework of the CELTIC-NEXT project AI-NET-ANTILLAS (C2019/3-3) (grant agreement 16KIS1316) and within the project Open6GHub (grant agreement 16KISK010) funded by the German Federal Ministry of Education and Research (BMBF). Alexander von Bank, Eike-Manuel Edelmann, Jonathan Mandelbaum, and Laurent Schmalen Abstract Spiking neural networks (SNNs) promise energy-efficient data processing by imitating the event-based behavior of biological neurons. In previous work, we introduced the enlarge-likelihood-each-notable-amplitude spiking-neural-network (ELENA-SNN) decoder, a novel decoding algorithm for low-density parity-check (LDPC) codes. The decoder integrates SNNs into belief propagation (BP) decoding by approximating the check node (CN) update equation using SNNs. However, when decoding LDPC codes with a small variable node (VN) degree, the approximation gets too rough, and the ELENA-SNN decoder does not yield good results. This paper introduces the multi-level ELENA-SNN (ML-ELENA-SNN) decoder, which is an extension of the ELENA-SNN decoder. Instead of a single SNN approximating the CN update, multiple SNNs are applied in parallel, resulting in a higher resolution and higher dynamic range of the exchanged messages. We show that the ML-ELENA-SNN decoder performs similarly to the ubiquitous normalized min-sum decoder for the ( 38400 , 30720 ) (38400,\,30720) regular LDPC code with a VN degree of d v = 3 d_{v}=3 and a CN degree of d c = 15 d_{\mathrm{c}}=15 . I Introduction Low-density parity-check (LDPC) codes are forward error-correcting codes that have been integrated into various modern communication standards, such as 5G and WLAN, due to their excellent error-correction capabilities when using iterative message-passing decoding, often known as belief propagation (BP) decoding. In particular, the sum-product algorithm (SPA) together with LDPC codes can achieve performance close to the channel capacity [ 1 ] . However, the SPA involves computationally demanding check node (CN) updates [ 2 ] . To reduce this complexity, simplified methods like the min-sum (MS) algorithm [ 3 ] , the normalized MS (NMS) algorithm [ 4 ] , and differential decoding with binary message passing (DD-BMP) [ 5 ] have been proposed. In [ 6 ] , it was shown that suppressing small values during MS decoding can boost performance. By integrating memory into the decoding procedure, the decoding performance of the SPA and MS decoder can be further enhanced [ 7 ] . One of the most powerful and efficient processing units is the human brain, solving challenging real-world problems while only consuming roughly 25\text{\,}\mathrm{W} of power. Neuromorphic engineering tries to build scalable, low-power systems by implementing the essential features of the brain, allowing an efficient emulation of brain-inspired neural networks [ 8 ] . Most recent neuromorphic hardware emulates spiking neural networks (SNNs) [ 9 , 10 ] , which mimic the event-based behavior of biological neurons [ 11 , pp. 12,13] . In [ 12 ] , we introduced the enlarge-likelihood-each-notable-amplitude spiking-neural-network (ELENA-SNN) decoder, which approximates computational heavy parts of the SPA decoder by SNNs . While a threshold operation suppresses messages with small amplitudes in an all-or-nothing manner, spiking neurons introduce memory to the decoder; both can enhance the decoding performance. In [ 12 ] , two binary (N,k) LDPC codes, which encode k data bits into N code bits, were used: the (273,\,191) and (1023,\,781) finite-geometry (FG) LDPC codes. For both codes, we showed that the ELENA-SNN decoder achieves approximately the performance of the NMS decoder with successive relaxation (SR) , outperforming other complexity-reduced versions of the SPA decoder and even outperforming the SPA decoder in high SNR regimes [ 12 ] . To the best of our knowledge, the ELENA-SNN decoder is the first SNN -based decoder. However, we observed that the ELENA-SNN decoder yields poor results when decoding LDPC codes with low variable node (VN) degrees. The combination of the threshold operation and the low VN degrees leads to a low dynamic range and low resolution of the messages. This paper expands the ELENA-SNN decoder to the multi-level ELENA-SNN (ML-ELENA-SNN) decoder. Multiple thresholds are used instead of a single threshold, introducing a finer granularity of the message levels and a wider dynamic range of the messages. For an LDPC code with low VN degree d_{\mathrm{v}}=3 , we show that the ML-ELENA-SNN decoder performs close to the NMS decoder, while the ELENA-SNN decoder does not show competitive performance. The code for reproducing the results is available at [ 13 ] . II Spiking Neural Networks SNNs are networks of interconnected, spiking neurons with a state which try to mimic the event-based signal processing of the mammalian brain. The exchange of information between the neurons consists of short, uniform pulses, so-called spikes. A sequence of spikes s(t)\in\{0,1\} is called a spike train, where t\in\mathbb{R} is the time. Like the biological neuron, a spiking neuron receives inputs, i.e., spike trains, from upstream connected neurons, processes those, and generates an output spike train s_{\mathrm{out}}(t) [ 11 , p. 13] . The connection between two neurons is called a synapse and possesses a weight, which defines the strength of the connection [ 11 , p. 14] . Hence, each incoming spike train s_{j}(t) is scaled by a weight {w_{j}\in\mathbb{R}} , where s_{j}(t) denotes the spike train received by the j -th upstream connected neuron and w_{j} models the connecting synapse. The incoming spike trains induce the synaptic current i(t) , which then charges the internal state of the neuron, the membrane potential v(t) . In parallel, the neuron leaks towards its initial state v_{\mathrm{rest}} . Hence, the neuron can be described as a leaky integrator of its input signals [ 14 ] . If v(t) exceeds a fixed threshold v_{\mathrm{th}} , the neuron emits an output spike, and the membrane potential v(t) is reset to v_{\mathrm{rest}} . A widely used spiking neuron model is the leaky integrate-and-fire (LIF) neuron model since it captures the fundamental concepts of the biological neuron while being computationally attractive. At discrete time instants t=\kappa\Delta t,\,\kappa\in\mathbb{N} , the discrete dynamics of the LIF neuron are [ 14 ] \displaystyle v[\kappa+1] \displaystyle=v[\kappa]\cdot\mathrm{e}^{-\frac{\Delta t}{\tau_{\mathrm{m}}}}+i[\kappa]\cdot\mathrm{e}^{-\frac{\Delta t}{\tau_{\mathrm{m}}}}\,, \displaystyle i[\kappa+1] \displaystyle=i[\kappa]\cdot\mathrm{e}^{-\frac{\Delta t}{\tau_{\mathrm{s}}}}+\sum_{j}s_{j}[\kappa]w_{j}\,, where \Delta t is the sampling time of the system, \tau_{\mathrm{m}} denotes the time constant describing the leakage of the membrane potential, and \tau_{\mathrm{s}} describes the temporal dynamics of the processes of the synapse. Output spikes are generated according to \displaystyle s_{\mathrm{out}}[\kappa]=\Theta\left(v[\kappa]-v_{\mathrm{th}}\right)=\begin{cases}1,&\text{if}\quad v[\kappa]>v_{\mathrm{th}}\,,\\ 0,&\text{otherwise}\,,\end{cases} where \Theta(\cdot) denotes the Heaviside step function. If an output spike is generated, the state of the neuron is reset, i.e., {v[\kappa]\leftarrow v_{\mathrm{rest}}} . Fig. 1 shows an exemplary realization of the LIF neuron dynamics. Another common spiking neuron model is the leaky integrate (LI) neuron model. It exhibits the same temporal dynamics as the LIF neuron; however, the spiking functionality is deactivated. Hence, it does not generate an output spike, and the LI neuron acts as a temporary memory. In this paper, we use the PyTorch-based framework Norse [ 15 ] for the simulation of SNNs . t threshold v_{\mathrm{th}} s_{\mathrm{in}}(t) i(t) v(t) s_{\mathrm{out}}(t) Fig. 1 : Example of the dynamics of an LIF neuron. The input spikes s_{\mathrm{in}}(t) ,