On the Optimality of Single-label and Multi-label Neural Network Decoders
Yunus Can Gültekin , Péter Scheepers, Yuncheng Yuan, Federico Corradi , and Alex Alvarado · 2025
Read On the Optimality of Single-label and Multi-label Neural Network Decoders by Yunus Can Gültekin , Péter Scheepers, Yuncheng Yuan, Federico Corradi , and Alex Alvarado (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18758.txt.
Introduction
Yunus Can Gültekin and four co-authors settle a question about neural decoders in a paper from two thousand twenty-five. Neural networks have been proposed to decode error-correcting codes, with single-label and multi-label designs each claiming near-optimal results on short codes. Whether they can truly match the best possible decoding was open.
The authors prove that certain versions of both designs can always realize optimal decoding, for any code. Their architectures are fixed by the codebook itself, with binary weights and no training needed. Tests on three classic short codes confirm optimal performance with simpler designs than earlier proposals. A catch remains: the approach explodes in size for longer codes. Optimality, they show, is achievable but not yet practical at scale.
Catalog record
| CREATOR | Yunus Can Gültekin , Péter Scheepers, Yuncheng Yuan, Federico Corradi , and Alex Alvarado |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18758
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On the Optimality of Single-label and Multi-label Neural Network Decoders Thanks: This work has been partially funded by the Eindhoven Hendrik Casimir Institute (EHCI) Collaborative Projects Program. This paper was presented in part at the International ITG Conference on Systems, Communications and Coding (SCC), Karlsruhe, Germany, March 2025 [ 1 ] . Thanks: Y. C. Gültekin, F. Corradi, and A. Alvarado are with the Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands (e-mails: [email protected], [email protected], [email protected]). Thanks: P. Scheepers was with the Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands. He is now with the School of Electrical Engineering and Computer Science, Technical University of Berlin, Berlin, Germany (e-mail: [email protected]). Thanks: Y. Yuan was with the Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands (e-mail: [email protected]). Yunus Can Gültekin , Péter Scheepers, Yuncheng Yuan, Federico Corradi , and Alex Alvarado Abstract We investigate the design of two neural network (NN) architectures recently proposed as decoders for forward error correction: the so-called single-label NN (SLNN) and multi-label NN (MLNN) decoders. These decoders have been reported to achieve near-optimal codeword- and bit-wise performance, respectively. Results in the literature show near-optimality for a variety of short codes. In this paper, we analytically prove that certain SLNN and MLNN architectures can, in fact, always realize optimal decoding, regardless of the code. These optimal architectures and their binary weights are shown to be defined by the codebook, i.e., no training or network optimization is required. Our proposed architectures are in fact not NNs, but a different way of implementing the maximum likelihood decoding rule. Optimal performance is numerically demonstrated for Hamming ( 7 , 4 ) (7,4) , Polar ( 16 , 8 ) (16,8) , and BCH ( 31 , 21 ) (31,21) codes. The results show that our optimal architectures are less complex than the SLNN and MLNN architectures proposed in the literature, which in fact only achieve near-optimal performance. Extension to longer codes is still hindered by the curse of dimensionality. Therefore, even though SLNN and MLNN can perform maximum likelihood decoding, such architectures cannot be used for medium and long codes. I Introduction Forward error correction (FEC) is essential for reliable data transmission over noisy channels [ 2 ] . FEC is also a fundamental part of next-generation wireless and optical communication systems [ 3 , 4 ] . Designing FEC codes and decoders that achieve optimum performance, with low complexity and low latency, has been the objective of coding theory for decades. Maximum likelihood decoding minimizes the error probability, and thus, it is performance-wise optimal [ 5 ] . However, maximum likelihood decoding is impractical, as its complexity grows exponentially with increasing code blocklength n for a given coding rate R=k/n [ 6 ] . In practice, instead of maximum likelihood decoding, low-complexity and low-latency suboptimal decoding algorithms are preferred. Bounded distance decoding (BDD) [ 7 ] or decoding based on Chase-Pyndiah algorithms [ 8 , 9 ] are widely used in combination with algebraic codes [ 10 ] for optical communications. Successive cancellation (SC) [ 11 ] and SC list [ 12 ] decoding of polar codes are used in 5G cellular communications [ 13 ] . Low-density parity-check (LDPC) codes, which also find application in quantum key distribution systems [ 14 ] and quantum error correction [ 15 ] , are typically decoded via message-passing belief propagation [ 16 , 17 ] . Following recent advances in machine learning, decoding FEC codes via neural network (NN) architectures has attracted considerable attention in the literature [ 18 ] . The advantages of these data-driven decoders are especially apparent for applications where no accurate mathematical model for the channel is available, or where the corresponding traditional decoder performs poorly for the code under consideration. As an example, consider quantum LDPC codes constructed via the Calderbank-Shor-Steane approach [ 19 ] . The Tanner graphs of these codes have short cycles of length 4 . These cycles are known to significantly degrade the performance of BP and cause very high error floors [ 20 ] . As a solution to this problem, neural BP was recently proposed in [ 21 ] . Other examples of NN-based decoders include, e.g., [ 22 , 23 , 24 ] . Historically, NNs were first considered for decoding FEC codes in the early 1990s, e.g., in [ 25 ] . NN decoders were deemed suitable only for very short codes ( n<100 ) due to the curse of dimensionality [ 25 , 26 ] . Even considering the recent surge in NN decoders proposed in the literature, starting with [ 27 ] , we see from the comprehensive survey [ 18 ] and references therein that the codes considered for NN-based decoding are almost always limited to lengths below n=512 . Fig. 1: Communication system model considered in this work. Three possible encoders are used in combination with 4 different decoders. Maximum likelihood decoding is based on ( 2 ) or ( 8 ). The SLNN and MLNN decoders are those proposed in [ 28 , 29 , 30 , 31 ] . BDD decoding is based on hard-decisions (HDs) \hat{\boldsymbol{r}} made on \boldsymbol{r} . Recently, single-label NN (SLNN) and multi-label NN (MLNN) decoders have been introduced [ 28 , 29 , 30 , 31 ] . These decoders have a fully-connected architecture, with a few hidden layers. SLNN and MLNN decoders have been shown in [ 31 ] to provide codeword- and bit-wise near-optimal performance, respectively, for a variety of short ( n<34 ) FEC codes. The results in [ 31 ] were obtained by training the NNs at a fixed signal-to-noise ratio (SNR), and then, using the same NN for a wide range of SNRs. The NN architecture of these decoders does not rely on the knowledge of the specific code structure, i.e., they are model-free, and therefore, in principle, applicable to any code. Furthermore, these decoders are non-iterative. The computational complexity of the SLNN and MLNN decoders becomes infeasible for large input lengths k . To alleviate this problem, the iterative network pruning approach proposed in [ 32 ] has been used to prune weights in the SLNN and MLNN architectures in [ 33 ] . With this approach, the complexity of the SLNN and MLNN decoders has been reduced by up to 97 % without compromising the decoding performance [ 33 , Sec. 6] . The reduction in complexity has been demonstrated for the Hamming (7,4) and Polar (16,8) codes. SLNN and MLNN with weight pruning have the potential to be scalable for longer codes, without significantly sacrificing performance [ 33 ] . For the aforementioned reasons, at first glance, SLNN and MLNN decoders seem to be a very interesting alternative for decoding short codes. However, only heuristic rules for the design of these decoders exist in the literature. For example, it is unclear how many hidden layers or neurons per layer these decoders should use for different codes. Furthermore, to the best of our knowledge, the performance of SLNN and MLNN has not been rigorously analyzed in the literature, and no comparisons to traditional decoders exist. In this paper, we study the