5G LDPC Linear Transformer for Channel Decoding
Mario Hernandez Fernando Pinero University of Puerto Rico University of Puerto Rico [email protected] [email protected] · 2025
Read 5G LDPC Linear Transformer for Channel Decoding by Mario Hernandez Fernando Pinero University of Puerto Rico University of Puerto Rico [email protected] [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-14102.txt.
Introduction
Mario Hernandez and Fernando Pinero build transformer decoders for 5G codes in a 2025 paper. The codes are LDPC codes used in 5G New Radio. Standard transformers scale quadratically, which is costly for long blocks.
Their design runs in linear time and stays fully differentiable. Performance is compared with belief propagation, the production decoder for these codes. Error rates matched a regular transformer decoder. Results also beat one round of belief propagation. Timing stayed competitive even for larger blocks. Tests ran in an Nvidia research platform for reproducibility. Readers learn a route to learned decoders that could fit real receivers.
Catalog record
| CREATOR | Mario Hernandez Fernando Pinero University of Puerto Rico University of Puerto Rico [email protected] [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-14102
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
5G LDPC Linear Transformer for Channel Decoding Mario Hernandez Fernando Pinero University of Puerto Rico University of Puerto Rico [email protected] [email protected] Abstract This work introduces a novel, fully differentiable linear-time complexity transformer decoder and a transformer decoder to correct 5G New Radio (NR) LDPC codes 1 1 1 All code is available at: https://github.com/pollyjuice74/5G-Decoder . . We propose a scalable approach to decode linear block codes with O ( n ) O(n) complexity rather than O ( n 2 ) O(n^{2}) for regular transformers. The architectures’ performances are compared to Belief Propagation (BP), the production-level decoding algorithm used for 5G New Radio (NR) LDPC codes. We achieve bit error rate performance that matches a regular Transformer decoder and surpases one iteration BP, also achieving competitive time performance against BP, even for larger block codes. We utilize Sionna [ 11 ] , Nvidia’s 5G & 6G physical layer research software, for reproducible results. 1 Introduction In the age of information, robust digital communication has become a cornerstone of modern wireless networks, ensuring data integrity across noisy transmission channels. Central to achieving reliability is the design of efficient error-correcting decoders that balance decoding time and error-correcting capabilities to provide practical solutions as we approach maximum likelihood (ML) decoding [ 8 ] , a theoretical optimum constrained by NP-hard complexity. Belief propagation decoding has emerged as the standard approach for 5G New Radio LDPC codes due to its iterative message-passing scheme and low computational overhead, offering a flexible number of iterations for varying channel conditions and data integrity requirements. While practical, it struggles to match the performance of ML decoding, particularly for short and intermediate block lengths—an ongoing challenge in modern communication systems. Recent breakthroughs in deep learning have introduced new possibilities for error correction and communication [ 20 , 7 , 10 , 13 , 15 ] . Among them, graph neural networks [ 2 , 1 ] have been proposed for their scalability and low-overhead integration into belief propagation decoding, leveraging parameterized iterative decoding. Error-correcting code transformers [ 3 ] represent a notable advancement in decoding architectures, designed as unified solutions for linear block codes. These models leverage the attention mechanism, which operates effectively as a fully connected graph, capturing intricate relationships between codeword elements. This graph-based structure aligns naturally with the decoding requirements of linear block codes, which are traditionally decoded using graph-structured mechanisms like Tanner graphs. However, transformers are limited in scalability due to their exponential training complexity as code size increases. Consequently, their practical application is constrained, particularly for the larger block sizes required in digital communication. To address these challenges, we propose a novel transformer-based error correction decoder designed to overcome the scalability constraints of existing deep learning decoders. Our approach integrates domain knowledge by incorporating the code’s parity-check matrix into the self-attention mechanism, as described in [ 3 ] , effectively guiding the learning process while maintaining computational efficiency. By utilizing Nvidia’s Sionna 5G & 6G Physical Layer Research Software [ 11 ] , we ensure reproducible results across varying parity-check matrix sizes, enabling a fair evaluation of our method’s performance. A key innovation in our work is the introduction of a linear transformer decoder, using the attention proposed in [ 19 ] , that significantly reduces the computational complexity from quadratic O(n^{2}) to linear O(n) scaling. This improvement enables our approach to efficiently decode large block sizes without an exponential increase in training and inference time. To the best of our knowledge, this is the first transformer-based decoder that is not constrained by the curse of dimensionality when applied to 5G NR LDPC codes and other linear block codes. Our contributions can be summarized as follows: • We introduce transformer decoders for 5G NR LDPC codes. • We introduce a fully differentiable linear complexity transformer-based decoder for linear binary block codes. • We provide a benchmark comparison against state-of-the-art 5G NR LDPC decoders. • We present a hyper-parameter optimization study to guide future research on transformer-based decoding. • We ensure reproducible research by implementing our solution within the Sionna framework and making our code publicly available. This work aims to bridge the gap between deep learning and structured decoding algorithms, paving the way for future advancements in AI-driven error correction for next-generation communication systems. 2 Communication Channels 2.1 Encoding and Modulation The encoding process maps information bits b to a codeword c using an encoder \mathcal{E} : c=\mathcal{E}(b) (1) For uncoded transmission, c=b . The encoded bits are then mapped to complex symbols using a QAM mapper \mathcal{M} : x=\mathcal{M}(c) (2) If n is odd, zero-padding is applied to c before mapping. 2.2 Bipartite Graph Structure of the Parity Check Matrix Linear block codes such as Low-Density Parity-Check (LDPC) codes are often represented using bipartite graphs that consists of two sets of nodes: variable nodes (VNs) and check nodes (CNs). The edges connect VNs to CNs based on the parity-check matrix H . For a codeword c and parity-check matrix H , the relation is given by: Hc^{T}=0 (3) where H is an (n-k)\times n matrix over \mathbb{GF}(2) , representing the parity-check constraints, thus a recieved codeword can be decoded. 2.3 Channel Model The modulated symbols x are transmitted through an Additive White Gaussian Noise (AWGN) channel: y=x+w,\quad w\sim\mathcal{N}(0,N_{0}) (4) where the noise variance N_{0} is computed from the given E_{b}/N_{0} using the method described in [ 6 ] : N_{0}=\left(\frac{E_{b}}{N_{0}}\cdot\frac{rM}{E_{s}}\right)^{-1} (5) For coded transmission, the term M represents the average number of coded bits per constellation symbol, E_{s} is the average energy per symbol, and r is the coderate. For uncoded transmission, the coderate r is set to 1. 2.4 Demodulation and Decoding The received signal y is converted into LLRs using a demapper \mathcal{D} : \Lambda(c)=\mathcal{D}(y,N_{0})\in\mathbb{R}^{n\times 1} (6) Here, \Lambda(c) represents the log-likelihood ratio (LLR), quantifying the confidence in each bit c being 0 or 1 , bounded by \pm 20 . For syndrome-based decoding, the decoder input nodes are: \sigma=Hc^{T}\in\{0,1\}^{m\times 1}, (7) where H is the parity-check matrix. The decoder input is then formed as the following conatenation: X_{\text{nodes}}=\begin{bmatrix}\Lambda(c)\\ \sigma\end{bmatrix}\in\mathbb{R}^{(n+m)\times 1} (8) The decoder \mathcal{D}_{e} estimates the transmitted codeword: \hat{c}=\mathcal{D}_{e}(X_{\text{nodes}}) (9) where the estimated bits are obtained by thresholding: \hat{c}_{i}=\begin{cases}1,&\text{if }\hat{c}_{i}>0\text{ (logit threshold)}\\ 0,&\text{otherwise}\end{cases} (10) 3 LDPC Codes Here we illustrate the parity check matrix and mask, as proposed in [ 3 ] , for regular LDPC and 5G New Radio LDPC. We also point out the difference in construction. (a) Regular LDPC parity check matrix and mask . PCM with linearly independent rows, each row has c=6 check node degrees and each column has v=3 variable node degrees. [ 17 , 9 ] (b) 5G New Radio LDPC parity check matrix and mask . PCM constructed by lifting the selected base graph. [ 21 ] 3.1 Regular LDPC Code Construction A regular (v,c) Low-Density Parity-Check (LDPC) code is constructed by defining three key parameters [ 5 ] : v , the number of edges connected to each Variable Node (VN); c , the number of edges connected to each Check Node (CN); and n , the codeword length. These parameters determine the structure and connectivity of the LDPC code, ensuring that it adheres to the regularity constraints specified by