Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction

Uma Kishore Godavarti 1, 2 , K. Giridhar 2 , Vanani Prince Dharmendrabhai 1 , Anchit Panday 1 , and Madhan Raj Kanagarathinam 1 · 2026

Read Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction by Uma Kishore Godavarti 1, 2 , K. Giridhar 2 , Vanani Prince Dharmendrabhai 1 , Anchit Panday 1 , and Madhan Raj Kanagarathinam 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25401.txt.

Introduction

Base stations must guess how a wireless channel will change a moment ahead, yet the best guessing models are too heavy for station hardware. Uma Kishore Godavarti and colleagues present a lighter predictor in 2026 that fixes two design flaws in a leading model instead of merely shrinking it. Spatial and temporal attention blocks no longer run in sequence, which had distorted the timing features; they now read the same input in parallel and merge through a learned gate. A bottleneck autoencoder halves the deepest layer's width.

The resulting network carries fifty eight percent fewer parameters than its baseline. On simulated driving speed tests it predicted channels more accurately anyway, gaining several decibels at both town and faster speeds. An added reconstruction loss keeps the squeezed bottleneck from throwing away information the predictor needs.

Catalog record

CREATORUma Kishore Godavarti 1, 2 , K. Giridhar 2 , Vanani Prince Dharmendrabhai 1 , Anchit Panday 1 , and Madhan Raj Kanagarathinam 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25401

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction Uma Kishore Godavarti 1, 2 , K. Giridhar 2 , Vanani Prince Dharmendrabhai 1 , Anchit Panday 1 , and Madhan Raj Kanagarathinam 1 Abstract Accurate channel state information (CSI) prediction is essential for proactive beamforming and resource management in 5G massive MIMO systems, yet the deployment of high-accuracy transformer-based predictors on base-station hardware remains challenging because the most capable models carry upwards of 30 M parameters. This paper introduces Lightweight PCGAE-Net, which addresses the efficiency problem not by post-hoc compression but by correcting two architectural flaws in the current state of the art. The first is a sequential attention ordering bias: in CS3T-UNet, group-wise temporal attention (GTA) always operates on features that have already been transformed by cross-shaped spatial attention (CSA), distorting what temporal information GTA can capture. We remove this dependency by routing both attention modules to the same layer-normalized input and combining their independent outputs through a learned per-channel sigmoid CrossGate. The second flaw is an uncompressed bottleneck: applying full self-attention at the deepest encoder stage, where channel depth reaches 4 ​ C 4C , is quadratically expensive and carries redundant features. A Bottleneck AutoEncoder (BAE) with 1 × 1 1\times 1 convolutions halves this depth and uses an auxiliary reconstruction loss to prevent information collapse. Wrapping these components inside a shallower encoder-decoder with frequency-domain dimensionality reduction ( N f = 32 N_{f}\!=\!32 , C = 48 C\!=\!48 ) produces a model with just 8.54 M parameters—58% fewer than the CS3T-UNet baseline—that outperforms it by up to 3.26 dB at 5 km/h and 6.0 dB at 9 km/h in single-step prediction on QuaDriGa dataset. I Introduction The ability to predict future channel conditions from past observations has become an important signal processing step in 5G and beyond 5G systems. In massive MIMO systems, the base station needs accurate CSI to compute beamforming weights, but obtaining that information requires pilot transmission, feedback, and processing time that consume the coherence window [ 9 ] . A predictor that can accurately forecast the channel one or several time steps ahead, using only the history of observed frames, lets the base station prepare precoding matrices in advance and tolerate feedback delays that would otherwise degrade throughput [ 5 ] . For several years, recurrent architectures were the natural choice for this task. LSTM and GRU networks can model sequential temporal evolution, and their predictions improve noticeably over simple extrapolation [ 13 ] . However, recurrent models process time steps one at a time, which limits parallelism and makes it hard to capture long-range dependencies across a channel sequence of tens of frames. CNN-based approaches [ 12 ] avoid this by treating the channel matrix as a 2D spatial image, but trade temporal modeling for spatial feature extraction rather than handling both jointly. Transformers changed this picture. The self-attention mechanism can attend to arbitrary positions in both spatial and temporal dimensions simultaneously, and several recent channel prediction systems have used transformer-based encoder-decoder architectures to reach state-of-the-art NMSE [ 10 , 14 ] . CS3T-UNet [ 3 ] is among the most capable, combining a U-Net backbone with cross-shaped spatial attention (CSA) and group-wise temporal attention (GTA). At 20.34 M parameters, it achieves strong results, but two aspects of its design limit both performance and efficiency. The first is the order in which it applies attention. In CS3T-UNet, CSA runs first and GTA is applied to its output, meaning GTA always sees a spatially-transformed version of the channel features. When spatial and temporal dynamics are partially coupled—as they are in MIMO systems where physical angle-of-arrival varies over time—applying spatial processing before temporal processing biases what GTA can extract. The model partially compensates during training, but the structural constraint remains. The second issue is the bottleneck: at the deepest encoder layer, feature depth reaches 4C (Where C is Embedding dimension), and CS3T-UNet applies cross-shaped attention at full resolution. Self-attention cost scales quadratically with channel depth, so this is the most expensive operation in the network, applied to a feature volume that carries unavoidable redundancy at such depth. These are architectural problems, not weight-level inefficiencies, so post-hoc techniques like pruning or quantization cannot resolve them. The clean solution is to redesign the attention mechanism. This paper presents Lightweight PCGAE-Net, which makes the following specific contributions: 1. Parallel CrossGate Attention Block: CSA and GTA are computed simultaneously on identical layer-normalized inputs, removing the ordering dependency. Their outputs are combined by a learned per-channel sigmoid gate that adapts the spatial-temporal mixture based on the actual feature content, with zero-initialization ensuring a balanced starting point. 2. Bottleneck AutoEncoder with Auxiliary Supervision: At the deepest encoder stage, 1\times 1 convolutions compress feature depth from 4C to 2C , cutting attention cost by roughly 75%. An auxiliary reconstruction loss prevents the compressed bottleneck from discarding channel structure needed by the decoder. 3. Slim Encoder-Decoder and Delay Domain Dimensionality Reduction: Reducing the number of merge/expand block pairs from three to two, and cropping the input to N_{f}\!=\!32 most energetic ADP delay bins with base channel width C\!=\!48 , brings total model size to 8.54 M parameters—73% fewer than the full PCGAE-Net and 58% fewer than CS3T-UNet. 4. Systematic evaluation: Five model variants are evaluated at three UE speeds (5, 9, 12 km/h) and two prediction horizons ( L\!\in\!\{1,5\} ) on QuaDriGa [ 2 ] under 3GPP 38.901 UMa NLOS conditions, with ablation for each design choice. The paper is organized as follows. Section II reviews related work. Section III describes the system model. Section IV presents the proposed architecture. Section V details the experimental setup. Section VI reports results and analysis. Section VII concludes. II Related Work II-A Channel Prediction with Deep Learning The deep learning work on channel prediction adapted models developed for time-series forecasting. LSTM networks were applied to OFDM channel matrices by treating the sequence of channel observations as a multivariate time series [ 13 ] . This works well at low antenna counts but does not scale well to massive MIMO, where the spatial dimension has hundreds of entries, and the spatial structure is as informative as the temporal one. GRU-based approaches [ 6 ] offer a lighter recurrent alternative and remain competitive at low mobility. CNNs entered through the CSI feedback compression literature [ 12 ] , where autoencoders learned to compress and reconstruct channel matrices. The encoder half naturally learns spatial feature representations, and it was a short step to reuse those representations for prediction. The limitation is that CNNs have fixed receptive fields and cannot model dependencies between frames that are far apart in time without deep architectures that carry correspondingly large parameter counts. Transformers solved this by replacing convolutions with self-attention, which has a global receptive field by construction [ 10 ] . Several groups have since applied transformer architectures to channel prediction, generally following the pattern of a spatial feature extractor followed by a temporal attention module [ 14 ] . CS3T-UNet [ 3 ] achieves the best published NMSE on QuaDriGa dataset by combining CSA with GTA inside a U-Net encoder-decoder. Our work retains this backbone structure, but restructures the attention to run in parallel rather than in

Read the complete text