AoI-Aware Machine Learning for Constrained Multimodal Sensing-Aided Communications
Abolfazl Zakeri 1 , Nhan Thanh Nguyen 1 , Ahmed Alkhateeb 2 , and Markku Juntti 1 · 2025
Read AoI-Aware Machine Learning for Constrained Multimodal Sensing-Aided Communications by Abolfazl Zakeri 1 , Nhan Thanh Nguyen 1 , Ahmed Alkhateeb 2 , and Markku Juntti 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-01406.txt.
Introduction
Abolfazl Zakeri and colleagues decide when a wireless system should bother sensing. Sensor data about the environment can guide beam selection and cut training overhead. But fresh data may be unavailable at inference because of sensing limits or sensor failures. So the sensing rate itself must be budgeted.
Their framework pairs a deep Q-network, which makes the sensing decisions, with a neural beam predictor that picks the best beam from a codebook. The age of information, how stale the latest sensor data is, enters the training of both parts, and a Lyapunov optimization reward enforces the average sensing constraint. On a real world data set, age aware training raised top one and top three beam prediction accuracy greatly under a strict sensing budget, with gains fading as the budget loosens.
Catalog record
| CREATOR | Abolfazl Zakeri 1 , Nhan Thanh Nguyen 1 , Ahmed Alkhateeb 2 , and Markku Juntti 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-01406
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AoI-Aware Machine Learning for Constrained Multimodal Sensing-Aided Communications Abolfazl Zakeri 1 , Nhan Thanh Nguyen 1 , Ahmed Alkhateeb 2 , and Markku Juntti 1 Abstract Using environmental sensory data can enhance communications beam training and reduce its overhead compared to conventional methods. However, the availability of fresh sensory data during inference may be limited due to sensing constraints or sensor failures, necessitating a realistic model for multimodal sensing. This paper proposes a joint multimodal sensing and beam prediction framework that operates under a constraint on the average sensing rate, i.e., how often fresh sensory data should be obtained. The proposed method combines deep reinforcement learning, i.e., a deep Q-network (DQN), with a neural network (NN)-based beam predictor. The DQN determines the sensing decisions, while the NN predicts the best beam from the codebook. To capture the effect of limited fresh data during inference, the age of information (AoI) is incorporated into the training of both the DQN and the beam predictor. Lyapunov optimization is employed to design a reward function that enforces the average sensing constraint. Simulation results on a real-world dataset show that AoI-aware training improves top- 1 1 and top- 3 3 inference accuracy by 44.16 % 44.16\% and 52.96 % 52.96\% , respectively, under a strict sensing constraint. The performance gain, however, diminishes as the sensing constraint is relaxed. 1 CWC-RT, University of Oulu, Finland, Email: {abolfazl.zakeri, nhan.nguyen, markku.juntti}@oulu.fi 2 The School of Electrical, Computer, and Energy Engineering, Arizona State University, USA, Email: [email protected] I Introduction Beam management is essential for achieving high performance in millimeter-wave (mmWave) and terahertz (THz) communication systems. Conventional approaches typically rely on beam scanning to identify the best beam, which introduces substantial beam training overhead. Recently, advancements in deep neural networks (DNNs) have enabled learning-based approaches that predict the optimal beam directly from sensory inputs such as position, vision, and LiDAR data [ 1 , 2 , 3 , 4 ] ; this is broadly referred to as multimodal sensing-aided communications. These methods effectively reduce the beam search space to a small set of high-probability candidates, thereby significantly lowering the beam training overhead. However, the practical deployment of multimodal sensing-aided beam training methods requires careful consideration of the complexities associated with data acquisition, processing, and the underlying DNN models. Furthermore, the availability of fresh sensory data during inference is often limited by constrained network resources and possible sensor failures. Motivated by these observations, this work investigates a beam prediction problem under constrained multimodal sensing. We propose a dynamic sensing and beam prediction framework for a single user and single base station (BS) subject to a constraint on the average number of multimodal sensing operations, which accounts for resource constraints such as bandwidth, power, and processing overhead. The proposed framework integrates a deep Q-network (DQN) for dynamic sensor data acquisition and a deep neural network (DNN) for predicting beams using position and camera data. To model the effect of limited fresh data availability during inference, we incorporate the age of information (AoI) into the training process of both models, where the AoI of each sample represents the time elapsed since the sensory data was captured [ 5 , 6 ] . We define a reward function to enforce the sensing constraint using Lyapunov optimization. Simulation results highlight some insightful trade-offs between data freshness and the top- k accuracy of the beam predictor. Related Work: Patel and Heath [ 7 ] proposed a multimodal fusion framework that combines visual and positional sensing with radio-frequency (RF) features for multi-user beamforming. Kim et al. [ 8 ] exploited vision-aided positioning to improve beam focusing accuracy in THz systems. Work [ 9 ] demonstrated a camera-assisted beam tracking prototype capable of maintaining reliable links under mobility. Arnold et al. [ 10 ] utilized vision-assisted digital twins to enable context-aware beam management, while Vuckovic et al. [ 11 ] reexamined performance metrics for multimodal beam prediction. To reduce the complexity of the underlying DNN, [ 12 ] proposes the use of knowledge distillation, and [ 13 ] presents a data-free knowledge distillation method that enables model compression without requiring access to the original training data. In this context, Park et al. [ 14 ] developed a resource-efficient multimodal beam prediction framework that distills knowledge from LiDAR-, radar-, RGB-, and GPS-based models into a lightweight radar-only network. Collectively, these studies demonstrate that incorporating visual and spatial modalities can significantly improve beam alignment efficiency and robustness compared to RF-only schemes. Nonetheless, the impact of sensing constraints and the availability of fresh data during inference remain largely unexplored. The most closely related works to this paper are [ 15 ] and [ 16 ] . In [ 15 ] , the authors developed a multimodal machine-learning framework that integrates vision and GPS (position) data with mmWave beam training measurements to predict optimal beams in real vehicular communications scenarios. Building on this, our work extends [ 15 ] to a constrained multimodal sensing framework and further extends our previous study [ 16 ] to multimodal sensing that incorporates AoI into the beam predictor training, thereby improving beam prediction performance under strictly limited sensing conditions. II System Model and Problem Formulation We consider a downlink mmWave communications system consisting of a BS and a single-antenna mobile user equipment (UE). The BS is equipped with N antennas and an RGB camera. The BS adopts a predefined analog beamforming codebook \mathcal{F}=\{\mathbf{f}_{1},\dots,\mathbf{f}_{M}\} for signal transmission, where \mathbf{f}_{m}\in\mathbb{C}^{N\times 1} with {\|\mathbf{f}_{m}\|_{2}^{2}=1} , {m=1,2,\dots,M} . Denote by \mathbf{h}(t)\in\mathbb{C}^{N\times 1} the channel between the BS and the UE at time slot t . Furthermore, let {x}(t)\in\mathbb{C} be the transmit data symbol from the BS to the UE at slot t , with \mathbb{E}\{|{{x}(t)}|^{2}\}=P , where P is the transmit power. Suppose the beamforming vector \mathbf{f}(t)\in\mathcal{F} is chosen at slot t , the received signal at the UE is then given by \displaystyle y(t)=\mathbf{h}^{\textsf{H}}(t)\mathbf{f}(t){s}(t)+n(t). (1) Here n(t)\in\mathbb{C} is additive white Gaussian noise (AWGN) following the distribution \mathcal{CN}(0,\sigma^{2}) , with \sigma^{2} denoting the noise variance at the UE. In the conventional beam scanning methods, best beam m at time t is chosen from the codebook to maximize the received signal-to-noise ratio (SNR) \dfrac{|\mathbf{h}^{\mathsf{H}}(t)\mathbf{f}_{m}(t)|^{2}}{\sigma^{2}} . This requires the channel state information or certain search-based beam training procedures. The former is typically hard to obtain in mmWave/THz systems, and the latter incurs excessive beam training overhead and latency. To address these challenges, the primary goal of this paper is to develop a multimodal sensing-aided beam prediction that utilizes visual and position data to predict the optimal beam to maximize the received SNR. Particularly, we will utilize RGB images captured by the camera installed at the BS and the real-time position information of the UE. Let \mathbf{p}(t)\in\mathbb{R}^{2} be the two-dimensional position vector (latitude and longitude) of the UE. Furthermore, let \mathbf{X}(t)\in\mathbb{R}^{W\times H\times C} be the RGB image captured at slot t , where W , H , and C are respectively the width, height, and the number of color channels of the image. Existing works on multimodal sensing-aided beam prediction assume the availability of the freshest data , i.e., \{\mathbf{p}(t),\mathbf{X}(t)\} , (during inference) to predict the best beam at time slot t . This, however, necessitates continuous sensing, bandwidth, and power, and in