Age-of-information minimization under energy harvesting and non-stationary environment
Akanksha Jaiswal, Arpan Chattopadhyay · 2025
Read Age-of-information minimization under energy harvesting and non-stationary environment by Akanksha Jaiswal, Arpan Chattopadhyay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-04916.txt.
Introduction
Akanksha Jaiswal and Arpan Chattopadhyay tackle stale data in sensor networks in a 2025 paper. Their sources harvest energy from the world around them, then sample and send readings to a sink. Data has an age, and the goal is to keep that age low. At each step a scheduler probes one source's channel, then decides if it should send.
For one source, the team models the choice as a Markov decision process and proves the best rule has a threshold shape. For unknown, shifting rates, they build learning methods that borrow that shape. For many sources, the problem splits into parts per source. A Whittle index rule then sets which source to probe, paired with a threshold rule for sending.
Catalog record
| CREATOR | Akanksha Jaiswal, Arpan Chattopadhyay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-04916
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Age-of-information minimization under energy harvesting and non-stationary environment Thanks: The authors are with the Department of Electrical Engineering, Indian Institute of Technology, Delhi (email: [email protected], [email protected]. Thanks: This paper’s preliminary version was published in [ 1 ] . Akanksha Jaiswal, Arpan Chattopadhyay Abstract This work addresses the problem of minimizing the age of information (AoI) in a wireless sensor network consisting of a sink node and multiple potentially mobile energy harvesting (EH) sources inducing non-stationarity in the environmemnt. At each time instant, the central scheduler selects one of the mobile sources to probe the quality of its channel to the sink node, and then the assessed channel quality is utilized to determine whether a source will collect a sample and send the packet to the sink. For the single source case, we assume that the channel quality is known at each time instant, model the problem of AoI minimization as a Markov decision process (MDP), and prove the optimal sampling policy’s threshold structure. We then use this threshold structure and propose an age-energy-channel based sliding window upper confidence bound reinforcement learning (AEC-SW-UCRL2) algorithm to handle unknown and time-varying energy harvesting rate and channel statistics, motivated by the popular SW-UCRL2 algorithm for non-stationary reinforcement learning (RL). This algorithm is applicable when an upper bound is available for the total variation of each of these quantities over a time horizon. Furthermore, in situations where these variation budgets are not accessible, we introduce a novel technique called age-energy-channel based bandit-over-reinforcement learning (AEC-BORL) algorithm, motivated by the well-known BORL algorithm. For the multiple source case, we demonstrate that the AoI minimization problem can be formulated as a constrained MDP (CMDP) which can be relaxed using a Lagrange multiplier. We decouple the problem into various sub-problems across source nodes, and also derive Whittle’s index based source scheduling policy for probing as well as an optimal threshold policy for source sampling. We next leverage this Whittle’s index and threshold structure to develop a Whittle’s index and threshold based sliding window upper confidence bound reinforcement learning (WIT-SW-UCRL2) algorithm for unknown, time-varying energy harvesting rates and channel statistics under their respective variation budgets. Moreover, we also proposed a Whittle’s index and threshold based bandit-over-reinforcement learning (WIT-BORL) algorithm for unknown variation budgets. Finally, we numerically demonstrate the efficacy of our algorithms. I Introduction For many time-sensitive sensing and communication applications such as IoT systems, vehicle tracking, environment monitoring, smart home systems, etc., the data available at the decision-making node needs to be as fresh as possible. Age of Information (AoI [ 2 ] ) is a popular measure of the freshness of information in such systems and is studied heavily to realize ultra-reliable low-latency communication (URLLC [ 3 ] ) in 5G systems. At any time t , if the decision-making node finds that the latest available monitoring information comes from a packet whose time-stamped generation instant was t^{\prime} , then the decision making node computes the AoI as (t-t^{\prime}) . Nevertheless, battery constraints and restricted communication bandwidth pose two significant obstacles to the implementation of such real-time status update systems. To address the scarcity of bandwidth, it is common for multiple sources to share a common frequency band in order to transmit their time-sensitive status updates to a monitoring node. On the other hand, the challenge of battery limitations can be resolved using energy harvesting source (EH) nodes. However, in order to prevent interference among these nodes and to fully leverage the EH feature available at the source nodes, it is necessary to implement intelligent scheduling policies. Furthermore, for mobile sensors such as UAVs and mobile robots, both energy harvesting characteristics and wireless channel statistics may vary significantly with the nodes’ location and trajectory, which necessitates adaptive, learning-based scheduling and sampling policies that can handle time-varying environments. The recent efforts towards energy and bandwidth-efficient URLLC has led to a number of works on AoI minimization in EH systems. For instance, the papers [ 4 , 5 , 6 , 7 , 8 , 9 ] have proposed optimal strategies to minimize AoI for a single EH source in various network scenarios. In particular, [ 4 ] has investigated the optimal status update policy for a EH source sending update packets through a noiseless channel, and has adopted an energy-aware, threshold-type status update policy. The paper [ 5 ] discusses the trade-off between age and energy, and presents an optimal threshold policy for AoI minimization. The paper [ 6 ] focuses on a threshold policy to minimize AoI for a single EH sensor using the Hybrid Automatic Repeat Request (HARQ) protocol and also provides RL algorithm for an unknown environment. A threshold-based age-optimal policy for an RF-powered source node providing status updates to the destination has been presented by the authors of [ 7 ] where the policy considers the scenario that the source can collect energy or transfer data at a time. The AoI minimization problem was formulated as an MDP for a single EH source with channel probing capabilities in our previous paper [ 8 , 9 ] , which demonstrated threshold structure in the optimal channel probing and source sampling strategies. On the other hand, AoI minimization in networks with several source nodes vying for a common channel has also been extensively studied in the literature [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 ] . Using CMDP formulation, the authors of [ 10 ] minimized the weighted average AoI subject to transmission limitations in a multi-source relaying system with independent sources and error-prone connections. The paper [ 12 ] presented the Maximum Age First (MAF) scheduling policy and Zero wait sampling policy to reduce the total average peak age for the optimal decoupled scheduling and sampling policy. However, the most effective sampling policy for reducing the total average age was determined to be a threshold policy. The paper [ 14 ] has proposed a semi-distributed sampling and updating learning algorithm (via Q-functions) for multiple IoT devices under sampling/updating cost, a block-fading channel, and average energy constraint. To reduce the average AoI for a resource-constrained IoT network with a cache-enabled edge node between users and EH sensors, the authors of [ 15 ] have proposed a relax-then-truncate optimal policy. However, the literature mostly deals with stationary systems where the statistical behaviour of the environment does not change with time. While there have been some recent efforts towards AoI minimization in non-stationary environment [ 19 , 20 , 16 ] . For example, the authors of [ 16 ] have decoupled the multi-sensor scheduling problem into a single-sensor CMDP and utilized the optimal threshold policy structure of each decoupled sensor to design a scheduling algorithm for the AoI minimization problem in a multi-sensor network having time-varying channel states under bandwidth and power consumption constraints. However, no prior work has addressed AoI minimization in EH system under non-stationary environment. On the other hand, Whittle’s Index [ 21 ] for restless multi-armed bandits (RMABs) has been extensively employed to mitigate the computational complexity associated with resolving constrained MDPs ( i.e., CMDPs) for multi-source systems, as it offers a nearly optimal solution. Numerous papers have utilized Whittle’s Index based policies