TY - JOUR

T1 - AoI-minimal Power Adjustment in RF-EH-powered Industrial IoT Networks

T2 - A Soft Actor-Critic-Based Method

AU - Ge, Yiyang

AU - Xiong, Ke

AU - Wang, Qiong

AU - Ni, Qiang

AU - Fan, Pingyi

AU - Letaief, Khaled Ben

PY - 2024/2/5

Y1 - 2024/2/5

N2 - This paper investigates the radio-frequency-energy-harvesting-powered (RF-EH-powered) wireless Industrial Internet of Things (IIoT) networks, where multiple sensor nodes (SNs) are first powered by a wireless power station (WPS), and then collect status updates from the industrial environment and finally transmit the collected data to the monitor with their harvested energy. To enhance the timeliness of data, age of information (AoI) is used as a metric to optimize the system. Particularly, an expected sum AoI (ESA) minimization problem is formulated by optimizing the power adjustment policy for the SNs under multiple practical constraints, including the EH, the minimal signal-to-noise-plus-interference ratio (SINR) and the battery capacity constraints. To solve the non-convex problem with no explicit AoI expression, we transform it into a Markov decision problem (MDP) with continuous state space and action space. Then, inspired by the Soft Actor-Critic (SAC) framework in deep reinforcement learning, a SAC-based age-aware power adjustment (SAPA) method is proposed by modeling the power adjustment as a stochastic strategy. Furthermore, to reduce the communication overhead of SAPA, a multi-agent version of SAPA, i.e., MSAPA, is proposed, with which each SN is able to adjust its transmit power based on its local observations. The communication overhead of SAPA and MSAPA is also analyzed theoretically. Simulation results show that the proposed SAPA and MSAPA converge well with different numbers of SNs. It is also shown that the ESA achieved by the proposed SAPA and MSAPA is lower than that achieved by the baseline methods.

AB - This paper investigates the radio-frequency-energy-harvesting-powered (RF-EH-powered) wireless Industrial Internet of Things (IIoT) networks, where multiple sensor nodes (SNs) are first powered by a wireless power station (WPS), and then collect status updates from the industrial environment and finally transmit the collected data to the monitor with their harvested energy. To enhance the timeliness of data, age of information (AoI) is used as a metric to optimize the system. Particularly, an expected sum AoI (ESA) minimization problem is formulated by optimizing the power adjustment policy for the SNs under multiple practical constraints, including the EH, the minimal signal-to-noise-plus-interference ratio (SINR) and the battery capacity constraints. To solve the non-convex problem with no explicit AoI expression, we transform it into a Markov decision problem (MDP) with continuous state space and action space. Then, inspired by the Soft Actor-Critic (SAC) framework in deep reinforcement learning, a SAC-based age-aware power adjustment (SAPA) method is proposed by modeling the power adjustment as a stochastic strategy. Furthermore, to reduce the communication overhead of SAPA, a multi-agent version of SAPA, i.e., MSAPA, is proposed, with which each SN is able to adjust its transmit power based on its local observations. The communication overhead of SAPA and MSAPA is also analyzed theoretically. Simulation results show that the proposed SAPA and MSAPA converge well with different numbers of SNs. It is also shown that the ESA achieved by the proposed SAPA and MSAPA is lower than that achieved by the baseline methods.

KW - Electrical and Electronic Engineering

KW - Computer Networks and Communications

KW - Software

U2 - 10.1109/tmc.2024.3356229

DO - 10.1109/tmc.2024.3356229

M3 - Journal article

JO - IEEE Transactions on Mobile Computing

JF - IEEE Transactions on Mobile Computing

SN - 1536-1233

ER -