Reinforcement learning for energy efficiency improvement in UAV-BS access networks: A knowledge transfer scheme. (April 2023)

• Record Type: Journal Article
• Authors: Hu, Zhiqun; Zhang, Yujing; Huang, Hao; Wen, Xiangming; Agbodike, Obinna; Chen, Jenhui
• Is Part Of: Engineering applications of artificial intelligence. Volume 120(2023)
• Publication Date: 2023-04
• Subjects: Algorithm -- Energy harvesting -- NP-hard -- Resource management -- Reinforcement learning -- UAV; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2023.105930
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: Recently the possibility of forming unmanned aerial vehicle base station (UAV-BS) network systems with energy harvesting capabilities to support persistent wireless access services for pedestrian users has been validated. Due to the need of sustaining wireless access services of the UAV-BSs, we investigate an optimal policy to maximize the overall energy utilization efficiency (renewable energy) of the UAV-BSs during their active in-flight network access operations. Since the natural sources of renewable energy (e.g., solar energy or wind energy harvesting) have stochastic properties with respect to the arrival rate of the dynamics of the unknown environment, we exploit an actor–critic reinforcement learning framework, which considers the continuous-valued states and action space for learning the best policy during interaction with the environment. To enhance and expedite the learning process, a transfer asynchronous advantage actor–critic (TA3C) algorithm is proposed, which enables UAV-BSs to transfer (i.e., share) knowledge gained in historical periods, during parallel task asynchronous executions on multiple instances of the environment. Numerical results reveal that the proposed TA3C algorithm surpasses the classic A3C and A2C algorithms in terms of throughput and optimal energy utilization efficiency.