Federated reinforcement learning approach for detecting uncertain deceptive target using autonomous dual UAV system. Issue 2 (March 2023)

• Record Type: Journal Article
• Authors: Bany Salameh, Haythem; Alhafnawi, Mohannad; Masadeh, Ala'eddin; Jararweh, Yaser
• Is Part Of: Information processing & management. Volume 60:Issue 2(2023)
• Publication Date: 2023-03
• Subjects: Cooperative learning -- Federated learning -- Artificial intelligence -- Emerging UAV -- Indoor environment; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.103149
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: This paper develops a cooperative federated reinforcement learning (RL) strategy that enables two unmanned aerial vehicles (UAVs) to cooperate in learning and predicting the movements of an intelligent deceptive target in a given search area. The proposed strategy allows the UAVs to autonomously cooperate, through information exchange of the gained experience to maximize the target detection performance and accelerate the learning speed while maintaining privacy. Specifically, we consider a monitoring model that includes a search area, a charging station, two cooperative UAVs, an intelligent deceptive uncertain moving target, and a fake (false) target. Each UAV is equipped with a limited-capacity rechargeable battery and a communication unit for exchanging the gained experience. The problem of maximizing the detection probability of the uncertain deceptive target using cooperative UAVs is mathematically modeled as a search-benefit maximization problem, which is then reformulated as a Markov decision process (MDP) due to the uncertainty nature of the problem. Because there is no prior information on the targets' movement, a cooperative RL, is utilized to tackle the problem. The proposed cooperative RL-based algorithm is a distributed collaborative mechanism that enables the two UAVs, i.e., agents, to individually interact with the operating environment and maximize their cumulative rewards by converging to a shared policy while achieving privacy. Simulation results indicate that a cooperative RL-based dual UAV system can noticeably improve the target detection probability, reduce the detection performance, and accelerate the learning speed. Highlights: UAV cooperative learning for target detection in indoor environments is investigated. The detection problem of deceptive target using 2-UAVs is mathematically formulated. The 2-UAV system optimization is reformulated as MDP and solved using cooperative RL.