Cooperative pursuit of unauthorized UAVs in urban airspace via Multi-agent reinforcement learning. (July 2021)

• Record Type: Journal Article
• Authors: Du, Wenbo; Guo, Tong; Chen, Jun; Li, Biyue; Zhu, Guangxiang; Cao, Xianbin
• Is Part Of: Transportation research. Volume 128(2021)
• Publication Date: 2021-07
• Subjects: Urban Air Mobility (UAM) -- Unmanned Aerial Vehicle (UAV) -- Multi-agent Reinforcement Learning (MARL); Transportation -- Periodicals; Transportation -- Technological innovations -- Periodicals; 388.011
• DOI: 10.1016/j.trc.2021.103122
• Languages: English
• ISSNs: 0968-090X

Abstract

Highlights: A multi-agent reinforcement learning-based approach can address cooperative pursuit without knowing the strategy of evader. The communications among pursuers have a critical impact on their cooperation. The curriculum learning is helpful for pursuers to learn more sophisticated pursuit strategies. Abstract: Urban Air Mobility (UAM) is an emergent concept for future air transportation. With UAM, cargo and passengers will be transported on-demand in urban airspace. UAM has shown a promising prospect in mitigating ground congestion and providing people with an alternative mobility option. However, unauthorized unmanned aerial vehicles (UAVs) in urban airspace present a significant threat to safety of UAM, drawing significant attention from research communities recently. Among all solutions, cooperative pursuit using a team of UAVs is an effective countermeasure for unauthorized UAVs in urban airspace. In this paper, we model cooperative pursuit as a pursuit-evasion game problem (PEG) and propose a multi-agent reinforcement learning (MARL) based approach to solve the problem efficiently. The proposed approach incorporates novel cellular-enabled parameter sharing and curriculum learning schemes to enhance the capability of pursuer UAVs in capturing faster unauthorized UAVs in urban airspace. Extensive experiments have been conducted using simulated urban airspace in order to evaluate the performance of the proposed method. Experimental results demonstrate that by incorporating the parameter sharing scheme, the proposed methods provide much higher capturing rates in a shorter time. Such superiority is more evident when communication constraints are more stringent and/or unauthorized UAVs are faster.