An efficient actor‐critic reinforcement learning for device‐to‐device communication underlaying sectored cellular network. (21st January 2020)

• Record Type: Journal Article
• Authors: Khuntia, Pratap; Hazra, Ranjay; Chong, Peter
• Other Names: Souihi Sami guestEditor.; Bitam Salim guestEditor.; Mellouk Abdelhamid guestEditor.; Abreu Thiago guestEditor.; Hoceini Said guestEditor.; Fowler Scott guestEditor.; Medileh Saci guestEditor.; De Swades guestEditor.; Shami Abdallah guestEditor.
• Is Part Of: International journal of communication systems. Volume 33:Number 10(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-01-21
• Subjects: actor‐critic reinforcement learning -- cell sectoring -- device‐to‐device communication -- k‐means clustering -- queuing model -- resource allocation; Telecommunication systems -- Periodicals; 621.382
• DOI: 10.1002/dac.4315
• Languages: English
• ISSNs: 1074-5351

Abstract

Summary: In this paper, a novel reinforcement learning (RL) approach with cell sectoring is proposed to solve the channel and power allocation issue for a device‐to‐device (D2D)‐enabled cellular network when the prior traffic information is not known to the base station (BS). Further, this paper explores an optimal policy for resource and power allocation between users intending to maximize the sum‐rate of the overall system. Since the behavior of wireless channel and traffic request of users in the system is stochastic in nature, the dynamic property of the environment allows us to employ an actor‐critic RL technique to learn the best policy through continuous interaction with the surrounding. The proposed work comprises of four phases: cell splitting, clustering, queuing model, and channel allocation and power allocation simultaneously using an actor‐critic RL. The implementation of cell splitting with novel clustering technique increases the network coverage, reduces co‐channel cell interference, and minimizes the transmission power of nodes, whereas the queuing model solves the issue of waiting time for users in a priority‐based data transmission. With the help of continuous state‐action space, the actor‐critic RL algorithm based on policy gradient improves the overall system sum‐rate as well as the D2D throughput. The actor adopts a parameter‐based stochastic policy for giving continuous action while the critic estimates the policy and criticizes the actor for the action. This reduces the high variance of the policy gradient. Through numerical simulations, the benefit of our resource sharing scheme over other existing traditional scheme is verified. Abstract : In this paper, a novel reinforcement learning (RL) approach with cell sectoring is proposed to solve the channel and power allocation issue for a device‐to‐device (D2D) enabled cellular network. Since the behavior of wireless channel and traffic request of users in the system is stochastic in nature, the dynamic property of the environment, allows us to employ an actor‐critic RL technique to learn the best policy for resource allocation through continuous interaction with the surrounding.