Asset correlation based deep reinforcement learning for the portfolio selection. (1st July 2023)

• Record Type: Journal Article
• Authors: Zhao, Tianlong; Ma, Xiang; Li, Xuemei; Zhang, Caiming
• Is Part Of: Expert systems with applications. Volume 221(2023)
• Publication Date: 2023-07-01
• Subjects: Portfolio selection -- Deep reinforcement learning -- Asset correlations; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2023.119707
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Portfolio selection is an important application of AI in the financial field, which has attracted considerable attention from academia and industry alike. One of the great challenges in this application is modeling the correlation among assets in the portfolio. However, current studies cannot deal well with this challenge because it is difficult to analyze complex nonlinearity in the correlation. This paper proposes a policy network that models the nonlinear correlation by utilizing the self-attention mechanism to better tackle this issue. In addition, a deterministic policy gradient recurrent reinforcement learning method based on Monte Carlo sampling is constructed with the objective function of cumulative return to train the policy network. In most existing reinforcement learning-based studies, the state transition probability is generally regarded as unknown, so the value function of the policy can only be estimated. Based on financial backtest experiments, we analyze that the state transition probability is known in the portfolio, and value function can be directly obtained by sampling, further theoretically proving the optimality of the proposed reinforcement learning method in the portfolio. Finally, the superiority and generality of our approach are demonstrated through comprehensive experiments on the cryptocurrency dataset, S&P 500 stock dataset, and ETF dataset. Highlights: Using self-attention mechanism to model nonlinear correlations among asset prices. Proposing a deterministic policy gradient recurrent reinforcement learning method. The theory proves the superiority of the proposed reinforcement learning method. The superiority of the proposed method is proved by comprehensive experiments.