An integrated multi-head dual sparse self-attention network for remaining useful life prediction. (May 2023)

• Record Type: Journal Article
• Authors: Zhang, Jiusi; Li, Xiang; Tian, Jilun; Luo, Hao; Yin, Shen
• Is Part Of: Reliability engineering & system safety. Volume 233(2023)
• Publication Date: 2023-05
• Subjects: Multi-head self-attention -- Remaining useful life -- Transformer -- Sparse strategy -- Prediction; Reliability (Engineering) -- Periodicals; System safety -- Periodicals; Industrial safety -- Periodicals; Fiabilité -- Périodiques; Sécurité des systèmes -- Périodiques; Sécurité du travail -- Périodiques; 620.00452
• DOI: 10.1016/j.ress.2023.109096
• Languages: English
• ISSNs: 0951-8320

Abstract

Abstract: Committed to accident prevention, prediction of remaining useful life (RUL) plays a crucial role in prognostics health management technology. Conventional convolutional neural network and long-short-term memory network have notable limitations in the size of convolution in processing temporal data and the associations between non-adjacent data when predicting the RUL, respectively. Although the proposal of the Transformer provides an opportunity to solve the shortcomings mentioned above, Transformer still has some limitations. Precisely, the Transformer model awaits in-depth research focusing on vital local regions and decreasing computational complexity. In this sense, this paper proposes a novel integrated multi-head dual sparse self-attention network (IMDSSN) based on a modified Transformer to predict the RUL. From two sparse perspectives, the proposed IMDSSN includes a multi-head ProbSparse self-attention network (MPSN) and a multi-head LogSparse self-attention network (MLSN). Specifically, MPSN is designed to filter out the primary function of the dot product operation, thereby improving computational efficiency. Furthermore, considering the data inside the whole time window, a comprehensive logarithmic-based sparse strategy in MLSN is proposed to reduce the amount of computation. An aircraft turbofan engine dataset is used to verify the proposed IMDSSN, which demonstrates that the IMDSSN is better than some conventional approaches. Highlights: An MPSN is designed to filter out the dot product operation with the main role. A comprehensive logarithmic sparse strategy in MLSN is proposed for RUL prediction. The MPSN and MLSN are integrated to obtain great RUL prediction performance. Research results show the IMDSSN-based RUL prediction approach is excellent.