A temporal–spatial attention-based action recognition method for intelligent fault diagnosis. (June 2022)

• Record Type: Journal Article
• Authors: Luo, Wentao; Zhang, Jianfu; Feng, Pingfa; Yu, Dingwen; Wu, Zhijun
• Is Part Of: ISA transactions. Volume 125(2022)
• Page Start: 459
• Page End: 473
• Publication Date: 2022-06
• Subjects: Temporal-attention model -- Spatial-attention model -- Long-short term model -- Video fault diagnosis; Engineering instruments -- Periodicals; Engineering instruments; Periodicals; Electronic journals; 629.805
• DOI: 10.1016/j.isatra.2021.06.041
• Languages: English
• ISSNs: 0019-0578

Abstract

Abstract: The intelligent fault diagnosis of video data has become a demanding task in industrial applications. However, existing models require expensive computational cost and memory demand, which makes this technology applied in factories impossible. To address this problem, a temporal–spatial attention-based action recognition method (TARM) integrating TAB (temporal-attention-based frame splitting model), SAB (spatial-attention-based agent focusing mode) and LSB (long-short term feature learning mode) is proposed. TAB first extracts important frames from raw videos. Then, SAB refines video data by reinforcing their essential features and weakening unnecessary features. Furthermore, LSB monitors action type of video data by establishing recurrent convolutional architectures. Finally, the performance of TARM in terms of training time and fault diagnosis accuracy are validated by comparing with six state-of-the-art video diagnosis methods. Highlights: An effective video diagnosis model is used in industry. A novel temporal attention and spatial attention mechanism is proposed. Six state-of-the-art models are tested in our real engineering task. Tricks on how to select attention models for corresponding tasks are detailed in the discussion part.