A temporal–spatial attention-based action recognition method for intelligent fault diagnosis. (June 2022) - Luo, Wentao; Zhang, Jianfu; Feng, Pingfa; Yu, Dingwen; Wu, Zhijun
Abstract: The intelligent fault diagnosis of video data has become a demanding task in industrial applications. However, existing models require expensive computational cost and memory demand, which makes this technology applied in factories impossible. To address this problem, a temporal–spatial attention-based action recognition method (TARM) integrating TAB (temporal-attention-based frame splitting model), SAB (spatial-attention-based agent focusing mode) and LSB (long-short term feature learning mode) is proposed. TAB first extracts important frames from raw videos. Then, SAB refines video data by reinforcing their essential features and weakening unnecessary features. Furthermore, LSB monitors action type of video data by establishing recurrent convolutional architectures. Finally, the performance of TARM in terms of training time and fault diagnosis accuracy are validated by comparing with six state-of-the-art video diagnosis methods. Highlights: An effective video diagnosis model is used in industry. A novel temporal attention and spatial attention mechanism is proposed. Six state-of-the-art models are tested in our real engineering task. Tricks on how to select attention models for corresponding tasks are detailed in the discussion part.