A multi-branch deep neural network model for failure prognostics based on multimodal data. (April 2021)

• Record Type: Journal Article
• Authors: Yang, Zhe; Baraldi, Piero; Zio, Enrico
• Is Part Of: Journal of manufacturing systems. Volume 59(2021)
• Page Start: 42
• Page End: 50
• Publication Date: 2021-04
• Subjects: Prognostics -- Multimodal data -- Deep learning -- Multi-branch deep neural network; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2021.01.007
• Languages: English
• ISSNs: 0278-6125

Abstract

Highlights: We use numerical data, images and texts for fault prognostics. The method is able to extract prognostic features from multimodal data. The method is applied to multimodal data obtained from steam generators. The use of multimodal data allows obtaining more accurate predictions. Abstract: Non-numerical data, such as images and inspection records, contain information about industrial system degradation, but they are rarely used for failure prognostic tasks given the difficulty of automatic analysis. In this work, we present a novel method for prognostics using multimodal data, i.e. both numerical and non-numerical data. The proposed method is based on the development of a multi-branch Deep Neural Network (DNN), each branch of which is a neural network designed for processing a certain type of data. The method is applied to a case study properly designed to reproduce the problem of prognostics using multimodal data by referring to the operation of steam generators. The results show that it is able to accurately predict future degradation level using multimodal data, outperforming other methods using fewer sources of information.