Deep learning visual interpretation of structural damage images. (15th November 2022)

• Record Type: Journal Article
• Authors: Gao, Yuqing; Mosalam, Khalid M.
• Is Part Of: Journal of building engineering. Volume 60(2022)
• Publication Date: 2022-11-15
• Subjects: 0000 -- 1111; Deep learning -- Explainable artificial intelligence -- Structural health monitoring -- Guided map analysis -- Visual interpretation; Building -- Periodicals; 690.05
• DOI: 10.1016/j.jobe.2022.105144
• Languages: English
• ISSNs: 2352-7102

Abstract

Abstract: In the past decade, Deep Convolutional Neural Network (DCNN) achieved the state-of-the-art performance in computer vision tasks. However, DCNN is usually treated as a "black box", whose internal working principle is hard to understand. This drawback significantly limits its usage in real-world applications, e.g., vision-based Structural Health Monitoring (SHM), where wrong predictions may lead to catastrophic consequences. To resolve this problem, a framework for the interpretation of the Deep Learning (DL) results called S tructural I mage G uided M ap A nalysis Box (SIGMA-Box or Σ -Box) is proposed. In the Σ -Box, visual interpretation results (saliency maps) are produced and used for model quality evaluation along with human experts' domain knowledge. In this study, the use of the Σ -Box is explored in vision-based SHM applications. Firstly, understanding trained DCNN's performance in concrete cover spalling detection is investigated. Besides, learning procedure at different epochs, learned feature from different network depths, influence of training techniques, and level of semantic abstraction are studied. The experiments demonstrate the good interpretable performance of the Σ -Box which facilitates the understanding of the DCNN models' recognition capabilities, preferences, and limitations. In conclusion, this study sheds light on the high potential of interpreting the trained DCNN in vision-based SHM, providing confidence to the engineers for practical engineering applications involving DL. Highlights: Develop a systematic and human-in-the-loop XAI framework, namely SIGMA-Box. Achieve, by experiments, better understanding of how the DCNN works in vision-based SHM. Increase the confidence of engineers in DCNNs for practical problems using SIGMA-Box.