The classification and localization of crack using lightweight convolutional neural network with CBAM. (15th January 2023)

• Record Type: Journal Article
• Authors: Chen, Liujie; Yao, Haodong; Fu, Jiyang; Tai Ng, Ching
• Is Part Of: Engineering structures. Volume 275(2023)Part B
• Publication Date: 2023-01-15
• Subjects: MobileNetV3-Large -- CBAM (Convolution Block Attention Module) -- Imbalance dataset -- Focal Loss -- Sliding window -- Image classification -- Image detection and localization; Structural engineering -- Periodicals; Structural analysis (Engineering) -- Periodicals; Construction, Technique de la -- Périodiques; Génie parasismique -- Périodiques; Pression du vent -- Périodiques; Earthquake engineering; Structural engineering; Wind-pressure; Periodicals; 624.105
• DOI: 10.1016/j.engstruct.2022.115291
• Languages: English
• ISSNs: 0141-0296

Abstract

Highlights: The combination of MobileNetV3-Large and CBAM is suitable for crack classification with satisfactory accuracy. Focal Loss is employed to solve the imbalance of dataset categories and hard samples. The trained MobileNetV3-Large-CBAM and moving windows are used to classify, locate and identify cracks in three random reconstruction images. The proposed model has satisfactory performance in crack detection and localization. The crack detection is computationally fast and has strong generalization in mobile scenarios with limited calculation conditions. Abstract: Convolutional Neural Networks (CNNs) are currently often used for crack detection. However, the crack datasets collected in real engineering are imbalanced datasets and are affected by interference factors such as different illumination issues and the coexistence of various material crack images. Therefore, the generalization ability of the model itself and the ability to face imbalanced datasets is critical. In addition, a real engineering environment is usually low computational power environment. Therefore, it is undoubtedly more beneficial for the model to have a lightweight feature for practical applications. To address the above challenges in crack detection, MobileNetV3-Large is employed as the backbone combined with CBAM (Convolutional Block Attention Module) to gain MobileNetV3-Large-CBAM in this study. The classification and identification of crack are studied by using the open-source bridge crack dataset. MobileNetV3-Large-CBAM is compared with cutting-edge CNNs, and it verifies that the proposed model combined with the preferred Focal Loss has good performance in dealing with imbalanced datasets and hard samples. To verify the generalization ability of the proposed model, this paper further studies the crack datasets with various material and huge-width cracks under different illumination issues. Finally, the sliding window is adopted to perform crack detection and localization on the three randomly reconstructed crack images. The research results show that, compared with other CNNs, the proposed lightweight MobileNetV3-Large-CBAM combined with the preferred Focal Loss has better comprehensive performance, and the model size is 16.6 MB. I. For imbalanced datasets, the proposed model obtains the best results for crack classification. The Overall Accuracy (OA), F1-score, training speed, and classification speed of MobileNetV3-Large-CBAM are 95.90 %, 95.89 %, 101 images/second and 48 images/second, respectively. The proposed model has a balanced recognition accuracy for different crack categories, and the recognition accuracy for hard samples-irregular crack reaches 94.40 %. II. The proposed model has excellent generalization ability. For two test sets - various material cracks and huge-width cracks under different illumination issues, the OA of MobileNetV3-Large-CBAM reaches 99.66 % and 99.69 %, respectively, the accuracy of crack identification is 99.50 % and 100.00 %, and the accuracy of non-crack identification is 99.90 % and 99.50 %, respectively. III. For crack detection and localization, the model proposed in this paper combined with a sliding window, the accuracy of crack detection for three reconstructed images achieves 100 %, and the average crack localization accuracy achieves 98.40 %.