Effect of different imaging modalities on the performance of a CNN: An experimental study on damage segmentation in infrared, visible, and fused images of concrete structures. (December 2022)

• Record Type: Journal Article
• Authors: Pozzer, Sandra; De Souza, Marcos Paulo Vieira; Hena, Bata; Hesam, Setayesh; Rezayiye, Reza Khoshkbary; Rezazadeh Azar, Ehsan; Lopez, Fernando; Maldague, Xavier
• Is Part Of: NDT & E international. Volume 132(2022)
• Publication Date: 2022-12
• Subjects: Infrared thermography -- Infrastructure -- Concrete -- Deep learning -- Multimodal images; Nondestructive testing -- Periodicals; Contrôle non destructif -- Périodiques; Electronic journals; 620.1127
• DOI: 10.1016/j.ndteint.2022.102709
• Languages: English
• ISSNs: 0963-8695

Abstract

Abstract: This study investigates the semantic segmentation of common concrete defects when using different imaging modalities. One pre-trained Convolutional Neural Network (CNN) model was trained via transfer learning and tested to detect concrete defect indications, such as cracks, spalling, and potential subsurface defects. We compared the model's performance using datasets of visible, thermal, and fused images. In addition, the impact of using different image enhancement techniques, such as histogram equalization and resolution improvement, was investigated. The data was collected from four different concrete structures using four infrared cameras with distinct sensitivities and resolutions, with imaging campaigns conducted during autumn, summer, and winter. Although specific defects can be detected in monomodal images, the results demonstrated that a larger number of defect classes could be detected using fused images with the same viewpoint and resolution as the single-sensor image without significant loss of information. In addition, the output of one hypothesis test showed that the image enhancement techniques provided no significant improvement in the CNN performance for this case of study, even though they resulted in enhanced images with higher information content (entropy) than the original images. Highlights: Concrete defects have strengthened appearance in either thermal or visible images. Segmentation of concrete defects is compared in visible, thermal, and fused images. Defects are detected in fused images without significant loss of information. Enhancement methods increased the images' information content but provided no significant improvement in the CNN performance.