Development of an unwanted-feature removal system for Structure from Motion of repetitive infrastructure piers using deep learning. (October 2020)

• Record Type: Journal Article
• Authors: Saovana, Natthapol; Yabuki, Nobuyoshi; Fukuda, Tomohiro
• Is Part Of: Advanced engineering informatics. Volume 46(2020)
• Publication Date: 2020-10
• Subjects: Structure from motion -- Unwanted-feature removal -- Deep convolutional neural network -- Point cloud -- Infrastructure pier; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2020.101169
• Languages: English
• ISSNs: 1474-0346

Abstract

Highlights: A new unwanted-feature removal procedure for Structure from Motion (SfM) is proposed. Deep convolutional neural network (DCNN) erases unwanted features from the SfM input. An evaluation of the SfM outputs from original and modified images is conducted. Our system raises the number of cloud points and reduces processing time. Abstract: Structure from Motion (SfM) is a photogrammetric technique that uses similar features between images to construct a point cloud or model. These outputs can be utilized for monitoring and inspection of civil infrastructure. However, feature matching is very delicate and prone to errors that can lead to the failure of the outputs. In our previous research, removal of unwanted features such as vegetation and boulders before processing raised the number of feature matches inside the region of interest (ROI). In theory, raising the number of feature matches should raise the quality of the SfM point cloud, but to date no quantitative evidence has been found to support this. Moreover, the removal of unwanted features must be performed manually during each inspection and is thus labor-intensive and time-consuming. To address these issues, in this study, a deep learning-based system was developed to assist the removal of unwanted features by using a deep convolutional neural network to segment and remove unwanted features from the input images before processing into an SfM application. The results showed that the removal of unwanted features increased the number of cloud points inside the ROI. The proposed system decreased the processing time by 75.33% and 85.85% compared with the manual process in the monorail pier and motorway pier samples, respectively, without complicating the image alignment. The proposed system could potentially support the monitoring and the inspection of infrastructure construction and maintenance projects, which have numerous similar components, by archiving more cloud points inside the ROI and lowering the burden from the manual removal of unwanted features.