Deep learning-based sewer defect classification for highly imbalanced dataset. (November 2021)

• Record Type: Journal Article
• Authors: Dang, L. Minh; Kyeong, SeonJae; Li, Yanfen; Wang, Hanxiang; Nguyen, Tan N.; Moon, Hyeonjoon
• Is Part Of: Computers & industrial engineering. Volume 161(2021)
• Publication Date: 2021-11
• Subjects: Sewer network -- Crack classification -- Deep learning -- CCTV -- Text recognition -- Imbalanced data; Engineering -- Data processing -- Periodicals; Industrial engineering -- Periodicals; 620.00285
• DOI: 10.1016/j.cie.2021.107630
• Languages: English
• ISSNs: 0360-8352

Abstract

Graphical abstract: Highlights: A manually collected sewer defect dataset that contains over 38, 000 images. An efficient deep learning-based sewer defect classification framework. Effectively deal with the imbalanced data problem using various approaches. Subtitle recognition that gives more information about detected defects. A novel frame reduction algorithm that significantly reduces the computational time. Abstract: Sanitary sewer systems play a fundamental role in protecting water quality and the public well-being. Structural, civil, and functional operations of any sewer network can deteriorate at accelerated levels due to harsh environments inside the sewer pipes. The existing maintenance procedures are usually deemed inefficient in terms of the assessment accuracy, reliability, safety, and the cost due to the difficulty of detecting and diagnosing defects inside the sewer network. As a result, this paper proposes a robust and efficient deep learning-based framework that can detect and evaluate the defects automatically with high accuracy. The main contributions of the work include (1) a fine-tuned deep learning-based sewer defect detection framework that is based on the block-based architecture, which contains a series of convolutional layers that can efficiently extract the abstract features from the defective regions, (2) hybrid extensions of the proposed model that apply the ensemble-based approach and the cost-sensitive learning-based method in order to cope with the imbalanced data problem (IDP) efficiently, and (3) a novel frame reduction algorithm that is based on analyzing the contextual information of the closed-circuit television (CCTV) videos. The experimental results indicated that the proposed framework obtained a state-of-the-art performance compared to the previous sewer defect detection systems, and it was robust against the IDP. The benefits of the proposed defect detection framework are that it motivates more efficient defect analysis algorithms and promotes a complete integration of deep learning-based approaches in real-world sewer defect analysis applications.