Automated detection of workers and heavy equipment on construction sites: A convolutional neural network approach. (August 2018)

• Record Type: Journal Article
• Authors: Fang, Weili; Ding, Lieyun; Zhong, Botao; Love, Peter E.D.; Luo, Hanbin
• Is Part Of: Advanced engineering informatics. Volume 37(2018)
• Page Start: 139
• Page End: 149
• Publication Date: 2018-08
• Subjects: Deep learning -- Image -- Improved Faster R-CNN -- Object detection -- Construction site; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2018.05.003
• Languages: English
• ISSNs: 1474-0346

Abstract

Highlights: An automated computer vision-based method using CNN is developed to detect objects on construction sites. The improved Faster R-CNN method is proposed, which achieves a good performance on detection of multi-scale objects. Outperforms comparable results to the current state-of-the-art descriptor methods to detect construction objects on images. A huge construction object database is established, and the robustness and generalization of CNN are demonstrated. Abstract: Detecting the presence of workers, plant, equipment, and materials (i.e. objects) on sites to improve safety and productivity has formed an integral part of computer vision-based research in construction. Such research has tended to focus on the use of computer vision and pattern recognition approaches that are overly reliant on the manual extraction of features and small datasets (<10k images/label), which can limit inter and intra-class variability. As a result, this hinders their ability to accurately detect objects on construction sites and generalization to different datasets. To address this limitation, an Improved Faster Regions with Convolutional Neural Network Features (IFaster R-CNN) approach is used to automatically detect the presence of objects in real-time is developed, which comprises: (1) the establishment dataset of workers and heavy equipment to train the CNN; (2) extraction of feature maps from images using deep model; (3) extraction of a region proposal from feature maps; and (4) object recognition. To validate the model's ability to detect objects in real-time, a specific dataset is established to train the IFaster R-CNN models to detect workers and plant (e.g. excavator). The results reveal that the IFaster R-CNN is able to detect the presence of workers and excavators at a high level of accuracy (91% and 95%). The accuracy of the proposed deep learning method exceeds that of current state-of-the-art descriptor methods in detecting target objects on images.