An enhanced 3D model and generative adversarial network for automated generation of horizontal building mask images and cloudless aerial photographs. (October 2021)

• Record Type: Journal Article
• Authors: Ikeno, Kazunosuke; Fukuda, Tomohiro; Yabuki, Nobuyoshi
• Is Part Of: Advanced engineering informatics. Volume 50(2021)
• Publication Date: 2021-10
• Subjects: Deep learning -- Generative adversarial network -- Semantic segmentation -- Mask image -- Training data -- Urban planning and design; UAV unmanned aerial vehicle -- AI artificial intelligence -- PC personal computer -- VR virtual reality -- GAN generative adversarial network -- 3D three-dimensional -- GPU graphics processing unit -- IoU intersection over union; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2021.101380
• Languages: English
• ISSNs: 1474-0346

Abstract

Graphical abstract: Highlights: Automatic generation method for building mask images and aerial photos is proposed. The method significantly reduced the time required to generate training datasets. Training accuracy was improved by using GAN to remove clouds in aerial photos. The trained model could detect buildings with IoU = 0.651. This method makes it possible to detect buildings in areas with no prior dataset. Abstract: Information extracted from aerial photographs is widely used in the fields of urban planning and design. An effective method for detecting buildings in aerial photographs is to use deep learning to understand the current state of a target region. However, the building mask images used to train the deep learning model must be manually generated in many cases. To overcome this challenge, a method has been proposed for automatically generating mask images by using textured three-dimensional (3D) virtual models with aerial photographs. Some aerial photographs include clouds, which degrade image quality. These clouds can be removed by using a generative adversarial network (GAN), which leads to improvements in training quality. Therefore, the objective of this research was to propose a method for automatically generating building mask images by using 3D virtual models with textured aerial photographs. In this study, using GAN to remove clouds in aerial photographs improved training quality. A model trained on datasets generated by the proposed method was able to detect buildings in aerial photographs with IoU = 0.651.